Unattended intelligent catering service method based on multi-mode perception and AI scheduling

Abstract

The invention provides an unattended intelligent catering service method based on multi-mode perception and AI scheduling, and relates to the technical field of intelligent catering scheduling, and the method comprises the steps: building a catering service state model based on a catering service knowledge graph, carrying out the unified semantic representation of a dining intention, an order demand, equipment resources and environment constraints, and carrying out the unified semantic representation of a catering state model; and forming a catering service scheduling state data set. A scheduling strategy model based on a graph neural network is adopted, feature aggregation and decision making are carried out on intention nodes, resource nodes and process nodes, an artificial intelligence scheduling control instruction set is generated, and automatic cooking, storage and heat preservation, distribution and carrying and terminal delivery equipment are driven to cooperatively execute tasks. Time consumption, energy consumption and abnormal information are collected to form an operation feedback data set, and the operation feedback data set is used for incrementally updating a knowledge graph and optimizing scheduling strategy parameters, so that a subsequent scheduling result is adaptively optimized, and efficient and reliable unattended catering service under the condition of passenger flow and resource configuration change is realized.

Description

Technical Field

[0001] This invention relates to the field of intelligent catering scheduling technology, and in particular to an unattended intelligent catering service method based on multimodal perception and AI scheduling. Background Technology

[0002] Currently, unmanned and intelligent systems are being applied in some scenarios in the catering service industry, such as self-service ordering machines, smart vending machines, unmanned mini-supermarkets, and pre-prepared meal pickup lockers in some airports and office buildings. These systems typically allow ordering and payment via touchscreens or mobile terminals, with the kitchen then preparing the food manually or semi-automatically, followed by delivery via fixed pickup lockers or simple conveyor systems. In more advanced applications, cameras, RFID tags, and voice interaction terminals are added for identity verification, anti-theft monitoring, and basic customer guidance. Simultaneously, restaurants and centralized kitchens are gradually introducing automated devices such as cooking robots, automated food preparation lines, and automated handling equipment to achieve mechanized and information-based management of certain processes.

[0003] However, most of the aforementioned unmanned and intelligent catering solutions adopt a loosely coupled approach between ordering systems, payment systems, kitchen management systems, and some automated equipment. These modules lack unified scene semantic modeling and overall scheduling decisions. The systems typically rely on single types of data, such as order data, RFID results, or localized video streams, for partial control, and have not yet developed comprehensive perception and intelligent collaboration capabilities covering customers, orders, food, equipment, and the environment.

[0004] With the rapid development of artificial intelligence technologies such as computer vision, speech recognition, natural language processing, and reinforcement learning, coupled with the widespread application of various sensors and IoT devices in commercial scenarios, the catering service industry is evolving towards higher perception, higher decision-making, and higher collaboration. On the one hand, the industry hopes to comprehensively utilize visual, voice, RFID, and environmental sensing information in catering scenarios to achieve more refined perception of customer behavior and preferences, food and ingredient status, equipment operation, and the on-site environment. On the other hand, with the continuous enrichment of execution units such as robotic delivery, automated cooking equipment, and intelligent storage and insulation devices, how to coordinate order production rhythm, equipment allocation schemes, and movement path planning under complex resource constraints through a unified artificial intelligence scheduling strategy has become a key issue in improving the unmanned catering experience and operational efficiency.

[0005] In the future development process, more and more catering companies and intelligent equipment suppliers are attempting to build conceptual models such as digital twin kitchens and virtual catering factories, abstracting orders, equipment, inventory, and environmental elements into programmable digital objects, and hoping to use algorithms such as deep learning and reinforcement learning to achieve dynamic scheduling and resource optimization. However, most existing technology implementation solutions still focus on the intelligent improvement of a single link, such as optimizing kitchen scheduling or robot path planning only. Significant shortcomings remain in the overall collaborative control across links and equipment, as well as the real-time scheduling capabilities driven by multimodal perception.

[0006] Existing unmanned and intelligent catering systems suffer from the following main problems. First, most systems still rely primarily on structured order data, supplemented only by a small amount of single-type sensory data, such as single camera monitoring or single RFID results. This lacks a unified semantic representation of customer intent, food status, equipment condition, and environmental conditions, making it difficult to understand real-world business scenarios in a timely and accurate manner. Second, existing scheduling mechanisms largely depend on manually set rules or simple first-come, first-served strategies, focusing on localized optimization around kitchen production scheduling or equipment queuing. They lack a global scheduling framework that models intent, resources, and processes holistically, making it difficult to consistently maintain service quality in high-concurrency, multi-category, and complex equipment combinations in unmanned catering environments. Third, the vast majority of systems have not yet formed a closed-loop self-learning mechanism from multimodal perception and AI decision-making to execution feedback. This prevents the dynamic feedback of equipment malfunction information, execution delays, and customer experience from being continuously optimized back into the scheduling strategy model, resulting in insufficient adaptability when dealing with fluctuations in customer flow, changes in menu combinations, and adjustments to equipment configuration. Summary of the Invention

[0007] To overcome the shortcomings of existing technologies, the purpose of this invention is to provide an unattended intelligent catering service method based on multimodal perception and AI scheduling. This invention solves the problems in existing technologies such as fragmented multimodal perception, lack of unified graph modeling of intent and resource flow, and difficulty in closed-loop self-learning and adaptive optimization.

[0008] To achieve the above objectives, the present invention provides the following solution: A method for unattended intelligent catering services based on multimodal perception and AI scheduling includes: Collect a multimodal scene dataset of the unattended catering service area; wherein, the multimodal scene dataset includes: customer behavior, food and raw material status, equipment operating status and environmental parameters; The multimodal scene dataset is time-synchronized, spatially registered, and cross-modal feature aligned to obtain a scene feature parameter set; A catering service knowledge graph containing intent nodes, resource nodes, and process nodes is constructed based on the scene feature parameter set. The aforementioned catering service knowledge graph is used to perform a unified semantic representation of dining intentions, resource allocation, and service process dependencies in unattended catering services, thereby obtaining a catering service state model. Based on the aforementioned catering service status model, the matching relationship between current order demand and equipment resources is analyzed to obtain a catering service scheduling status dataset; A scheduling strategy model based on graph neural networks is used to evaluate and make decisions on the catering service scheduling status dataset, resulting in an artificial intelligence scheduling control instruction set. Based on the AI ​​scheduling and control instruction set, the automatic cooking equipment, storage and heat preservation equipment, delivery and handling equipment, and terminal delivery equipment are driven to perform corresponding tasks, and the task completion status, time consumption, energy consumption data and equipment abnormal information are collected synchronously to obtain the operation feedback dataset. The intent node attributes, resource node attributes, and process node attributes in the catering service knowledge graph are updated based on the operational feedback dataset, and the parameters of the scheduling strategy model are adjusted to obtain an adaptive scheduling parameter set. The adaptive scheduling parameter set is used to adaptively optimize the subsequently generated artificial intelligence scheduling and control instruction set, so as to obtain an optimized set of unattended intelligent catering service execution results under the conditions of changes in passenger flow and resource allocation.

[0009] The present invention discloses the following technical effects: This invention provides an unattended intelligent catering service method based on multimodal perception and AI scheduling. By introducing multimodal scene data collection and time-space-semantic integrated alignment, this invention constructs a catering service knowledge graph containing intent nodes, resource nodes, and process nodes. Based on this, a catering service state model and a scheduling strategy model based on graph neural networks are established, enabling unified modeling and joint decision-making of order demand, equipment resources, and service processes. Simultaneously, operational feedback data is used to perform closed-loop updates on the knowledge graph attributes and scheduling strategy model parameters, forming an adaptive scheduling parameter set. This allows for adaptive optimization of subsequent scheduling instruction sets, enabling the system to dynamically balance service efficiency, energy consumption, and equipment safety under conditions of fluctuating customer flow and changing resource allocation. This significantly improves order completion rate, resource utilization, and service stability, overcoming the shortcomings of traditional rule-based scheduling, such as poor robustness to environmental changes and difficulty in continuous optimization. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 A flowchart of an unattended intelligent catering service method based on multimodal perception and AI scheduling is provided for an embodiment of the present invention; Figure 2 A flowchart illustrating the construction process of the scene feature parameter set provided in this embodiment of the invention. Detailed Implementation

[0012] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0013] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0014] like Figure 1 As shown, this invention provides a method for unattended intelligent catering services based on multimodal perception and AI scheduling, including: Step 100: Collect a multimodal scene dataset of the unattended catering service area; wherein, the multimodal scene dataset includes: customer behavior, food and raw material status, equipment operating status, and environmental parameters; Step 200: Perform time synchronization, spatial registration, and cross-modal feature alignment on the multimodal scene dataset to obtain a scene feature parameter set; Step 300: Construct a catering service knowledge graph containing intent nodes, resource nodes, and process nodes based on the scene feature parameter set; Step 400: Utilize the aforementioned catering service knowledge graph to perform a unified semantic representation of dining intentions, resource allocation, and service process dependencies in unattended catering services, thereby obtaining a catering service state model; Step 500: Analyze the matching relationship between current order demand and equipment resources based on the catering service status model to obtain a catering service scheduling status dataset; Step 600: Using a scheduling strategy model based on graph neural networks, evaluate and make decisions on the catering service scheduling status dataset to obtain an artificial intelligence scheduling control instruction set; Step 700: Drive the automatic cooking equipment, storage and heat preservation equipment, delivery and handling equipment and terminal delivery equipment to perform corresponding tasks according to the artificial intelligence scheduling and control instruction set, and synchronously collect task completion status, time consumption, energy consumption data and equipment abnormal information to obtain operation feedback dataset; Step 800: Update the intent node attributes, resource node attributes, and process node attributes in the catering service knowledge graph according to the running feedback dataset, and adjust the parameters of the scheduling strategy model to obtain an adaptive scheduling parameter set; Step 900: Adaptively optimize the subsequently generated AI scheduling control instruction set using the adaptive scheduling parameter set to obtain an optimized set of unattended intelligent catering service execution results under the conditions of changes in passenger flow and resource allocation.

[0015] Furthermore, in step 100 of this embodiment, visual acquisition devices, voice interaction devices, radio frequency identification (RFID) devices, equipment status acquisition devices, and environmental sensing devices are first deployed in the unattended catering service area. The installation locations and field of view of each acquisition device are planned and calibrated according to the spatial layout of the dining area, processing area, and delivery area. This embodiment uses visual acquisition devices to continuously acquire video and image data covering dining behavior, food pickup behavior, and processing operations to reflect customer behavior characteristics such as staying, queuing, food pickup, and interaction with equipment in different areas. Voice interaction devices collect customer ordering voices, inquiry voices, and feedback voices in real time. In this embodiment, the voice data is synchronously recorded with corresponding timestamps and interactive terminal identifiers for subsequent multi-source mapping with visual and RFID data.

[0016] Building upon this foundation, this embodiment utilizes RFID equipment to periodically poll and perform event-triggered reading and writing of food labels, ingredient labels, and tableware labels. This allows for the acquisition of unique identifiers, inventory quantities, entry and exit times, and flow status information for each food item and ingredient across dining, processing, and storage areas. Simultaneously, an equipment status acquisition device obtains operational status parameters such as operating modes, start / stop status, fault alarms, and current load for automated cooking equipment, storage and warming equipment, delivery and handling equipment, and terminal delivery equipment. Furthermore, environmental sensors collect environmental parameters such as temperature, humidity, illuminance, and air quality within the unattended catering service area. This embodiment achieves synchronous perception of equipment operating status and environmental status by attaching precise timestamps and pre-configured spatial location identifiers to various operational and environmental parameters on the equipment side.

[0017] Finally, this embodiment performs unified data preprocessing and alignment on the aforementioned subsets of visual scene data, voice interaction data, RFID data, and device and environment data: on the one hand, timestamps from different data sources are calibrated and resampled according to a unified time base to ensure aggregation into the same multimodal event window within the same time slice; on the other hand, based on a spatial layout model, device identifiers, terminal identifiers, and spatial location identifiers carried in each data source are mapped to unified spatial coordinates or region numbers, and multi-source data is associated and integrated accordingly. Through the above-mentioned dual temporal and spatial association, this embodiment organizes customer behavior, food and ingredient status, equipment operating status, and environmental parameters into a structured multimodal scene dataset, providing a complete and consistent raw data foundation for subsequent feature extraction, knowledge graph construction, and intelligent scheduling.

[0018] Furthermore, such as Figure 2 As shown, in step 200 of this embodiment, a timeline is first constructed based on a preset unified time reference for the visual scene data subset, voice interaction data subset, RFID data subset, and device and environment data subset obtained in step 100. The original timestamps of each data subset are then calibrated and quantized. This embodiment performs time alignment and resampling on data with different sampling frequencies, uniformly mapping the visual frame timestamps, voice segment start and end times, RFID event times, and device status and environmental parameter sampling times onto discrete time slices. Within each time slice, events and observations from different data sources are aggregated to form a time-synchronized multimodal data set, ensuring subsequent joint analysis of customer behavior, device status, and environmental changes within the same time window.

[0019] After time synchronization is completed, this embodiment utilizes a pre-constructed spatial layout model of the unattended catering service area to perform spatial coordinate mapping and area calibration on the spatial information such as camera identifiers, RFID reader identifiers, equipment identifiers, and sensor location identifiers carried in the time-synchronized multimodal data set. This embodiment maps various identifiers to unified spatial coordinates or area numbers, assigning data from different acquisition devices to spatial units such as dining areas, processing areas, storage areas, and delivery areas. The spatial adjacency relationships and area type labels corresponding to the data are calculated in the spatial layout model, resulting in a spatially registered multimodal data set. This ensures that each multimodal observation possesses both time-slice attributes and a clear spatial location and area semantics.

[0020] Based on time synchronization and spatial registration, this embodiment extracts feature vectors representing customer behavior patterns, food and ingredient status, equipment operating conditions, and environmental constraints from spatially registered visual scene data, voice interaction data, RFID data, and equipment and environment data, respectively. These feature vectors are then organized into a multimodal feature vector set. This embodiment further performs feature dimension transformation and semantic alignment on the multimodal feature vector set, mapping features from different modalities to a unified feature space. This makes behavioral features, resource status features, and environmental features related to the same dining scenario within the same time slice and spatial region semantically comparable and fusionable, thereby forming a scene feature parameter set that can be directly used for subsequent knowledge graph construction and state modeling.

[0021] Specifically, in this embodiment, the spatial layout model is used to discretize and structurally model the physical space within the unattended catering service area to support subsequent spatial registration and regional-level analysis of multimodal data. Specifically, this embodiment first divides the dining area, processing area, storage and heat preservation area, delivery and handling channel, and terminal delivery area into several non-overlapping or partially overlapping spatial units based on the actual floor plan layout of the catering service scenario. A spatial node is set for each spatial unit, and this spatial node records the geometric position of the spatial unit in a planar coordinate system or a three-dimensional coordinate system. For example, the coordinate values ​​obtained from actual measurements are used to represent the physical position of each spatial node. This embodiment also configures a spatial attribute vector for each spatial node. The spatial attribute vector includes information such as area type, the ability to deploy equipment and resources, customer access priority, and safety constraints. For example, the area type can be a discrete category label such as dining area, processing area, storage area, equipment maintenance area, etc. The deployment capability can indicate whether the location allows the deployment of automatic cooking equipment, storage equipment, or only environmental sensors. The access priority can be set to different priority values ​​according to the attributes of the location in the main passage, secondary passage, or non-access area. The safety constraints can reflect the safety distance requirements between the location and fire exits, emergency exits, or high-temperature equipment.

[0022] Based on the establishment of spatial nodes and their attributes, this embodiment describes the spatial association strength between any two spatial nodes by constructing spatial metric relationships and adjacency strength functions. Specifically, this embodiment first transforms the physical coordinates recorded by each spatial node into a unified layout reference coordinate system through a preset linear mapping. This linear mapping can be determined based on actual on-site survey results and design drawings, ensuring that data from different coordinate sources are within the same reference frame after mapping. Then, this embodiment calculates the spatial distance between two spatial nodes based on their positional differences in this reference coordinate system. The spatial distance value comes from the geometric distance between two points and is used to reflect their degree of proximity or distance from each other in planar or three-dimensional space. After obtaining the spatial distance, this embodiment uses a function containing a decay factor to map shorter spatial distances to larger adjacency strengths and longer spatial distances to smaller adjacency strengths. Simultaneously, a maximum neighborhood radius is set during the calculation process. When the distance between two spatial nodes exceeds this radius, their adjacency strength is considered zero, and they are no longer considered directly adjacent in space. Through the above method, this embodiment obtains a spatial adjacency strength function. The output of this function can be used to construct a spatial adjacency matrix to help identify spatially adjacent regional units, such as adjacent dining tables, processing workbenches and side passages, thereby playing a constraining role in the spatial aggregation and path planning of multimodal data.

[0023] In this embodiment, the size of the spatial node set is determined by the number of spatial units divided on-site. It can be configured according to the area and accuracy requirements of the unattended catering service area. For example, in a large dining area, spatial units can be divided according to tables or table groups, while in narrow corridors and passageways, continuous spatial nodes can be divided according to a certain step size. The coordinate values ​​of each spatial node are derived from actual on-site measurements, architectural floor plans, or 3D modeling results, and are used to accurately indicate the relative positional relationships of the data acquisition equipment, cooking equipment, and passageways. Parameters such as area type, deployment capacity, passage priority, and safety constraints in the spatial attribute vector can be pre-configured through site planning and design and operational specifications. For example, the passage priority of the main passageway can be set to a higher value, and stricter safety constraints can be set for areas near high-temperature or high-pressure equipment to limit the deployment of equipment and the stay of personnel in that area. The calculation of spatial distance relies on the above coordinate values ​​and does not require additional manual adjustment; the attenuation factor in the adjacency strength can be selected based on site scale experience to control the speed at which distance affects the adjacency relationship. For example, in scenarios with narrow passages, a parameter with a slower attenuation rate can be selected so that a relatively longer spatial distance is still considered to have a certain degree of connectivity; the maximum neighborhood radius depends on the maximum physical distance that you want to consider to be adjacent to each other, such as the center distance between two adjacent rows of tables or the length of a short passage.

[0024] More specifically, in this embodiment, taking an unattended catering service area of ​​approximately 200 square meters as an example, the area is divided into 20 spatial nodes, with the total number of spatial nodes N being 20. The first node z1 in the set of spatial nodes Z corresponds to the center position of a four-person table in the dining area, and its spatial coordinate vector can be set as (2, 5). The second node z2 corresponds to the center position of the adjacent table, and its spatial coordinate vector is set as (4, 5). At this time, under the unified layout reference frame, the spatial metric distance d(z1, z2) between these two nodes can be calculated as 2. When only considering local neighborhoods with a distance of no more than 5, the distance threshold D is set as follows. max Let it be 5. Since 2 is less than 5, the indicator function 1(d(z1, z2) ≤ D) max The value of ) is 1. If the spatial distance attenuation factor α is set to 0.5, the value of the adjacency strength function Γ(z1, z2) can be expressed as a positive number between 0 and 1, calculated using the distance and attenuation factor, for example, about 0.37, used to characterize the spatial proximity between two tables. Regarding the setting of the linear mapping matrix P, this embodiment assumes that the unit of the original physical plan is centimeters, while the unit of the layout reference coordinate system is meters. Therefore, P can correspond to a transformation matrix that scales the coordinates by a ratio of 0.01, such that physical coordinates (200, 500) are converted to layout reference coordinates (2, 5). The 2-norm symbol ‖·‖2 is used here to calculate the length of this type of coordinate difference vector. For the spatial attribute set A, in this embodiment, the attribute vector a1 of the first node consists of four components: area type identifier c1, deployment capability parameter λ1, passage priority parameter ρ1, and safety constraint parameter η1. Here, c1 can be 1 to represent a dining area, λ1 can be 3 to represent that the location allows the deployment of 1 ordering terminal and 2 environmental sensors, ρ1 can be 2 to represent that the location is in a secondary passage area rather than the main passage, and η1 can be 5 to represent that a safety distance of at least 1.5 is required from fire lanes. For the second node, the area type identifier c2 can also be 1, the deployment capability parameter λ2 can be 2 to represent that only 2 environmental sensors are allowed to be deployed and no additional large equipment is allowed, the passage priority parameter ρ2 can be 1 to represent proximity to the main passage, and the safety constraint parameter η2 can be 4 to represent that a safety distance of at least 1.0 is required from high-temperature equipment. By setting the specific values ​​mentioned above, the spatial layout model Mspace in this embodiment can fully characterize the relative positional relationships and availability constraints between various functional areas within the unattended catering service area using discrete nodes and their spatial attributes, spatial distances, and adjacency strength functions, providing a quantifiable basis for subsequent multimodal data registration and path planning.

[0025] Furthermore, in step 300 of this embodiment, the scene feature parameter set obtained in step 200 is first used to aggregate and parse multimodal features within the same time slice and spatial area to identify customer dining needs, taste preferences, dining time constraints, and number of diners. Specifically, this embodiment performs joint analysis on the number of people queuing, seated, and picking up food extracted from visual features, and the order content, taste description, and dining time requirements parsed from voice interaction data. It aggregates the temporally continuous and spatially adjacent behavioral features of the same customer or the same order to obtain a structured dining need record, from which semantic information such as the customer's desired food type, taste category, dining start time, acceptable waiting time limit, and number of companions is extracted. This embodiment abstracts each dining need into an intent node and stores taste preferences, dining time constraints, and number of diners as attribute fields of that intent node, thereby forming an intent node set containing multiple intent nodes, enabling subsequent graph-structured representation of service needs for different customers or different orders.

[0026] After constructing the intent nodes, this embodiment utilizes a scene feature parameter set to uniformly extract and summarize the availability, processing capacity, capacity limits, and spatial location information of meals, raw materials, automatic cooking equipment, storage and heat preservation equipment, delivery and handling equipment, and terminal delivery equipment within the unattended catering service area. Specifically, this embodiment obtains the current inventory quantity, batch information, and location of each meal and raw material from RFID features; obtains the current operating mode, supported dish types, maximum processing quantity per unit time, and capacity limits and temperature control capabilities of each automatic cooking equipment from equipment status features; and obtains the spatial node number and area type of each equipment and material point from the spatial layout model. Based on the above information, this embodiment abstracts each available meal, each raw material, and various types of equipment into resource nodes, marks the availability status as available, busy, or faulty, records the processing capacity and capacity limits in numerical form, and records the spatial location in the form of corresponding spatial node numbers or coordinate indices, constituting the attribute fields of the resource nodes, thereby forming a resource node set, providing a detailed resource description for subsequent resource scheduling and occupancy relationship modeling.

[0027] Based on this, this embodiment, according to the business processes of order generation, food preparation, storage and heat preservation, delivery and handling, and final delivery in unattended intelligent catering services, decomposes the above business links into several process units with clear input-output relationships, and abstracts each process unit as a process node, such as order generation node, food preparation node, cooking node, plating node, heat preservation and storage node, conveying node, and food pickup and delivery node. Combining the records of time sequence and resource consumption in the scene feature parameter set, this embodiment establishes sequential and parallel relationships within the set of process nodes. For example, within the same order, the food preparation node must be constrained to execute before the cooking node, while the food preparation nodes of multiple orders can be executed in parallel if resources permit. Simultaneously, this embodiment establishes intent association edges based on the matching relationship between each intent node and resource nodes that can meet the corresponding dining needs, and records the service priority, candidate resource set, and currently adopted service strategy in the edge attributes; it establishes resource occupation edges based on the occupation relationship of each process node on food, raw materials, and equipment during execution, and records the start time, end time, occupation method, and occupation intensity in the edge attributes; it establishes process affiliation edges based on which order or group of customer needs each process node serves, and records the order identifier, task identifier, and current service status in the edge attributes. Finally, this embodiment organizes the intent node set, resource node set, process node set, and the above three types of edges into a unified graph structure to obtain a catering service knowledge graph containing intent nodes, resource nodes, and process nodes, providing a structured knowledge foundation for subsequent state modeling and intelligent scheduling strategy generation.

[0028] Furthermore, in step 400, the catering service status model is used to perform unified semantic fusion and quantitative expression of the graph structure service status, order and dining demand status, equipment and resource configuration status, and environmental constraint status in the unattended catering service scenario, so as to provide calculable state input for subsequent scheduling strategy generation. Specifically, in this embodiment, at each discrete moment, the graph structure service status obtained based on the catering service knowledge graph, the set of valid orders and dining demands at that moment, the set of currently available equipment and resource configurations, and the set of environmental constraints and environmental parameters are input into the state mapping module. The state mapping module maps the above four types of information into a state vector containing three components through a series of feature encoding and nonlinear combination operations. The first component of the state vector is used to continuously characterize the overall service load and pressure level, the second component is used to characterize the degree of satisfaction of various constraints, and the third component is used to represent the net sufficiency of current resources, thereby reflecting the comprehensive operating status of the unattended catering service at that moment in a unified numerical form. In this embodiment, the state mapping function is a custom term used to refer to the calculation rule for converting multi-source state inputs into state vectors. Its function is to jointly encode graph structure features, demand features, resource features and environmental features and output a fixed-dimensional state representation.

[0029] In the implementation process, this embodiment first extracts a graph embedding vector for the graph structure service status obtained based on the catering service knowledge graph through a graph feature aggregation and encoding module. This graph embedding vector can be understood as a high-dimensional feature representation obtained after considering intent nodes, resource nodes, process nodes, and three types of edge relationships. For example, in a specific embodiment, the graph embedding vector can be set as a real number vector of length 64, and its components come from multi-layer aggregation of node features and edge features. Then, this embodiment extracts an order demand feature vector from the order and dining demand set. This feature vector may include information such as the number of currently uncompleted orders, the demand quantity counted by dish type, the demand distribution counted by flavor type, and the remaining waiting time for each order. For example, a real-number vector of length 16 can be constructed, where the first four components represent the demand quantity of different dish categories, the middle components represent the number of orders due within different time windows, and the last few components represent the proportion of high-priority orders, etc. Similarly, this embodiment extracts a resource supply feature vector from the set of equipment and resource configurations. For example, a real-number vector of length 16 can be constructed, where some components represent the number of idle units of various automatic cooking equipment, some components represent the remaining capacity of storage and heat preservation equipment, and some components represent the number of available vehicles or track trolleys for delivery and handling equipment. An environmental feature vector is extracted from the set of environmental constraints and environmental parameters. For example, a real-number vector of length 8 can be constructed, which includes the current temperature, humidity, average illuminance, air quality index, and whether there are environmental alarm signs exceeding the safety threshold, etc. The above four vectors serve as input features of the state model. In this embodiment, they are linearly transformed by linear transformation matrices that match their dimensions. The values ​​of these linear transformation matrices are automatically learned from historical operating data during the model training phase and remain fixed during the actual deployment phase, used to project the original features of each modality onto a unified feature space.

[0030] After the linear transformation, this embodiment inputs the linear combination result of graph embedding features, order demand features, resource supply features, and environmental features into a nonlinear activation function to obtain continuous state components representing the joint service load state at that moment. The nonlinear activation function is a common neural network activation operator. In this embodiment, a function that compresses the input to the interval between 0 and 1 can be selected. For example, a function with threshold and slope adjustment parameters can be used. Its function is to smoothly compress excessively large linear combination results while enhancing the ability to express the differences in features within the intermediate interval. To characterize the coupling relationship between demand and resources, and demand and environment, this embodiment also calculates the inner product between order demand features and resource supply features, as well as the inner product between order demand features and environmental features. These two inner product values ​​are then input into a combination function. The combination function refers to a computational module that weights and combines several scalar or vector features to output new semantic features. Its function is to numerically reflect the degree of matching between the current demand structure, resource allocation structure, and environmental conditions. For example, when order demand is highly concentrated on a certain cooking method and the corresponding equipment is largely idle, the output of the combination function can correspond to a higher matching score; when the ambient temperature is abnormally high and a large number of orders require high-temperature cooking, the combination function can also output a lower constraint satisfaction level. In this embodiment, the output of the combination function serves as the second component, representing the comprehensive satisfaction level of environmental constraints, resource constraints, and service quality constraints at that moment.

[0031] To measure the overall difference between resource supply and order demand, this embodiment constructs a column vector with all components equal to 1, performs vector summation on the resource supply feature vector and the order demand feature vector respectively, and obtains the total resource supply and total order demand at the current moment. The difference between the two is used as the net resource scalar, and the result less than 0 is truncated to 0 through a half-threshold operation, retaining only the non-negative scalar component, to obtain the third state component representing the resource scalar at this moment.

[0032] More specifically, in this embodiment, taking a specific moment as an example, the catering service state vector can be set as a vector of length 10 containing three parts: continuous service load component, constraint satisfaction component, and resource margin component. For example, the values ​​are "0.75, 0.60, 0.20, and the other 7 components are 0". Here, 0.75 indicates that the current overall service load is at a pressure level of 75%, 0.60 indicates that the comprehensive satisfaction of various time constraints, safety constraints, and service quality constraints is 60%, and 0.20 indicates that there is still about 20% resource margin.

[0033] The state mapping function receives four types of inputs at this moment: First, the graph structure service state, for example, a graph embedding vector of length 64 obtained by aggregating and encoding graph features of the catering service knowledge graph, where each component fluctuates between 0 and 1; Second, the order demand feature vector extracted from the order and dining demand set, for example, of length 16, where the first 4 components can be the number of unfulfilled orders for the current four main types of dishes "20, 15, 10, 5", the middle components are the number of orders due in different time windows "8, 6, 4", and the remaining components are the proportion of high-priority orders "0.30, 0.20", etc.; Third, the resource supply extracted from the equipment and resource allocation set. The feature vector, for example, has a length of 16, and each component can correspond to the number of idle automatic cooking equipment "5, 3, 2", the remaining capacity of storage and heat preservation equipment "40, 30", the number of available delivery and handling vehicles "4, 2", and the standard service unit capacity that various types of equipment can currently provide "50, 40, 30", etc.; the fourth is the environmental feature vector extracted from the environmental constraints and environmental parameter set, for example, has a length of 8, and includes the current ambient temperature "26", humidity "55", average illuminance "300", air quality index "20", whether there is a high temperature alarm "0", whether there is a smoke alarm "0", environmental noise level "45", and comprehensive environmental constraint score "0.85".

[0034] For the four vectors mentioned above, this embodiment configures a set of linear transformation matrices for each type of feature. After training, these matrices compress the graph embedding features into an intermediate representation of length 32, and map the order demand features, resource supply features, and environmental features into intermediate representations of length 16. These intermediate representations are then weighted and summed and input into a nonlinear activation function, for example, using a compression function with an output range of 0 to 1, resulting in a final continuous service load component of approximately 0.75. This is to describe the coupling relationship between demand and resources, and demand and environment; This embodiment calculates the inner product of the order demand feature vector and the resource supply feature vector, resulting in a scalar of approximately 1800 in this example. Calculating the inner product of the order demand feature vector and the environmental feature vector yields a scalar of approximately 900. These two scalars are then input into a combination function. Through normalization and weighting, the constraint satisfaction component of 0.60 is output, reflecting the degree of matching between the current resource allocation and environmental conditions and the demand structure. Further, this embodiment obtains the total resource supply by summing all components of the resource supply feature vector, for example, 200 standard service units. It also obtains the total order demand by summing all components of the order demand feature vector, for example, 160 standard service units. The difference between the two is 40. After applying the negative value truncation operation, the result remains 40. Dividing this by an upper limit of 200 used for normalization yields the resource margin component of 0.20, thus providing a complete picture of the net margin between resource supply and order demand at that moment.

[0035] Furthermore, in step 500 of this embodiment, based on the catering service status model constructed in step 400 and the feature parameters related to order information and customer dining needs in the scene feature parameter set, the current valid orders and their corresponding intent node information are extracted. In specific implementation, orders that are in an incomplete state in the catering service knowledge graph and have not yet exceeded the dining time constraint are marked as current valid orders. Each order is associated with its corresponding dining intent through the intent node index, and key fields such as dining time constraint, service priority, number of diners, and taste preferences are read from the intent node attributes to form a subset of order demand status. For example, for 10 valid orders at the same time, this embodiment records structured information such as order identifier, expected dining start time, acceptable waiting time, priority level, and whether it is a special dietary need for each order, so that the subset of order demand status can fully reflect the service demand intensity and time constraints at the current order level.

[0036] After obtaining the subset of order demand states, this embodiment extracts the availability, processing capacity, remaining capacity, and spatial location information of each resource node from the catering service state model based on the features related to the status of food and raw materials and the operating status of equipment in the scenario feature parameter set, thus constructing a subset of equipment and resource states. Specifically, for each resource node, this embodiment reads its current availability status marker (e.g., idle, occupied, faulty and awaiting repair), the number of food items that can be processed or the number of processes that can be completed per unit time, the current remaining storage or heat preservation capacity, and the spatial node number or coordinate location of the resource, and organizes the above information into a resource state record in a unified format. For example, this embodiment can record the availability, capacity limit, and current load of 5 automatic cooking devices, 3 heat preservation cabinets, 2 delivery tracks, and several key raw material storage points, so as to infer which resources may be occupied and how long the process may last from the time the order enters the system to the time of delivery based on the spatial location and processing capacity. Through this process, this embodiment obtains a complete subset of equipment and resource states, providing data support for subsequent order and resource matching calculations.

[0037] After constructing the order demand state subset and the equipment and resource state subset, this embodiment utilizes the pre-established relationships between intent nodes, resource nodes, and process nodes in the catering service state model to quantitatively analyze the matching relationship between current order demand and equipment resources. Specifically, this embodiment first enumerates candidate resource sequences that can serve each order from the knowledge graph based on the business path defined by the process nodes. For example, a certain stir-fry order can be completed by a specific food preparation station, a wok with the corresponding power and cuisine capability requirements, an adjacent plating station, and the nearest warming cabinet and delivery terminal. Then, based on the dining time constraints and order priority in the order demand state subset, combined with the current availability and processing capacity in the equipment and resource state subset, this embodiment calculates the estimated completion time under the candidate resource combination by estimating the queuing time and processing time of each process, and comprehensively evaluates the resource occupation cost based on indicators such as equipment operating cost, resource occupation time, and resource switching frequency to obtain the initial matching result set of orders and resources.

[0038] Building upon this foundation, this embodiment introduces environmental parameter constraints, such as kitchen ambient temperature, load limits, or noise restrictions during specific time periods. Matches in the initial matching result set that do not meet dining time constraints or violate environmental safety constraints are eliminated or adjusted. Furthermore, when multiple feasible matching schemes exist for the same order, priority is given to retaining the schemes that can be completed within the specified time and have lower resource consumption costs, forming a set of order and resource constraint matching results. Finally, this embodiment organizes the order demand status subset, the equipment and resource status subset, and the order and resource constraint matching result set according to a unified data structure. For example, it uses a representation method with the order as the primary key and resource allocation scheme and estimated completion time as core fields to construct a catering service scheduling status dataset. This dataset provides comprehensive and computable status input for subsequent automatic scheduling, dynamic rescheduling, and anomaly recovery decisions.

[0039] Furthermore, in step 600 of this embodiment, the catering service knowledge graph and the catering service scheduling status dataset obtained in step 500 are first used to annotate the intent nodes, resource nodes, and process nodes in the knowledge graph one by one, forming a graph structure scheduling status dataset. Specifically, this embodiment writes fields related to order demand status in the scheduling status dataset, such as whether the order is pending scheduling, whether it has been partially executed, the remaining waiting time, and the order service priority, into the node features of the corresponding intent node; writes fields related to equipment and resource status, such as the current availability of resources, estimated idle time, processing capacity per unit time, remaining capacity, and spatial location, into the node features of the corresponding resource node; and writes fields related to the order and resource constraint matching results, such as the candidate resource combination currently selected for a certain order, the estimated completion time, and the resource occupation cost, into the edge features of the corresponding process node or the edges between nodes. Through the above method, this embodiment enables each node and edge to carry a numerical or discrete label reflecting the current scheduling status, thereby embedding the scheduling status information into the graph structure without changing the graph topology, thus forming a graph structure scheduling status dataset for subsequent graph neural network evaluation.

[0040] After obtaining the graph-structured scheduling state dataset, this embodiment inputs the dataset into a pre-trained graph neural network-based scheduling strategy model. Multi-layer graph convolution is used to aggregate features and semantically encode each intent node, resource node, and process node, resulting in a graph-structured scheduling feature representation set. Specifically, in the first layer of graph convolution, the initial feature vector of each node is used as input. Through weighted aggregation of features from its first-order neighbor nodes, an intermediate node representation containing local neighborhood information is obtained. For example, after aggregating features from connected process nodes and resource nodes, an intent node can explicitly obtain information such as "the current resource capacity level that can serve the order" and "the congestion level of the process stage where the order is located." In the second and higher layers of graph convolution, this embodiment further propagates and aggregates features within a larger neighborhood, enabling resource nodes to perceive the potential competition from multiple orders, and process nodes to perceive the load pressure and critical path constraints of preceding and following stages. After several layers of graph convolution operations, this embodiment obtains a set of high-dimensional scheduling feature representations for each intent node, resource node, and process node. These feature representations constitute a graph structure scheduling feature representation set, which can be used to score and rank various optional scheduling actions in the subsequent decision output layer.

[0041] Based on the graph structure scheduling feature representation set, this embodiment utilizes the pre-set decision output layer in the scheduling strategy model to evaluate the optional scheduling actions formed by the combination of each intent node and candidate resource node and generate an artificial intelligence scheduling control instruction set. Specifically, this embodiment first constructs an optional scheduling action for each "intent node-candidate resource node-process stage" triple according to the graph structure scheduling feature representation, such as "assigning order A to cooking equipment 1 at the current time to perform the stir-frying stage" and "assigning order B to delivery track 2 in the next time slice to perform the delivery stage," etc., and calculates a scheduling priority score for each optional scheduling action through the decision output layer. This score comprehensively considers factors such as order service priority, estimated completion time, resource occupation cost, and impact on overall load balancing. Subsequently, in this embodiment, the scheduling actions that violate the constraints are filtered from the candidate scheduling action set, taking into account dining time constraints, service priorities, equipment safety operation constraints, and environmental parameter constraints. For example, actions that would cause the order to exceed the latest completion time, actions that would cause the equipment to exceed the safe load, or actions that would trigger high-risk processes under environmental alarm conditions are removed. The remaining scheduling actions are then sorted according to scheduling priority to generate a sequence of scheduling actions that meet the constraints, thus obtaining a scheduling decision result set.

[0042] Finally, based on the scheduling decision result set, this embodiment automatically maps each scheduling action into specific equipment control instructions, task allocation parameters, and execution timing information. For example, it generates control instruction entries including target equipment identifier, start time, execution recipe or process parameters, expected end time, and synchronization relationship with preceding and following process links. These instructions are then sent to automatic cooking equipment, storage and heat preservation equipment, delivery and handling equipment, and terminal delivery equipment, thereby generating an artificial intelligence scheduling and control instruction set to achieve intelligent scheduling and automatic control of the entire unattended catering service process.

[0043] Furthermore, in step 700 of this embodiment, each scheduling action is first sent to the corresponding automatic cooking equipment, storage and heat preservation equipment, delivery and handling equipment, and terminal delivery equipment based on the artificial intelligence scheduling and control instruction set obtained in the previous step. Specifically, in this embodiment, each instruction in the scheduling and control instruction set clearly specifies the target equipment identifier, task type, key process or operating parameters, and planned start time and expected end time. For example, a control instruction is sent to an automatic wok equipment to "start at a certain time, execute a certain recipe, set the heating power to a certain level, and set the stirring frequency to a certain value"; a control instruction is sent to a heat preservation cabinet to "receive a certain order of food at a certain time and maintain the temperature within a certain range"; and a control instruction is sent to a delivery cart to "collect food from a certain workstation within a certain time window and deliver it to a certain terminal along a certain path". After receiving the corresponding instructions, each type of equipment completes the food preparation, storage and heat preservation, delivery and handling, and terminal delivery tasks sequentially according to the execution sequence and parameters given in the instructions, thereby implementing the artificial intelligence scheduling strategy at the physical level.

[0044] Throughout the entire process of tasks performed by various automatic cooking equipment, storage and heat preservation equipment, delivery and handling equipment, and terminal delivery equipment, this embodiment simultaneously collects operational status data and meticulously records the execution process corresponding to each scheduled action. Specifically, this embodiment records the task start timestamp at the start of the task and the task end timestamp at the completion or termination of the task, and calculates the time consumption accordingly; at the same time, it collects task completion status information to distinguish different states such as normal completion, timeout completion, interruption, or failure; regarding energy consumption data, this embodiment obtains the cumulative energy consumption value and instantaneous power curve during task execution from the built-in power metering module or energy consumption monitoring unit of the equipment, and assigns them according to task granularity; regarding equipment anomaly information, this embodiment connects to the fault alarm interface of various equipment to record whether abnormal events such as abnormal temperature, motion stagnation, communication interruption, and power fluctuation occur during execution, as well as the time, duration, and alarm code of the anomaly. Through the above process, this embodiment generates a task execution process data record for each scheduling action, including task start time, task end time, task completion status, time consumption, energy consumption data, and equipment abnormality information.

[0045] After collecting data during the task execution process, this embodiment structures and organizes the raw operational data, establishing a one-to-one correspondence with the corresponding scheduling actions to form an operational feedback dataset. Specifically, this embodiment maps the time, energy consumption, completion status, and exception record fields in the task execution data to a unified data structure according to the scheduling action identifiers in the scheduling control instruction set. For example, for each scheduling action, a feedback record is created containing fields such as "scheduling action identifier, target device identifier, order or task identifier, actual start time, actual end time, actual time consumption, actual energy consumption, completion status, exception type, and exception description." For multiple scheduling actions involved in the same order, this embodiment further supports aggregating operational feedback information by order dimension to evaluate the execution performance of the entire process from order preparation to delivery. Through the above structured organization and identifier association, the operational feedback dataset obtained in this embodiment can be directly used for subsequent evaluation of the scheduling strategy's effectiveness, analysis of equipment operational health status, and online updates and iterations of the catering service status model and scheduling strategy model, providing a complete and traceable operational basis for the continuous optimization of the unattended catering service system.

[0046] Furthermore, in step 800 of this embodiment, based on the operational feedback dataset obtained in step 700, the intent nodes, resource nodes, and process nodes in the catering service knowledge graph are first associated with feedback node by node to form a node-level operational feedback subset. Specifically, this embodiment matches each record in the operational feedback dataset that contains order completion status, time consumption, energy consumption data, and equipment anomaly information with the corresponding scheduling action identifier, order identifier, and equipment identifier in the scheduling phase: For order-level feedback data, such as whether an order was completed on time, whether a user complaint or order cancellation occurred, this embodiment associates it with the corresponding intent node; for equipment-level feedback data, such as the actual duration, actual energy consumption, and abnormal fault information of an automatic cooking device in a task, this embodiment associates it with the corresponding resource node; for process-level feedback data, such as the average delay, bottleneck level, and timeout probability of a certain food preparation, cooking, or delivery stage, this embodiment associates it with the corresponding process node. Through the above mapping relationship, this embodiment generates a node-level runtime feedback subset containing multiple historical task records for each node in the knowledge graph, providing a refined data foundation for subsequent attribute statistical analysis and incremental updates.

[0047] After obtaining the node-level operational feedback subset, this embodiment performs statistical analysis and incremental updates on the intent node attributes, resource node attributes, and process node attributes in the catering service knowledge graph to construct a knowledge representation that better reflects actual operational conditions. Specifically, at the intent node level, this embodiment updates the actual average waiting time, actual completion time distribution, and timeout probability for different types of dining needs based on multiple order completion statuses and time consumption. The original dining time constraint is adjusted from a static empirical value to a statistical quantity combined with historical data. For example, for weekday lunch peak short-time fast food orders, the average completion time is updated to 8 minutes and the timeout probability is 5%. At the resource node level, this embodiment updates the actual processing capacity, unit task energy consumption, and failure rate of each device under different loads based on the energy consumption and anomaly information recorded in the operation feedback data. For example, the average processing time of a certain automatic cooking device under standard dish tasks is corrected to 210 seconds, the average energy consumption is corrected to 0.25 kWh, and the failure rate in high-temperature mode is recorded as 2% for risk assessment during subsequent scheduling. At the process node level, this embodiment statistically analyzes the actual execution time distribution, bottleneck degree in the overall process, and common anomaly types of each business link based on feedback from multiple batches of tasks, and writes these statistical results back to the attribute fields of the corresponding process nodes. Through the above incremental update operations, this embodiment obtains an updated catering service knowledge graph, which can dynamically reflect the real operating characteristics of the unattended catering system under different time periods and different load levels.

[0048] After updating the knowledge graph, this embodiment further utilizes the runtime feedback dataset and the set of AI scheduling control instructions used in the scheduling process to construct a scheduling sample set. Based on this scheduling sample set, the parameter optimization training of the graph neural network-based scheduling strategy model is performed to generate an adaptive scheduling parameter set. Specifically, this embodiment first pairs each scheduling control instruction with its corresponding runtime result to form a supervision sample with "input as the graph structure scheduling status and executed scheduling actions, and output as indicators such as actual completion time, actual energy consumption, and whether constraints are met." On this basis, this embodiment divides these supervision samples into training and validation sets according to time sequence or scenario type, and inputs them into the graph neural network scheduling strategy model. The graph convolutional layer weight parameters, decision output layer weight parameters, and related normalization coefficients and bias terms are jointly optimized, enabling the model to more accurately predict the real effects of different scheduling actions on the updated catering service knowledge graph and automatically adjust the balance weights of time constraints, energy consumption, and equipment health objectives. After training, this embodiment solidifies a set of model parameters that are consistent with the latest knowledge graph structure and node attributes into an adaptive scheduling parameter set, which is used to replace or incrementally update the parameters of the original scheduling strategy model, so that the unattended catering service system has the ability to learn and continuously optimize in long-term operation.

[0049] Furthermore, in step 900 of this embodiment, the previously obtained adaptive scheduling parameter set is used to continuously and adaptively optimize the subsequently generated artificial intelligence scheduling control instruction set to adapt to the dynamic changes in passenger flow intensity and resource allocation. Specifically, when a new catering service scheduling status dataset is input into the scheduling strategy model based on a graph neural network, this embodiment no longer uses the initial fixed parameters, but loads the latest adaptive scheduling parameter set. This allows the model to automatically adjust the weights of multiple optimization objectives such as time constraints, energy consumption, equipment health, and overall efficiency based on real-time changes in current order arrival rate, dining time distribution, equipment idle ratio, available personnel assistance level, and environmental constraints when generating the artificial intelligence scheduling control instruction set. This results in outputting a scheduling decision result that is more suitable for the current operating scenario.

[0050] In its implementation, this embodiment maps new batches of orders and resource statuses to updated graph-structured scheduling status data through feature encoding. Based on this, a scheduling strategy model carrying an adaptive scheduling parameter set is applied to aggregate multi-layered features of intent nodes, resource nodes, and process nodes, evaluating the priority and risk level of various optional scheduling actions. For example, when customer flow surges and critical cooking equipment approaches saturation, this embodiment automatically increases sensitivity to dining time constraints and service priorities based on the adaptive scheduling parameter set, prioritizing high-priority and time-sensitive orders, and dynamically adjusting resource allocation and execution order for some low-priority orders. When certain equipment frequently malfunctions or experiences high energy consumption, this embodiment adds penalty weights to equipment health status and unit task energy consumption during the decision-making process, guiding the scheduling results to select more stable and energy-efficient resource nodes to reduce failure risks and operating costs.

[0051] Through the aforementioned adaptive optimization mechanism, this embodiment, in each round of generating a new AI scheduling and control instruction set, uses the latest adaptive scheduling parameter set as a basis to implicitly modify and optimize the scheduling results. This ensures that the unattended intelligent catering service can maintain superior performance under various operating conditions such as changes in customer flow patterns, adjustments in resource allocation, equipment aging, and fluctuations in environmental conditions. Ultimately, this embodiment distributes the AI ​​scheduling and control instruction set optimized by adaptive parameters to various devices, forming an unattended intelligent catering service execution result set that outperforms the unoptimized solution in terms of order completion timeliness, customer waiting experience, overall energy consumption level, and equipment failure rate. This achieves continuous self-optimization and steady-state efficient operation of the catering service process.

[0052] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0053] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for unattended intelligent catering services based on multimodal perception and AI scheduling, characterized in that, include: Collect a multimodal scene dataset of the unattended catering service area; wherein, the multimodal scene dataset includes: customer behavior, food and raw material status, equipment operating status and environmental parameters; The multimodal scene dataset is time-synchronized, spatially registered, and cross-modal feature aligned to obtain a scene feature parameter set; A catering service knowledge graph containing intent nodes, resource nodes, and process nodes is constructed based on the scene feature parameter set. The aforementioned catering service knowledge graph is used to perform a unified semantic representation of dining intentions, resource allocation, and service process dependencies in unattended catering services, thereby obtaining a catering service state model. Based on the aforementioned catering service status model, the matching relationship between current order demand and equipment resources is analyzed to obtain a catering service scheduling status dataset; A scheduling strategy model based on graph neural networks is used to evaluate and make decisions on the catering service scheduling status dataset, resulting in an artificial intelligence scheduling control instruction set. Based on the AI ​​scheduling and control instruction set, the automatic cooking equipment, storage and heat preservation equipment, delivery and handling equipment, and terminal delivery equipment are driven to perform corresponding tasks, and the task completion status, time consumption, energy consumption data and equipment abnormal information are collected synchronously to obtain the operation feedback dataset. The intent node attributes, resource node attributes, and process node attributes in the catering service knowledge graph are updated based on the operational feedback dataset, and the parameters of the scheduling strategy model are adjusted to obtain an adaptive scheduling parameter set. The adaptive scheduling parameter set is used to adaptively optimize the subsequently generated artificial intelligence scheduling and control instruction set, so as to obtain an optimized set of unattended intelligent catering service execution results under the conditions of changes in passenger flow and resource allocation.

2. The unattended intelligent catering service method based on multimodal perception and AI scheduling according to claim 1, characterized in that, The collection of the multimodal scene dataset of the unattended catering service area includes: Visual acquisition devices deployed in unattended catering service areas are used to collect video and images of the dining area, processing area and delivery area to obtain a subset of visual scene data. By using voice interaction devices deployed in unattended catering service areas, customer ordering voices, inquiry voices, and feedback voices are collected in real time to obtain a subset of voice interaction data. Radio frequency identification (RFID) devices deployed in unattended catering service areas are used to identify and read food labels, ingredient labels, and tableware labels to obtain unique identification information, inventory status information, and circulation status information for food and ingredients, thus obtaining a subset of RFID data. By using equipment status acquisition devices and environmental sensors deployed in unattended catering service areas, the operating status parameters, temperature, humidity, illuminance and air quality of automatic cooking equipment, storage and heat preservation equipment, delivery and handling equipment and terminal delivery equipment are collected in real time to obtain a subset of equipment and environmental data. The visual scene data subset, voice interaction data subset, radio frequency identification data subset, and device and environment data subset are associated and integrated according to timestamps and spatial locations to generate a multimodal scene dataset covering customer behavior, food and raw material status, equipment operating status, and environmental parameters.

3. The unattended intelligent catering service method based on multimodal perception and AI scheduling according to claim 2, characterized in that, The process of performing time synchronization, spatial registration, and cross-modal feature alignment on the multimodal scene dataset yields a scene feature parameter set, including: The timestamps of the visual scene data subset, voice interaction data subset, radio frequency identification data subset, and device and environment data subset are calibrated, aligned, and aggregated using a preset unified time reference to obtain a time-synchronized multimodal data combination set; Using the spatial layout model of the unattended catering service area, the time-synchronized multimodal data set is spatially mapped and regionally calibrated to obtain a spatially registered multimodal data set. From the spatially registered visual scene data, voice interaction data, radio frequency identification data, and equipment and environment data, feature vectors representing customer behavior patterns, food and ingredient status, equipment operating conditions, and environmental constraints are extracted to obtain a multimodal feature vector set. The multimodal feature vector set is transformed in terms of feature dimension and semantic alignment to obtain the scene feature parameter set.

4. The unattended intelligent catering service method based on multimodal perception and AI scheduling according to claim 3, characterized in that, The expression for the spatial layout model is: ； in, ,satisfy: ； in, For spatial layout model; A set of discrete spatial nodes within an unattended catering service area; For the first Spatial coordinate vectors of spatial nodes; A set of spatial attributes corresponding to each spatial node; In order to be with the first The spatial attribute vector corresponding to each spatial node; To be in the unified layout reference frame The spatial node and the first Spatial metric distance between spatial nodes; For the first The spatial node and the first The adjacency strength function between spatial nodes; This is a linear mapping matrix that maps physical coordinates to the layout reference coordinate system; It is a norm 2; It is a nonnegative constant used to control the rate of spatial distance decay; This is an indicator function that takes the value 1 if the condition within the parentheses is true, and 0 otherwise. This is a non-negative distance threshold used to define the local neighborhood range in space; This represents the total number of spatial nodes. For the first Regional type identifier for each spatial node; For the first Parameters of the equipment and resource deployment capabilities supported by each spatial node; For the first The passage priority parameter of each spatial node in the customer's passage path; For the first Safety constraint parameters for each spatial node.

5. The unattended intelligent catering service method based on multimodal perception and AI scheduling according to claim 1, characterized in that, Based on the aforementioned scene feature parameter set, a catering service knowledge graph containing intent nodes, resource nodes, and process nodes is constructed, including: Based on the set of scene feature parameters, identify customer dining needs, taste preferences, dining time constraints, and number of diners; Each dining request is abstracted into an intent node, and the taste preference, dining time constraint, and number of diners information are used as attributes of the corresponding intent node to obtain a set of intent nodes; Based on the scene feature parameter set, identify the availability, processing capacity, capacity limit and spatial location of each meal, raw material, automatic cooking equipment, storage and heat preservation equipment, delivery and handling equipment and terminal delivery equipment; Each food item, raw material, automatic cooking equipment, storage and heat preservation equipment, and delivery and handling equipment are abstracted as resource nodes, and the available status, processing capacity, capacity limit, and spatial location are used as attributes of the corresponding resource nodes to obtain a set of resource nodes; Based on the business processes of order generation, food preparation, storage and heat preservation, delivery and handling and terminal delivery in unattended intelligent catering services, each business link is abstracted into a process node, and the sequential and parallel relationships between process nodes are established according to the scenario feature parameter set to obtain a set of process nodes; Based on the matching relationship between each intent node and the resource node that can meet the corresponding dining needs, an intent association edge is established between the intent node set and the resource node set, and the service priority, candidate resource set and service strategy are used as attributes of the intent association edge. Based on the occupancy relationship of each process node on food, raw materials and equipment during the execution process, a resource occupancy edge is established between the set of process nodes and the set of resource nodes, and the start time, end time, occupancy method and occupancy intensity are used as attributes of the resource occupancy edge; Based on the service affiliation relationship of each process node to the intent node, a process affiliation edge is established between the set of process nodes and the set of intent nodes, and the order identifier, task identifier and service status are used as attributes of the process affiliation edge. The intent node set, resource node set, and process node set, along with the corresponding intent association edges, resource occupation edges, and process affiliation edges, are organized into a graph structure to obtain a catering service knowledge graph containing intent nodes, resource nodes, and process nodes.

6. The unattended intelligent catering service method based on multimodal perception and AI scheduling according to claim 1, characterized in that, The expression for the catering service status model is: ； in, ,satisfy: ； in, For a moment The state vector of catering services; This is a state mapping function used to jointly map graph structure states, order demand states, resource states, and environmental states to the catering service state space; For at any time The graph structure service status is obtained based on the aforementioned catering service knowledge graph; For at any time A collection of orders and dining needs; For at any time The set of equipment and resource configurations; For at any time The set of environmental constraints and environmental parameters; For at any time Continuous state components characterizing the state of the federated service load; For at any time Semantic state components that represent the degree to which constraints are satisfied; For at any time Non-negative state components characterizing resource adequacy; For at any time Service status of the graph structure The graph embedding vector obtained by graph feature aggregation and encoding; A combination of orders and dining needs The extracted order demand feature vector; For a set of equipment and resource configurations The extracted resource supply feature vector; For the set of environmental constraints and environmental parameters The extracted environmental feature vector; These are the linear transformation matrices corresponding to the graph embedding vector, order demand feature vector, resource supply feature vector, and environmental feature vector, respectively. A nonlinear activation function to compress the linear transformation result into a preset interval; This is a combinatorial function that combines inner product features to characterize the coupling relationship between demand and resources, and demand and environment. The inner product between the order demand feature vector and the resource supply feature vector; The inner product between the order demand feature vector and the environment feature vector; For at any time The net resource scalar is calculated from the difference between the total resource supply and the total order demand; 1 is a column vector with all components set to 1, used to perform summation operations on the vector elements.

7. The unattended intelligent catering service method based on multimodal perception and AI scheduling according to claim 1, characterized in that, The analysis of the matching relationship between current order demand and equipment resources based on the catering service status model yields a catering service scheduling status dataset, including: Based on the feature parameters related to order information and customer dining needs in the scenario feature parameter set, the current valid orders, corresponding intent nodes, and dining time constraints and service priority information associated with each intent node are extracted from the catering service status model to obtain a subset of order demand status. Based on the feature parameters related to the status of food and raw materials and the operating status of equipment in the set of scene feature parameters, the available status, processing capacity, remaining capacity and spatial location of each resource node are extracted from the catering service status model to obtain a subset of equipment and resource status; Using the subset of order demand status and the subset of equipment and resource status, based on the pre-established association between intent nodes, resource nodes and process nodes in the catering service status model, the serviceability, estimated completion time and resource occupation cost between each order demand and candidate resource node are calculated to obtain the initial matching result set of orders and resources; In the initial matching result set of orders and resources, the initial matching relationship between each order and resource is filtered and adjusted in combination with dining time constraints, service priority and environmental parameter constraints, so as to generate a matching result set of orders and resources that meet the dining time constraints and environmental parameter constraints; The order demand status subset, the equipment and resource status subset, and the order and resource constraint matching result set are structured and organized to form a catering service scheduling status dataset that represents the current order demand and equipment resource matching relationship.

8. The unattended intelligent catering service method based on multimodal perception and AI scheduling according to claim 1, characterized in that, The method employs a graph neural network-based scheduling strategy model to evaluate and make decisions on the catering service scheduling status dataset, resulting in an artificial intelligence scheduling control instruction set, including: By utilizing the intent nodes, resource nodes, process nodes, and associated edges in the catering service knowledge graph, and combining the scheduling status information in the catering service scheduling status dataset that represents the current order demand status, equipment and resource status, and order and resource constraint matching relationship, the intent nodes, resource nodes, and process nodes are labeled to obtain a graph structure scheduling status dataset. The graph structure scheduling state dataset is input into the graph neural network-based scheduling strategy model. Multi-layer graph convolution is used to perform feature aggregation and semantic encoding on each intention node, resource node, and process node to obtain a graph structure scheduling feature representation set. Using the preset decision output layer in the graph neural network-based scheduling strategy model, the optional scheduling actions between each intention node and the candidate resource node formed by the graph structure scheduling feature representation set are evaluated to obtain a set of candidate scheduling actions and scheduling priorities. In the set of candidate scheduling actions, the candidate scheduling actions that do not meet the constraints are filtered by combining the constraints of meal time, service priority, equipment safe operation and environmental parameters. Based on the scheduling priority of the remaining candidate scheduling actions, a sequence of scheduling actions that meet the constraints is generated to obtain a set of scheduling decision results. Based on the scheduling decision result set, each scheduling action is mapped into specific control instructions, task allocation parameters, and execution timing information for automatic cooking equipment, storage and heat preservation equipment, delivery and handling equipment, and terminal delivery equipment, in order to generate an artificial intelligence scheduling control instruction set.

9. The unattended intelligent catering service method based on multimodal perception and AI scheduling according to claim 1, characterized in that, The system drives automatic cooking equipment, storage and heat preservation equipment, delivery and handling equipment, and terminal delivery equipment to execute corresponding tasks based on the artificial intelligence scheduling and control instruction set, and synchronously collects task completion status, time consumption, energy consumption data, and equipment anomaly information to obtain an operational feedback dataset, including: According to the AI ​​scheduling and control instruction set, each scheduling action is sent to the corresponding automatic cooking equipment, storage and heat preservation equipment, delivery and handling equipment and terminal delivery equipment, so as to control each equipment to complete the tasks of food preparation, storage and heat preservation, delivery and handling and terminal delivery according to the predetermined execution sequence. During the execution of tasks by various automatic cooking equipment, storage and heat preservation equipment, delivery and handling equipment and terminal delivery equipment, the task start time, task end time, task completion status, time consumption, energy consumption data and equipment abnormal information corresponding to each scheduling action are collected and recorded to obtain task execution process data. The task execution process data is structured and organized, and a one-to-one correspondence is established with the corresponding scheduling actions to obtain the operation feedback dataset.

10. The unattended intelligent catering service method based on multimodal perception and AI scheduling according to claim 1, characterized in that, The process involves updating the intent node attributes, resource node attributes, and process node attributes in the catering service knowledge graph based on the operational feedback dataset, and adjusting the parameters of the scheduling strategy model to obtain an adaptive scheduling parameter set, including: Based on the operational feedback data related to order completion status, time consumption, energy consumption data and equipment abnormality information in the operational feedback dataset, each operational feedback data is associated with the corresponding intent node, resource node and process node to obtain a node-level operational feedback subset; Based on the node-level operational feedback subset, statistical analysis and incremental updates are performed on the intent node attributes, resource node attributes, and process node attributes in the catering service knowledge graph to obtain the updated catering service knowledge graph. The runtime feedback dataset is matched with the set of artificial intelligence scheduling control instructions used in the scheduling process to construct a scheduling sample set that represents the relationship between scheduling actions and runtime results. Based on the scheduling sample set, the parameters of the scheduling strategy model based on the graph neural network are optimized and trained to obtain an adaptive scheduling parameter set consistent with the updated catering service knowledge graph.

Images