Automatic ingredient production system and method for prefabricated dishes

Abstract

The invention discloses a prefabricated dish automatic ingredient production system and method, and relates to the technical field of food intelligent manufacturing, and the method comprises the steps: extracting a global semantic feature vector of a denoised high-quality image, and generating process sequence constraint intensity through MLP; by taking the global semantic feature vector as a node and the process sequence constraint strength as an edge weight, constructing a transmembrane state time sequence diagram through a dynamic graph convolutional network, optimizing a graph propagation process through a time smooth constraint loss formula, and generating a time sequence confidence coefficient; and based on the time sequence confidence coefficient, a mechanical arm action regulation and control scheme is obtained, a fuzzy logic control mapping method is adopted for mapping, and a mechanical arm control instruction is generated and executed.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent food manufacturing, and in particular to an automatic ingredient production system for prepared dishes and a method thereof. Background Art

[0002] In recent years, with the rapid development of automation and artificial intelligence in the food industry, prepared meal production has gradually shifted from traditional manual operations to intelligent and digital control. In the ingredient preparation process, related technologies have evolved from simple automated operations such as mechanical sorting and quantitative feeding to comprehensive systems based on machine vision, sensor fusion, and collaborative industrial robots. With the deepening of the concept of Industry 4.0, prepared meal production is moving towards intelligent optimization, adaptive control, and green production. Process orchestration and resource scheduling technologies have become key to improving production efficiency.

[0003] The current automatic ingredient preparation technology for pre-prepared meals faces two major technical bottlenecks: first, in terms of processing complex process relationships, the existing technology has limited ability to coordinate the characteristics of multiple raw materials, making it difficult to flexibly respond to diversified production needs; second, in terms of dynamic optimization decision-making, the existing methods lack the perception and response mechanism for real-time working conditions, and have technical defects such as insufficient adaptability. Summary of the Invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides an automatic ingredient preparation method for pre-prepared dishes to solve the problems of insufficient process constraint processing capability and poor adaptability of dynamic optimization decision-making in the collaborative processing of multiple raw materials in the prior art.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions: In a first aspect, the present invention provides a method for automatically allocating ingredients for pre-prepared dishes, comprising: The food image sequence and process parameters are obtained and preprocessed; the food image is used to generate an industrial noise synthetic image through an adversarial noise generator, and the denoising action is selected through a reinforcement learning algorithm to output a denoised high-quality image; the global semantic feature vector of the denoised high-quality image is extracted, and the process sequence constraint strength is generated through MLP; with the global semantic feature vector as the node and the process sequence constraint strength as the edge weight, a trans-membrane state timing diagram is constructed through a dynamic graph convolutional network, and the graph propagation process is optimized through the time smoothing constraint loss formula to generate timing confidence; based on the timing confidence, the robot arm motion control plan is obtained, and the fuzzy logic control mapping method is used for mapping, and the robot arm control instructions are generated and executed.

[0007] As a preferred embodiment of the method for automatically allocating ingredients for prepared dishes of the present invention, the food image sequence refers to a continuous image sequence of food on a conveyor belt captured under different lighting conditions; Process parameters include temperature, humidity, and target ingredient sequence; Preprocessing includes time synchronization alignment, data quality enhancement, and data structured storage.

[0008] As a preferred embodiment of the method for automatically allocating ingredients for pre-prepared dishes according to the present invention, The food image is passed through the anti-noise generator to generate an industrial noise synthetic image. The specific steps are as follows: Extract the underlying features of the image based on the original image of the food through the pre-trained CNN network; Based on dynamic entropy calculation, the entropy value distribution of the noise area is counted, the noise complexity of the image area is quantified, and the noise type distribution is obtained; Based on the noise type distribution, the noise pattern encoder dynamically maps the underlying features of the image and fuses them with physical priors to generate a noise type conditional vector. Based on the CycleGAN architecture, the food image sequence and the noise type condition vector are input into the noise generator, and industrial noise synthetic images are generated through adversarial training.

[0009] As a preferred embodiment of the method for automatically allocating ingredients for pre-prepared dishes according to the present invention, The reinforcement learning algorithm is used to select the denoising action and output the denoised high-quality image. The specific steps are as follows: Based on industrial noise synthetic images, the global noise complexity of the synthetic images is calculated through grayscale conversion, probability distribution statistics and Shannon entropy formula; Based on multi-dimensional feature statistics and dynamic threshold division, the global noise complexity of the synthetic image is dynamically mapped and output as a three-level noise pattern; Based on the three-level noise pattern, the TD3 reinforcement learning algorithm dynamically selects and executes the optimal denoising action in the denoising action space to generate a high-quality denoising image.

[0010] As a preferred embodiment of the method for automatically allocating ingredients for pre-prepared dishes according to the present invention, The specific steps of extracting the global semantic feature vector of the high-quality image after denoising and generating the strength of the sequence constraint through the MLP process are as follows: Extract high-level semantic features by performing attention weighting on the denoised high-quality images; Analyze the contextual feature vectors of spatial relationships and interactions in the denoised high-quality images through the self-attention mechanism; Perform multi-level feature extraction and fusion on the denoised high-quality image through a task-specific branch network to extract task discriminant features; The temporal feature vector of the dynamic changes and temporal consistency of the denoised high-quality image is captured through the temporal convolutional network; Based on the extracted global semantic feature vector, the process sequence constraint strength between ingredients in the cooking process is calculated through MLP.

[0011] As a preferred embodiment of the method for automatically allocating ingredients for pre-prepared dishes according to the present invention, The specific steps of constructing a trans-membrane state timing diagram using a dynamic graph convolutional network with global semantic feature vectors as nodes and process sequence constraint strength as edge weights and optimizing the graph propagation process through a time smoothing constraint loss formula to generate timing confidence are as follows: Using global semantic feature vectors as nodes and process sequence constraint strength as edge weights, a cross-modal temporal graph is constructed using the ST-GCN++ dynamic graph convolutional network. The cross-modal time series graph propagation process is optimized using time smoothness constraint loss, and node embedding aggregation is performed on the optimized time series graph; The aggregated graph embedding is mapped to the confidence space through a fully connected layer, and the confidence sequence of each time step is output; The time smoothing constraint is used to perform temporal consistency and weighted fusion processing on the confidence sequence to generate the temporal confidence.

[0012] As a preferred embodiment of the method for automatically allocating ingredients for pre-prepared dishes according to the present invention, The specific steps of obtaining the manipulator motion control scheme based on the timing confidence, mapping it using the fuzzy logic control mapping method, generating the manipulator control instructions and executing them are as follows: Based on the statistical analysis of historical confidence distribution data, low confidence threshold and high confidence threshold are defined; When the current confidence value is higher than the high confidence threshold, the robot arm moves according to the original XYZ axis motion trajectory; The current confidence value is between the low confidence threshold and the high confidence threshold. The robot arm needs to fine-tune its position, posture, or grasping action to ensure that it can grasp the material more accurately or make the material fall into the target container more precisely. If the current confidence value is lower than the low confidence threshold, the robot arm will immediately stop the current operation and issue an early warning.

[0013] In the second aspect, the present invention provides an automatic ingredient production system for pre-prepared dishes, including a data acquisition module, an anti-noise enhancement module, a semantic feature encoding module, a cross-modal timing graph reasoning module, and an intelligent action decision and control module; the data acquisition module is used to obtain a sequence of food images and process parameters, and perform preprocessing; the anti-noise enhancement module is used to generate an industrial noise synthetic image from the food image through an anti-noise generator, and select a denoising action through a reinforcement learning algorithm and output a denoised high-quality image; the semantic feature encoding module is used to extract the global semantic feature vector of the denoised high-quality image, and generate the process sequence constraint strength through MLP; the cross-modal timing graph reasoning module is used to use the global semantic feature vector as a node and the process sequence constraint strength as an edge weight, construct a cross-modal timing graph through a dynamic graph convolutional network, and optimize the graph propagation process through a time smoothing constraint loss formula to generate a timing confidence; the intelligent action decision and control module is used to obtain a robotic arm motion control plan based on the timing confidence, and use a fuzzy logic control mapping method for mapping, generate a robotic arm control instruction and execute it.

[0014] In a third aspect, the present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, any step of the automatic ingredient preparation method for pre-prepared dishes as described in the first aspect of the present invention is implemented.

[0015] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, any step of the method for automatic ingredient preparation of pre-prepared dishes as described in the first aspect of the present invention is implemented.

[0016] The beneficial effects of the present invention are: through dynamic process constraint identification and online parameter adjustment, the production process is made flexible and intelligent, which effectively enhances the adaptability to diversified production needs; by constructing a process constraint model based on intelligent perception and data-driven, combined with real-time working condition feedback, the production parameters are dynamically adjusted, thereby improving the operating efficiency and resource utilization efficiency of the production system. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 Flowchart of a method for automatically allocating ingredients for prepared dishes.

[0019] Figure 2 Schematic diagram of the automatic ingredient batching system for pre-prepared dishes.

[0020] Figure 3 Flowchart for the process flow diagram for generating synthetic images of industrial noise.

[0021] Figure 4 Flowchart for global semantic feature extraction and process constraint strength generation. DETAILED DESCRIPTION

[0022] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0023] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0024] Secondly, the term "one embodiment" or "embodiment" herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in various places throughout this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive of other embodiments.

[0025] Reference Figures 1 to 4 , is an embodiment of the present invention, which provides a method for automatically allocating ingredients for pre-prepared dishes, comprising the following steps: S1: Obtain food image sequence and process parameters, and perform preprocessing; S1.1: Food image sequence refers to a continuous image sequence of food on a conveyor belt captured under different lighting conditions; Process parameters include temperature, humidity, and target ingredient sequence; Use high-frame-rate industrial cameras and adjustable LED light sources to capture food image sequences in real time, and use the OPC UA protocol to obtain process parameters; It should be noted that the food image sequence is achieved by deploying high-speed industrial cameras and an adjustable LED light source array to simulate the lighting conditions of dawn, dusk, day and night on the food processing automated production line, coordinating with the conveyor belt encoder synchronous trigger mechanism, and using the PLC control system to capture high-definition image sequences of food as it passes through the detection area in real time.

[0026] Based on the specifications and instructions of the OPC UA protocol, the process parameters periodically send requests to the temperature and humidity sensors to obtain the latest temperature and humidity data, and send read instructions to the storage of target batching sequence information through the OPC UA protocol to obtain the target batching sequence corresponding to the current production task.

[0027] S1.2: Preprocessing includes time synchronization alignment, data quality enhancement, and data structured storage; Time synchronization and alignment of multi-source data are achieved through high-precision clock protocols and filtering algorithms, and data quality is enhanced by combining image enhancement and sensor noise reduction technologies. Finally, a structured data system is constructed using layered containerized storage and metadata annotation, forming a complete data link from acquisition to application to complete data structured storage.

[0028] It should be noted that the captured food image sequences and process parameters, including temperature, temperateness, and target ingredient sequence, are first time-synchronized to eliminate temporal skew in the multimodal data and ensure strict correspondence between image frames and process parameters. Data quality enhancement is then performed to improve data usability, suppress noise, and enhance feature expression. Finally, structured data storage is implemented to build an efficient and scalable multimodal data warehouse.

[0029] S2: Generates industrial noise synthetic images from food images through an adversarial noise generator, selects denoising actions through a reinforcement learning algorithm, and outputs high-quality denoised images. Extract the underlying features of the image based on the original image of the food through the pre-trained CNN network; It should be noted that the food image is first resized to the required size for the CNN model and fed into a pre-trained convolutional neural network. The conv1 convolutional layer performs preliminary feature extraction on the input food image. The batch normalization layer then normalizes the feature map, and the ReLU activation function enhances the feature representation. The processed feature map then passes through a max pooling layer to reduce its spatial resolution and enters the residual block stage to extract the underlying features of the image.

[0030] Dynamic entropy calculation is used to calculate the entropy distribution of the noise area, quantify the noise complexity of the image area, and obtain the noise type distribution; It should be noted that the input image is first segmented into multiple non-overlapping local areas using a sliding window, and then the grayscale histogram of each window area is calculated. The Shannon entropy value of each window area is then calculated based on the grayscale histogram, and the value range is set to: The entropy threshold value (where The noise areas with high entropy values ​​are identified by clustering analysis. The K-means algorithm is used to divide the noise areas into multiple categories according to the entropy distribution characteristics. The average entropy value and entropy variance of each category are calculated. The multidimensional feature vector is constructed based on the entropy distribution. Finally, the noise areas are classified by SVM classifier, the complexity characteristics of noise areas of different categories are analyzed, and the noise type distribution of each area is determined. Based on the noise type distribution, the noise pattern encoder dynamically maps the underlying features of the image and fuses them with physical priors to generate a noise type conditional vector. It should be noted that, first, according to the noise type distribution results, the feature representations of different noise types are integrated, and then the underlying features of the image are dynamically mapped through the noise pattern encoder. The encoder adopts a multi-layer perceptron structure to map the underlying features of the input to a higher-dimensional feature space. At the same time, combined with physical prior knowledge, the mapped features are weighted fused. Finally, the fused feature vector is spliced ​​with the noise type distribution information to generate a conditional vector containing the noise type information.

[0031] Based on the CycleGAN architecture, the food image sequence and the noise type condition vector are input into the noise generator, and industrial noise synthetic images are generated through adversarial training; It should be noted that the food image sequence is first input into the encoder part of the noise generator, and the noise type condition vector is connected to the intermediate layer feature map of the encoder. Then the noise generator transforms the fused features through a series of residual blocks, and gradually upsamples to generate a synthetic image with the same size as the original image. The discriminator receives the real image and the synthetic image as input, and extracts multi-level features through the convolution layer and the LeakyReLU activation function. By alternately optimizing the parameters of the generator and the discriminator, and after a certain training cycle, the generator can stably output synthetic images containing industrial noise features such as oil stains and water stains.

[0032] The denoising action space is formed through the synergy of noise feature analysis, process constraint mapping and reinforcement learning strategy; Noise feature analysis uses a multimodal feature extraction network to extract noise representation vectors containing spatial, frequency and temporal features from industrial noise synthetic images. It should be noted that the industrial noise synthetic image is first input into the multimodal feature extraction network, which contains a parallel processing branch structure. The spatial feature extraction branch uses a convolutional neural network to process the spatial information of the image, and extracts spatial features such as edges and textures through multiple convolutional layers and pooling layers. The frequency domain feature extraction branch performs Fourier transform on the image to obtain spectral information, and then separates different frequency components through a filter group to extract the distribution characteristics of noise in the frequency domain. The temporal feature extraction branch processes the image sequence and uses a temporal convolutional network or a recurrent neural network to capture the inter-frame change pattern. The features extracted by each branch are integrated through the fusion layer to form a unified noise representation vector.

[0033] Process constraint mapping uses equipment capabilities and business rule constraint models to model the noise characterization vectors to form a set of feasible action candidates that meet physical constraints. It should be noted that the noise characterization vector containing spatial, frequency domain and timing characteristics is first input into the equipment capability and business rule constraint model. The model consists of an equipment capability evaluation unit and a business rule verification unit. The equipment capability evaluation unit screens the spatial features in the noise characterization vector according to hardware parameters such as the motion range and maximum acceleration of the robotic arm to eliminate the possibility of actions that exceed the equipment capability range; the business rule verification unit performs compliance checks on the timing characteristics based on preset operating specifications and filters out action options that violate the operating procedures. After dual constraint processing, the model outputs a set of feasible action candidates that meet physical constraints.

[0034] The reinforcement learning strategy uses the TD3 algorithm strategy network to form a denoising action space that dynamically adapts to different noise scenarios. It should be noted that the set of feasible action candidates that meet physical constraints is first input into the policy network of the TD3 algorithm, which consists of two parallel neural networks, one for action selection and the other for target action prediction; the policy network receives the current state features, including the spatial distribution features, frequency domain characteristics and time series change information in the noise representation vector, and processes the input features through a multi-layer perceptron structure to generate a continuous action value distribution. At the same time, two Q networks are used to evaluate the action value respectively, and the minimum value is taken as the target value to reduce the overestimation bias; after multiple iterative optimizations, the policy network outputs the denoised action space for different noise scenarios.

[0035] Based on the industrial noise synthetic image, the global noise complexity of the synthetic image is calculated through grayscale conversion, probability distribution statistics and Shannon entropy formula. The expression is: in, represents the global noise complexity, represents the grayscale index, Represents the image at position The pixel gray value, Represents the total number of pixels, is the gray level index, represents the horizontal coordinate of the pixel in the industrial noise synthetic image, represents the vertical coordinate of the pixel of the industrial noise synthetic image; It should be noted that the weighted averaging method is first used to convert the color industrial noise composite image into a grayscale image, and then the histogram statistics of the grayscale image are performed to calculate the probability of occurrence of each grayscale pixel to ensure that the sum of all probabilities is 1. Then, the global entropy value of the image is calculated based on the Shannon entropy formula. Finally, the noise complexity is evaluated according to the entropy value. The higher the entropy value, the more uneven the noise distribution in the image and the greater the complexity.

[0036] Through multi-dimensional feature statistics and dynamic threshold division, the global noise complexity of the synthetic image is dynamically mapped and output as a three-level noise pattern; It should be noted that the multi-dimensional features of the synthetic image including contrast, correlation, energy and homogeneity are first extracted, and the global entropy value and local entropy value distribution characteristics are combined. According to the characteristic distribution of different noise patterns in the training data set, three threshold intervals are dynamically determined. Finally, the calculated synthetic image feature value is compared with the dynamic threshold and mapped to the corresponding noise pattern level.

[0037] Based on the three-level noise pattern, the TD3 reinforcement learning algorithm dynamically selects and executes the optimal denoising action in the denoising action space to generate a high-quality denoising image.

[0038] It should be noted that the three-level noise pattern is first input as a state into the policy network of the TD3 algorithm. The policy network contains two fully connected hidden layers and uses the ReLU activation function. Then, based on the set of denoising operations available in the action space corresponding to the current noise pattern, the TD3 algorithm evaluates the value of different denoising actions through two Q networks and selects the action with the highest Q value as the optimal strategy. After executing the selected denoising action, the image is filtered through a convolutional neural network, using a deconvolution operation with a convolution kernel of appropriate size and step size. Finally, a denoised high-quality image is output.

[0039] S3: Extract the global semantic feature vector of the denoised high-quality image and generate the process order constraint strength through MLP; S3.1: The global semantic feature vector includes high-level semantic features, contextual features, task discriminant features, and temporal correlation information, where: Extract high-level semantic features by performing attention weighting on the denoised high-quality images; It should be noted that the denoised, high-quality image is first fed into the feature extraction network, where feature maps of varying scales are obtained through progressive downsampling. Multiple convolution and pooling operations are then performed in the intermediate or deep layers of the feature extraction network to obtain feature maps rich in semantic information. An attention mechanism is then constructed, where the feature maps are fed into the channel attention and spatial attention components of the attention mechanism, respectively, to calculate the corresponding channel weight maps and spatial weight maps. The two weight maps are then fused to obtain the final attention weight map. Finally, this attention weight map is used to weight the original feature map, highlighting important feature areas and suppressing irrelevant information through element-wise multiplication. This results in a high-level semantic feature map containing high-level semantic information, such as the food image contour features and key texture features.

[0040] Analyze the contextual feature vectors of spatial relationships and interactions in the denoised high-quality images through the self-attention mechanism; It should be noted that the denoised high-quality image is first divided into multiple local regions, and then the feature vector of each image block is mapped to the query vector, key vector and value vector space through the fully connected layer. Then, the similarity score between the query vector and all key vectors is calculated. The attention weight is calculated according to the similarity score, and the score is converted into a probability distribution through the Softmax function. The value vector is weighted and summed using the attention weight to obtain the contextual feature representation of each image block. Finally, the contextual features of all image blocks are recombined to form a complete contextual feature vector containing cross-regional spatial relationship and interaction information.

[0041] Perform multi-level feature extraction and fusion on the denoised high-quality image through a task-specific branch network to extract task discriminant features; It should be noted that first, a high-quality image is input into the backbone network, and feature maps of different levels are extracted through multiple convolution operations. Then, branch connections are set at multiple levels of the backbone network. Each branch network extracts features for a specific task. Subsequently, the features extracted by different branches are fused, and multi-level features are integrated by element-by-element addition or splicing operations. Finally, the fused features are further processed through a fully connected layer or a convolutional layer to generate task discriminant features.

[0042] The temporal feature vector of the dynamic changes and temporal consistency of the denoised high-quality image is captured through the temporal convolutional network; It should be noted that a series of denoised high-quality images arranged in chronological order are first input into a temporal convolutional network composed of multiple temporal convolutional layers stacked together. Batch normalization layers and ReLU activation functions are inserted between the temporal convolutional layers to enhance the network feature extraction capability. A sliding time window mechanism is used to process the input sequence, extracting spatiotemporal features frame by frame, and finally outputting a fixed-dimensional temporal feature vector through the fully connected layer.

[0043] S3.2: Based on the global semantic feature vector, the process sequence constraint strength between ingredients in the cooking process is calculated through MLP, where: The hidden layer expression is: in, Representative The weight matrix of the layer, Representative The bias vector of the layer, Representative The feature representation of the layer (e.g., the hidden state of the previous time step in a recurrent neural network), represents the ReLU function, is the activation value of the current layer; The output layer expression is: in, represents the weight vector of the output layer, represents the bias scalar of the output layer, Represents the feature representation of the last hidden layer (e.g., a feature vector that integrates high-level semantic information such as object parts and textures in a classification task). represents the Sigmoid function, Representative ingredients and exist The strength of the process sequence constraints at the moment; It should be noted that the global semantic feature vector corresponding to each ingredient is first input into the input layer of the MLP, and the dimension of the input layer is kept consistent with the dimension of the global semantic feature vector. Then the input data is passed to the hidden layer, which contains two fully connected layers and is nonlinearly transformed through the ReLU activation function. The output of the hidden layer is then passed to the output layer. The final output scalar value represents the process sequence constraint strength value between the two ingredients.

[0044] S4: Using global semantic feature vectors as nodes and process sequence constraint strength as edge weights, a dynamic graph convolutional network is used to construct a trans-membrane state timing graph and optimize the graph propagation process using a time-smoothing constraint loss formula to generate timing confidence. S4.1: Using global semantic feature vectors as nodes and process sequence constraint strength as edge weights, a cross-modal temporal graph is constructed through the ST-GCN++ dynamic graph convolutional network.

[0045] It should be noted that the global semantic feature vectors of all ingredients at each time step are first used as a set of nodes of the graph, where each node corresponds to the feature representation of an ingredient at that moment; then, the connection relationship between the nodes is established according to the calculation results of the process sequence constraint strength, and the calculated process sequence constraint strength value is used as the weight of the edge to form an initial static graph structure. The graph structure is then processed by the spatiotemporal convolution layer in the ST-GCN++ network, where the spatial convolution part uses a graph convolution operation to aggregate the information of adjacent nodes, and the temporal convolution part uses a temporal convolution operation to capture the change pattern between different time steps; during the processing, ST-GCN++ will dynamically adjust the graph structure of each time step, and control the propagation of information between different time steps through a gating mechanism. The network finally outputs a node representation after the fusion of spatiotemporal features. The updated node features contain both the temporal evolution information of the ingredient itself and the process sequence constraint relationship with other ingredients, finally forming a complete cross-modal temporal graph.

[0046] S4.2: Use the time smoothness constraint loss to optimize the cross-modal time series graph propagation process, aggregate the node embeddings of the optimized time series graph, map the aggregated graph embeddings to the confidence space through a fully connected layer, and output the confidence sequence of each time step; It should be noted that a time smoothing constraint loss function is first added to the propagation process of the cross-modal timing graph. This function ensures that the node features in the timing graph maintain a smooth transition over time by calculating the difference between the node features of adjacent time steps and minimizing the difference. Then, a node embedding aggregation operation is performed on the timing graph after time smoothing constraint optimization. All node features are aggregated using an exemplary weighted summation method. The weight is determined by the edge weight between nodes, that is, the strength of the process sequence constraint. Then, the graph embedding obtained by aggregation is input into the fully connected layer. The dimension of the fully connected layer is set according to the task requirements. Finally, the graph embedding is mapped to the confidence space through the nonlinear transformation of the fully connected layer, and a sequence containing the confidence value of each time step is output.

[0047] The time smoothing constraint is used to perform temporal consistency and weighted fusion processing on the confidence sequence to generate the temporal confidence, which is expressed as: in, Indicates time The confidence of the time series after smoothing, Indicates time The original confidence level when Indicates the smoothing weight coefficient (the range of values ​​is: (0≤ ≤1)), Indicates time The confidence value when Indicates time The confidence value when Indicates the The confidence level of the position, Indicates the The confidence level of the position, is a general index variable that can represent any time step or position, represents the sequence length, represents the boundary indicator function, represents the time step, represents the normalized denominator; It should be noted that first, the difference of the values ​​of adjacent time steps in the confidence sequence is calculated, and the difference results are weighted by the exponential moving average method. Then, the weighted difference results are fused with the original confidence sequence by using the normalized weighted sum method to ensure that the ratio of the temporal smoothing constraint term is coordinated with the original confidence. Then, the fused sequence is locally smoothed by the convolution layer to eliminate local noise fluctuations. Finally, the time smoothing constraint loss is applied to the processed sequence again for global consistency adjustment, so as to obtain the final confidence sequence with temporal consistency.

[0048] S5: Based on the timing confidence, the robot arm motion control plan is obtained and mapped using the fuzzy logic control mapping method to generate and execute the robot arm control instructions.

[0049] S5.1: Based on statistical analysis of historical confidence distribution data, define low confidence thresholds and high confidence thresholds; It should be noted that first, historical confidence sequence data are collected and organized to form a sample set containing confidence values ​​of multiple time steps. Then, quantile statistical analysis is performed on the confidence values ​​in the sample set, and the segmentation threshold is determined according to the confidence distribution characteristics. Finally, two clear dividing points are determined as the low confidence threshold and the high confidence threshold.

[0050] According to the comparison results of the confidence threshold and the timing confidence, the robot arm motion control plan is generated. The steps are as follows: When the current confidence value is higher than the high confidence threshold, the robot arm moves according to the original XYZ axis motion trajectory; The current confidence value is between the low confidence threshold and the high confidence threshold. The robot arm needs to fine-tune its position, posture, or grasping action to ensure that it can grasp the material more accurately or make the material fall into the target container more precisely. If the current confidence value is lower than the low confidence threshold, the robot arm will immediately stop the current operation and issue an early warning.

[0051] S5.2: Use fuzzy logic control mapping method to map the robot arm motion control scheme, generate the robot arm control instructions and execute them. The steps are as follows: By using the membership function to fuzzify the time series confidence, a fuzzy set of confidence levels is obtained; It should be noted that the confidence is first divided into different levels, and the corresponding membership function is defined for each level. Then, the time series confidence value of each time step is input into the corresponding membership function, and the membership of the value at each level is calculated; finally, a fuzzy set of confidence levels is obtained.

[0052] Based on fuzzy rule matching and reasoning algorithm, the control rules and the membership distribution of fuzzy control variables are derived from fuzzy sets; It should be noted that first, a fuzzy rule base containing multiple condition-action rules is established, and then the confidence fuzzy set and noise complexity fuzzy set of each time step are matched, and the activation strength of each rule is calculated. Then, through the fuzzy reasoning synthesis algorithm, the fuzzy set generated by each rule after activation is operated to obtain a comprehensive fuzzy control quantity membership distribution; finally, the output fuzzy control quantity membership distribution is defuzzified to obtain the final control rule and the corresponding fuzzy control quantity membership distribution.

[0053] The membership distribution is defuzzified by weighted average, and the robot arm control instructions are generated and executed through the inverse kinematic solution.

[0054] It should be noted that first, each value point in the membership distribution of the fuzzy control quantity is multiplied by the corresponding membership value, and then all weighted values ​​are summed and divided by the total membership to obtain the precise control quantity value. Then, the control quantity value is input into the kinematic inverse solution module, and the adjustment amount of each joint angle is calculated according to the current joint state of the robot arm and the target position of the end effector. Finally, the joint angle adjustment instruction is sent to the robot arm controller for execution to complete the spatial posture adjustment of the denoising action, so that the end effector of the robot arm moves accurately according to the calculated trajectory, thereby achieving precise operation of the target object.

[0055] This embodiment also provides a system for automatically preparing ingredients for prepared dishes, comprising: a data acquisition module, an anti-noise enhancement module, a semantic feature encoding module, a semantic feature coding module, and an intelligent action decision and control module; Data acquisition module, used to obtain food image sequences and process parameters and perform preprocessing; The anti-noise enhancement module is used to generate industrial noise synthetic images from food images through an adversarial noise generator, select denoising actions through a reinforcement learning algorithm, and output a high-quality denoised image; Semantic feature encoding module, which is used to extract the global semantic feature vector of the denoised high-quality image and generate the process order constraint strength through MLP; The cross-modal timing graph reasoning module uses global semantic feature vectors as nodes and process sequence constraint strengths as edge weights. It constructs a cross-modal timing graph through a dynamic graph convolutional network and optimizes the graph propagation process using a time-smoothing constraint loss formula to generate timing confidence. The intelligent action decision and control module is used to obtain the robot arm motion control plan based on the timing confidence, and use the fuzzy logic control mapping method to map it, generate the robot arm control instructions and execute them.

[0056] This embodiment also provides a computer device suitable for the automatic ingredient preparation method for pre-prepared dishes, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute computer-executable instructions to implement the automatic ingredient preparation method for pre-prepared dishes proposed in the above embodiment.

[0057] The computer device may be a terminal, comprising a processor, memory, a communication interface, a display, and an input device connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores an operating system and computer programs. The internal memory provides an environment for the operating system and computer programs stored in the non-volatile storage media. The communication interface of the computer device is used to communicate with external terminals via wired or wireless communication. Wireless communication may be achieved via Wi-Fi, a carrier network, NFC (near-field communication), or other technologies. The display of the computer device may be a liquid crystal display or an electronic ink display. The input device may be a touchscreen overlay on the display, buttons, a trackball, or a touchpad on the computer device housing, or an external keyboard, touchpad, or mouse.

[0058] This embodiment also provides a storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for automatically preparing ingredients for pre-prepared dishes as proposed in the above embodiment; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0059] In summary, the present invention realizes the flexibility and intelligence of the production process through dynamic process constraint identification and online parameter adjustment, effectively enhancing the adaptability to diversified production needs, and realizes dynamic adjustment of production parameters by constructing a process constraint model based on intelligent perception and data-driven, combined with real-time working condition feedback, thereby improving the operating efficiency and resource utilization efficiency of the production system.

[0060] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for automatically preparing ingredients for pre-prepared dishes, characterized by: include, Obtain food image sequences and process parameters and perform preprocessing; The food image is passed through the adversarial noise generator to generate an industrial noise synthetic image, and the reinforcement learning algorithm is used to select the denoising action and output the denoised high-quality image; Extract the global semantic feature vector of the denoised high-quality image and generate the process order constraint strength through MLP; Using global semantic feature vectors as nodes and process sequence constraint strength as edge weights, a dynamic graph convolutional network is used to construct a trans-membrane state timing graph and optimize the graph propagation process using a time-smoothing constraint loss formula to generate timing confidence. Based on the timing confidence, the robot arm motion control plan is obtained, and the fuzzy logic control mapping method is used for mapping to generate and execute the robot arm control instructions.

2. The automatic ingredient allocation method for pre-prepared dishes according to claim 1, wherein: The food image sequence refers to a continuous image sequence of food on the conveyor belt captured under different lighting conditions; Process parameters include temperature, humidity, and target ingredient sequence; Preprocessing includes time synchronization alignment, data quality enhancement, and data structured storage.

3. The automatic ingredient allocation method for pre-prepared dishes according to claim 2, characterized in that: The food image is passed through the anti-noise generator to generate an industrial noise synthetic image. The specific steps are as follows: Extract the underlying features of the image based on the original image of the food through the pre-trained CNN network; Based on dynamic entropy calculation, the entropy value distribution of the noise area is counted, the noise complexity of the image area is quantified, and the noise type distribution is obtained; Based on the noise type distribution, the noise pattern encoder dynamically maps the underlying features of the image and fuses them with physical priors to generate a noise type conditional vector. Based on the CycleGAN architecture, the food image sequence and the noise type condition vector are input into the noise generator, and industrial noise synthetic images are generated through adversarial training.

4. The automatic ingredient allocation method for pre-prepared dishes according to claim 3, wherein: The reinforcement learning algorithm is used to select the denoising action and output the denoised high-quality image. The specific steps are as follows: Based on industrial noise synthetic images, the global noise complexity of the synthetic images is calculated through grayscale conversion, probability distribution statistics and Shannon entropy formula; Based on multi-dimensional feature statistics and dynamic threshold division, the global noise complexity of the synthetic image is dynamically mapped and output as a three-level noise pattern; Based on the three-level noise pattern, the TD3 reinforcement learning algorithm dynamically selects and executes the optimal denoising action in the denoising action space to generate a high-quality denoising image.

5. The automatic ingredient allocation method for pre-prepared dishes according to claim 4, characterized in that: The global semantic feature vector of the high-quality image after denoising is extracted and the process sequence constraint strength is generated through MLP. The specific steps are as follows: Extract high-level semantic features by performing attention weighting on the denoised high-quality images; Analyze the contextual feature vectors of spatial relationships and interactions in the denoised high-quality images through the self-attention mechanism; Through the task-specific branch network, multi-level feature extraction and fusion are performed on the denoised high-quality image to extract task discriminant features; The temporal feature vector of the dynamic changes and temporal consistency of the denoised high-quality image is captured through the temporal convolutional network; Based on the extracted global semantic feature vector, the process sequence constraint strength between ingredients in the cooking process is calculated through MLP.

6. The automatic ingredient allocation method for pre-prepared dishes according to claim 5, characterized in that: The global semantic feature vector is used as a node and the process sequence constraint strength is used as the edge weight. Through the dynamic graph convolution network, a trans-membrane state timing diagram is constructed and the graph propagation process is optimized through the time smoothing constraint loss formula to generate the timing confidence. The specific steps are as follows: Using global semantic feature vectors as nodes and process sequence constraint strength as edge weights, a cross-modal temporal graph is constructed using the ST-GCN++ dynamic graph convolutional network. The cross-modal time series graph propagation process is optimized using time smoothness constraint loss, and node embedding aggregation is performed on the optimized time series graph; The aggregated graph embedding is mapped to the confidence space through a fully connected layer, and the confidence sequence of each time step is output; The time smoothing constraint is used to perform temporal consistency and weighted fusion processing on the confidence sequence to generate the temporal confidence.

7. The automatic ingredient allocation method for pre-prepared dishes according to claim 6, characterized in that: The specific steps of obtaining the manipulator motion control scheme based on the timing confidence, mapping it using the fuzzy logic control mapping method, generating the manipulator control instructions and executing them are as follows: Based on the statistical analysis of historical confidence distribution data, low confidence threshold and high confidence threshold are defined; When the current confidence value is higher than the high confidence threshold, the robot arm moves according to the original XYZ axis motion trajectory; The current confidence value is between the low confidence threshold and the high confidence threshold. The robot arm needs to fine-tune its position, posture, or grasping action to ensure that it can grasp the material more accurately or make the material fall into the target container more precisely. If the current confidence value is lower than the low confidence threshold, the robot arm will immediately stop the current operation and issue an early warning.

8. An automatic pre-prepared meal ingredient production system based on the automatic pre-prepared meal ingredient production method according to any one of claims 1 to 7, characterized in that: include, Data acquisition module, used to obtain food image sequences and process parameters and perform preprocessing; The anti-noise enhancement module is used to generate industrial noise synthetic images from food images through an adversarial noise generator, select denoising actions through a reinforcement learning algorithm, and output a high-quality denoised image; Semantic feature encoding module, which is used to extract the global semantic feature vector of the denoised high-quality image and generate the process order constraint strength through MLP; The cross-modal timing graph reasoning module uses global semantic feature vectors as nodes and process sequence constraint strengths as edge weights. It constructs a cross-membrane state timing graph through a dynamic graph convolutional network and optimizes the graph propagation process using a time-smoothing constraint loss formula to generate timing confidence. The intelligent action decision and control module is used to obtain the robot arm motion control plan based on the timing confidence, and use the fuzzy logic control mapping method to map it, generate the robot arm control instructions and execute them.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method for automatically allocating ingredients for pre-prepared dishes according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for automatically allocating ingredients for pre-prepared dishes according to any one of claims 1 to 7 are implemented.

Images