AI-driven food safety behavior analysis system

Abstract

The invention relates to the technical field of food safety monitoring, and discloses an AI-driven food safety behavior analysis system. The system comprises a specification analysis module which analyzes a specification text by using a converter model represented by a bidirectional encoder, and adjusts the priority of check items in combination with a frequent item set mining algorithm; the multi-modal data acquisition module is used for integrating the improved target detection model and the specific behavior recognition model to acquire multi-modal data; the behavior recognition module is used for evaluating operation compliance by using technologies such as a space-time diagram convolutional network; and the risk prediction module is used for predicting various risks based on the multi-task learning model. In addition, the system is further provided with a dynamic inspection module which generates a dynamic inspection route based on reinforcement learning. The system can comprehensively supervise food safety, improve supervision efficiency and accuracy, and effectively guarantee public diet safety.

Description

Technical Field

[0001] The present invention relates to the technical field of food safety monitoring, and specifically to an AI-driven food safety behavior analysis system. Background Art

[0002] In today's society, food safety issues have always been the focus of public attention, and their importance is self-evident. From the health protection of individual consumers, to the stable development of the entire food industry, and then to the normal operation of the social and economic order, food safety plays a crucial role. Once a food safety accident occurs, it will not only cause direct harm to the physical health of consumers, trigger diseases and even endanger lives, but may also lead to damage to the reputation of food enterprises and heavy economic losses, thereby affecting the trust and market stability of the entire industry.

[0003] However, there are many defects in the existing food safety supervision methods. The traditional manual inspection mode relies on a large amount of manpower, with low efficiency and easy omissions. When performing tasks, inspectors may be affected by subjective factors, such as fatigue, experience differences, etc., resulting in inaccurate and incomplete identification of food safety hazards. Moreover, manual inspections are often regular sampling inspections and cannot achieve real-time monitoring of the entire process of food production, processing, storage, etc., making many potential safety problems unable to be discovered and handled in a timely manner.

[0004] In terms of data collection, the existing technical means are also relatively single. In most cases, only simple sensors are relied on to collect some basic environmental parameters, such as temperature, humidity, etc., and there is a lack of effective collection methods for multi-dimensional information such as operation behaviors and personnel status. This leads to incomplete data obtained by the data acquirer and makes it difficult to accurately evaluate the food safety situation and predict risks. For example, in the food processing process, key information such as the hand hygiene of operators and whether the food ingredient handling steps are standardized cannot be obtained in real time through the existing data collection means.

[0005] In addition, the implementation effect of food safety regulations is not ideal. Due to the lack of in-depth analysis and effective transformation of the regulatory text, many regulatory provisions are difficult to implement in actual supervision. The regulations in the norms are often relatively general, lacking clear guidance in the implementation process, and it is not clear about the specific inspection focuses and priorities. Moreover, as food safety issues become increasingly complex and diverse, the norms need to be continuously updated and adjusted, but the existing supervision system is difficult to quickly adapt to the changes in the norms and timely adjust the supervision strategies.

[0006] In terms of risk prediction, the accuracy and timeliness of existing methods need to be improved. Most of them are based on simple statistical models or empirical judgments and cannot make full use of massive historical data and real-time data for in-depth analysis. This makes the prediction of microbial contamination risk, operation violation risk, and equipment abnormality risk inaccurate, and effective prevention and control measures cannot be taken in advance, thus increasing the probability of food safety accidents. Summary of the Invention

[0007] The purpose of the present invention is to provide an AI-driven food safety behavior analysis system to solve the problems mentioned in the above background technology.

[0008] To achieve the above purpose, the present invention provides the following technical solution: an AI-driven food safety behavior analysis system, the system includes: A specification parsing module for parsing food safety specification texts, realizing entity extraction and relationship mining through a Transformer model based on bidirectional encoder representations, generating structured rule entries, and dynamically adjusting the priority of inspection items in combination with the frequent item set mining algorithm; A multi-modal data acquisition module integrating an improved object detection model and a bidirectional long short-term memory behavior recognition model combined with a residual network, and collecting multi-modal data through video streams, intelligent morning check instruments, and intelligent sample retention cabinets; A behavior recognition module using a spatio-temporal graph convolutional network to perform temporal modeling on the operation behaviors in the video stream, extracting the key frame sequence of the behavior segment based on a multi-scale feature pyramid, and fusing long-term and short-term behavior dependency relationships through a temporal self-attention mechanism to output a compliance evaluation vector; A risk prediction module based on a hierarchical prediction model for multi-task learning, inputting the structured rule entries, multi-modal data, and compliance evaluation vector into a shared feature encoding layer, and generating joint prediction results of microbial contamination risk, operation violation risk, and equipment abnormality risk through a task-specific gating network.

[0009] Preferably, the specification parsing module realizes entity extraction and relationship mining through a Transformer model based on bidirectional encoder representations, including: Constructing a hierarchical encoder for the specification text, using a bidirectional long short-term memory network to extract the context semantic features of the text, and generating a preliminary entity embedding representation; Performing cross-paragraph correlation analysis on the specification clauses through a multi-head self-attention mechanism to identify the subordination relationship, constraint relationship, and conditional dependency relationship between entities; introducing a contrastive learning strategy to optimize the entity embedding, using the entities within the same clause as positive sample pairs and the entities in different clauses as negative sample pairs to maximize the semantic similarity of the positive sample pairs; Inputting the extracted entities and relationships into a rule graph construction layer, modeling multi-hop rule logic based on a hypergraph structure, and generating a dynamically updatable basic rule library.

[0010] Preferably, the combination of frequent itemset mining algorithms for dynamically adjusting the priority of inspection items includes: Extract the high-frequency co-occurrence patterns of violation operations from historical inspection records to construct a transaction database of violation item sets; Mine multi-level frequent item sets through the FP-Growth algorithm, and adopt a weight decay mechanism to weaken the influence of obsolete violation patterns; Calculate the priority score of inspection items based on the support and lift of item sets, and dynamically generate a priority queue in combination with the real-time risk prediction results.

[0011] Preferably, the multi-modal data acquisition module integrated with an improved object detection model includes: Construct a heterogeneous object detection framework for the food safety scenario, and define multi-level detection targets for device status, operation tools, and ingredient categories; adopt a deformable convolutional network to adapt to detection objects of different scales, and optimize the detection accuracy of small targets through a feature alignment loss function; Bind the detection results with timestamps and device identifiers and input them into the multi-modal feature fusion layer to generate a spatio-temporally aligned joint data representation.

[0012] Preferably, the bidirectional long short-term memory behavior recognition model combined with a residual network includes: Design a depthwise separable residual block to replace the standard convolutional layer; Construct a bidirectional gated recurrent unit chain structure, and aggregate the hidden states of different time steps through skip connections; introduce a temporal attention mechanism to dynamically adjust the feature weights of each frame, and focus on the video segments of key operation stages; Input the output behavior feature vector and the physiological data of the intelligent morning inspection device into the cross-modal contrast learning layer to enhance the correlation modeling of human behavior and physiological state.

[0013] Preferably, the behavior recognition module uses a spatio-temporal graph convolutional network to perform temporal modeling on operation behaviors, including: Define the topological structure of human key points as spatio-temporal graph nodes, and the joint movement trajectory as edge attributes; Simultaneously capture spatial joint relationships and temporal motion patterns through hierarchical graph convolutional operations to generate multi-level spatio-temporal features; Adopt a dynamic graph structure learning algorithm to optimize the node connection weights, and adaptively adjust the topological structure according to the behavior type; Input the final spatio-temporal features into a multi-classifier to output the compliance score and violation type label of the operation steps.

[0014] Preferably, the system further includes: The dynamic inspection module constructs an adaptive inspection path generation strategy based on the reinforcement learning framework. It inputs the risk prediction results and historical inspection records into the policy network, generates a candidate inspection node sequence through Monte Carlo tree search, and optimizes the node relevance based on the graph neural network to output a dynamic inspection route.

[0015] Preferably, the dynamic inspection module constructs an adaptive inspection path generation strategy based on the reinforcement learning framework, including: Define the state space as a joint encoding vector of the risk prediction result, the device distribution map, and the inspector's position; design a composite reward function, including risk coverage reward, path efficiency reward, and inspection intensity balance reward; Use the double delayed policy gradient algorithm to train the policy network, and stabilize the training process through the target network smoothing technique; construct a curriculum learning mechanism, initially limit the inspection area range, and gradually expand to full-scenario multi-object collaborative inspection.

[0016] Preferably, the generation of the candidate inspection node sequence through Monte Carlo tree search includes: Construct a decision tree for the inspection path, where each node represents a device or area to be inspected, and the edge weights are dynamically updated by the risk prediction values; use the upper confidence bound algorithm to balance exploration and exploitation, and preferentially visit the path branches with high potential rewards; Introduce a parallel simulation mechanism to synchronously generate multiple candidate paths and evaluate the comprehensive scores of the paths through the graph neural network; Input the optimal path and the sub-optimal path into the integrated optimization layer to generate a dynamic inspection plan that takes into account both efficiency and risk coverage.

[0017] Preferably, the video stream includes kitchen operation monitoring, food ingredient storage monitoring, and tableware cleaning monitoring.

[0018] Compared with the prior art, the beneficial effects of the present invention are: The AI-driven food safety behavior analysis system proposed by the present invention brings significant advantages in many aspects to solve the difficult problems of food safety supervision. In terms of specification parsing, entity extraction and relationship mining are realized through the Transformer model based on bidirectional encoder representations, which can deeply analyze food safety specification texts. A hierarchical encoder for specification texts is constructed, and bidirectional long short-term memory networks are used to extract context semantic features, which can accurately identify key entities in specification clauses. Contents such as various food production standards and operation specification requirements can be accurately extracted. The multi-head self-attention mechanism is used for cross-paragraph correlation analysis, which can clearly sort out the subordination relationships, constraint relationships and conditional dependence relationships between entities. For example, the connections between specific supervision standards corresponding to different food processing links can be clarified. A contrastive learning strategy is introduced to optimize entity embeddings, further improving the accuracy of entity extraction and relationship mining, making the conversion of specification clauses into structured rule entries more accurate, enabling supervisors to quickly grasp the core points of the specifications, and providing a solid specification basis for subsequent supervision work. At the same time, combined with the frequent item set mining algorithm to dynamically adjust the priority of inspection items, the high-frequency co-occurrence patterns of illegal operations are extracted from historical inspection records, and the FP-Growth algorithm is used to mine multi-order frequent item sets, which can timely pay attention to the links that often cause safety problems, and dynamically generate a priority queue according to the real-time risk prediction results, ensuring that supervision resources are preferentially invested in high-risk areas and greatly improving the supervision efficiency.

[0019] The design of the multi-modal data acquisition module also has great advantages. An improved object detection model is integrated to construct a heterogeneous object detection framework for food safety scenarios, and multi-level detection targets are defined. Equipment status, operation tools, food ingredient categories, etc. can all be accurately detected. The deformable convolutional network is used to adapt to detection objects of different scales, and the small object detection accuracy is optimized through the feature alignment loss function, ensuring the accurate identification of various targets and not missing any potential risk points. Combining the bidirectional long short-term memory behavior recognition model with the residual network, designing a depthwise separable residual block to replace the standard convolutional layer, effectively reducing the model's computational complexity while improving the feature extraction ability. A bidirectional gated recurrent unit chain structure is constructed and a temporal attention mechanism is introduced, which can focus on video segments of key operation stages and accurately identify personnel operation behaviors. The behavior feature vector and the physiological data of the intelligent morning inspection instrument are input into the cross-modal contrastive learning layer to realize the correlation modeling of personnel behavior and physiological state. For example, it can be found whether more illegal operations will occur when the operator is physically fatigued, providing more dimensions of information for preventing safety accidents.

[0020] The behavior recognition module uses a spatio-temporal graph convolutional network to perform temporal modeling on the operation behaviors in the video stream. It defines the topological structure of human key points as spatio-temporal graph nodes and the joint movement trajectories as edge attributes. Through hierarchical graph convolutional operations, it simultaneously captures the spatial joint relationships and temporal movement patterns, generating multi-level spatio-temporal features. The dynamic graph structure learning algorithm is used to optimize the node connection weights, which can adaptively adjust the topological structure according to different behavior types, making the behavior recognition more accurate. The finally output compliance evaluation vector can intuitively reflect whether the operation steps are compliant and the types of violations, helping the supervisors quickly locate the problems and correct the violations in a timely manner.

[0021] The risk prediction module is based on a hierarchical prediction model of multi-task learning. It inputs structured rule entries, multi-modal data, and the compliance evaluation vector into the shared feature encoding layer, and generates joint prediction results of microbial contamination risk, operation violation risk, and equipment anomaly risk through a task-specific gating network. This prediction method of multi-dimensional data fusion greatly improves the accuracy and timeliness of prediction compared with the traditional prediction methods based on a single data source. The supervisors can take corresponding prevention and control measures in advance according to the prediction results, reducing the probability of food safety accidents.

[0022] In addition, the dynamic inspection module in the system constructs an adaptive inspection path generation strategy based on the reinforcement learning framework. It inputs the risk prediction results and historical inspection records into the policy network, generates a candidate inspection node sequence through Monte Carlo tree search, and optimizes the node relevance based on the graph neural network, outputting a dynamic inspection route. By defining a reasonable state space and a composite reward function, training the policy network using the double-delayed policy gradient algorithm, and constructing a curriculum learning mechanism, the planning of the inspection path is made more scientific and reasonable. It can not only improve the risk coverage rate, ensure that high-risk areas are inspected, but also improve the path efficiency, reduce the workload of the inspectors, and at the same time ensure the balance of inspection intensity, avoiding the occurrence of inspection loopholes or over-inspection.

[0023] The AI-driven food safety behavior analysis system of the present invention realizes the all-round and multi-level supervision of food safety through the collaborative work of each module, effectively makes up for the deficiencies of the traditional supervision methods, greatly improves the efficiency and accuracy of food safety supervision, and provides strong technical support for ensuring the public's food safety. Brief Description of the Drawings

[0024] Figure 1 It is the working principle diagram of the AI-driven food safety behavior analysis system described in the present invention; Figure 2 It is the flow chart of entity extraction and relationship mining of the specification parsing module; Figure 3 It is the working flow chart of the improved object detection model of the multi-modal data acquisition module; Figure 4 It is the flowchart of the dynamic inspection module. Specific implementation manners

[0025] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0026] Please refer to Figures 1 - 4 , the AI-driven food safety behavior analysis system of the present invention aims to comprehensively and accurately analyze food safety behaviors by using artificial intelligence technology to ensure food safety. Its overall implementation solution is as follows: The specification parsing module, whose main function is to deeply parse the food safety specification text. By means of the Transformer model based on bidirectional encoder representations, entity extraction and relationship mining in the specification text are realized, and then structured rule entries are generated. On this basis, combined with the frequent item set mining algorithm, the priority of inspection items is dynamically adjusted according to the actual situation, making the inspection work more targeted and efficient.

[0027] The multi-modal data acquisition module integrates an improved object detection model and a bidirectional long short-term memory behavior recognition model combined with a residual network. Through video streams, intelligent morning inspection instruments and intelligent sample storage cabinets, multi-modal data are collected from multiple dimensions, providing rich and comprehensive data support for subsequent analysis. Among them, the video stream covers multiple key scenarios such as back kitchen operation monitoring, food ingredient storage monitoring and tableware cleaning monitoring; the intelligent sample storage cabinet is equipped with an image recognition and weight sensing unit, which can automatically record the storage time and environmental parameters of the sampled food ingredients; at the same time, personnel identification information, such as job numbers, health certificate status and training records, is also collected as important data.

[0028] The behavior recognition module uses a spatio-temporal graph convolutional network to perform temporal modeling on the operation behaviors in the video stream. Based on the multi-scale feature pyramid, the key frame sequence of the behavior segment is extracted, and then the long-term and short-term behavior dependence relationship is fused through the temporal self-attention mechanism. Finally, a compliance evaluation vector is output to judge whether the operation behavior conforms to the food safety specification.

[0029] The risk prediction module, based on a hierarchical prediction model of multi-task learning, inputs the structured rule entries generated by the specification parsing module, the multi-modal data collected by the multi-modal data acquisition module, and the compliance evaluation vector output by the behavior recognition module into the shared feature encoding layer. Through the task-specific gating network, joint prediction results of microbial contamination risk, operation violation risk and equipment anomaly risk are generated, providing a scientific basis for food safety risk prevention and control.

[0030] The implementation of the present invention will be further described below in conjunction with Embodiments 1 to 6.

[0031] Embodiment 1: In this embodiment, the specific process of entity extraction and relationship mining implemented by the specification parsing module through the Transformer model based on bidirectional encoder representations will be mainly described.

[0032] Construct a hierarchical encoder for the specification text. The bidirectional long short-term memory network (Bi-LSTM) is used to process the specification text. The bidirectional long short-term memory network has the advantage of being able to learn the semantic features of the forward and backward contexts of the text simultaneously. Assume that the input specification text is a series of word vector sequences , and through the calculation of the bidirectional long short-term memory network, the hidden state vector corresponding to each word will be obtained. These hidden state vectors initially contain the context semantic features of the text, and then a preliminary entity embedding representation is generated. For example, in a food safety specification text, for the sentence "During food processing, operators must wear masks", the bidirectional long short-term memory network can learn the meanings of words such as "operators" and "masks" in the entire text context and generate corresponding preliminary entity embedding representations.

[0033] Perform cross-paragraph correlation analysis on the specification clauses through the multi-head self-attention mechanism. The multi-head self-attention mechanism can capture the features of the text from different representation subspaces and better identify the relationships between entities. In this process, the membership relationship, constraint relationship, and conditional dependence relationship between entities can be identified. For example, in the specification clauses, there may be a membership relationship between "food enterprises" and "production workshops", and a constraint relationship between "food processing operations" and "hygiene standards". To further optimize the entity embedding, a contrastive learning strategy is introduced. Entities within the same clause are used as positive sample pairs, and entities in different clauses are used as negative sample pairs. By maximizing the semantic similarity of the positive sample pairs, the entity embedding can more accurately reflect the semantic features of the entities. Let the positive sample pair be , and the negative sample pair be . Through the contrastive learning algorithm, the distance between the positive sample pairs is made as small as possible, and the distance between the negative sample pairs is made as large as possible.

[0034] Input the extracted entities and relationships into the rule graph construction layer. Model the multi-hop rule logic based on the hypergraph structure, which can well represent complex relationship networks. In this structure, nodes represent entities and edges represent the relationships between entities, and in this way, a dynamically updatable basic rule library is generated. As new food safety regulations are introduced or existing regulations are revised, the basic rule library can be updated accordingly to ensure the timeliness and accuracy of rule parsing.

[0035] Embodiment 2: This embodiment details the specific implementation steps of dynamically adjusting the priority of inspection items in the rule parsing module by combining the frequent item set mining algorithm.

[0036] Extract the high-frequency co-occurrence patterns of illegal operations from historical inspection records to construct a transaction database of illegal item sets. Historical inspection records contain a large amount of data on food safety inspections. For example, in past inspections, it was found that the two illegal operations of "not washing tableware according to the specified time" and "unsanitary tableware storage environment" often occurred simultaneously. Organize this illegal operation information into the form of a transaction database, and each record represents the set of illegal operations that occurred in one inspection.

[0037] Mine multi-order frequent item sets through the FP-Growth algorithm. The FP-Growth algorithm is an efficient frequent item set mining algorithm that compresses data by constructing a frequent pattern tree (FP-Tree) to improve the mining efficiency. During the mining process, a weight decay mechanism is adopted to weaken the influence of obsolete illegal patterns. Over time, due to reasons such as strengthened supervision or the improvement of enterprises themselves, the frequency of some illegal patterns may gradually decrease, and their impact on the current food safety risks also decreases accordingly. Through the weight decay mechanism, the weights of these obsolete illegal patterns in the mining results can be reduced. Suppose a certain illegal pattern at time has a weight of , and the weight decay coefficient is , then after a period of time , its weight is updated to .

[0038] Calculate the priority score of the inspection item based on the support and lift of the item set. Support represents the frequency of an item set appearing in the transaction database, and lift measures the degree of influence of the appearance of one item set on the appearance of another item set. Suppose the support of item set is , and the lift is , the formula for calculating the priority score of the inspection item is , where and is the weight coefficient, which is adjusted according to the actual situation. A priority queue is dynamically generated in combination with the real-time risk prediction results. If within a certain time period, it is predicted that the risk of microbial contamination in a certain type of food processing area is relatively high, and some violations related to this area occur frequently, then the inspection items related to these violations will be ranked higher in the priority queue so that the inspectors can conduct inspections preferentially.

[0039] Embodiment 3: This embodiment mainly describes the specific implementation method of integrating an improved object detection model in the multi-modal data acquisition module.

[0040] Construct a heterogeneous object detection framework for the food safety scenario. In this framework, multi-level detection targets of device status, operation tools, and food ingredient categories are defined. For example, the device status can include whether the food processing equipment is operating normally, whether the temperature is within a reasonable range, etc.; the operation tools can be knives, cutting boards, etc.; the food ingredient categories can be divided into meat, vegetables, etc. The deformable convolutional network (DCN) is used to adapt to detection objects of different scales. The convolutional kernel size of the traditional convolutional network is fixed, and the detection effect for objects of different scales is not good. While the deformable convolutional network can dynamically adjust the sampling position of the convolutional kernel according to the shape and size of the object, so as to better adapt to detection objects of different scales. The small object detection accuracy is optimized through the feature alignment loss function. Small objects occupy fewer pixels in the image, have unclear features, and are difficult to detect. The feature alignment loss function can make the model pay more attention to the features of small objects and improve the detection accuracy of small objects. Let the feature alignment loss function be , and optimize the model by minimizing .

[0041] After binding the detection result with the timestamp and device identifier, it is input into the multi-modal feature fusion layer. The timestamp records the time when the detection occurs, and the device identifier can uniquely determine the detected device. For example, when it is detected that the temperature of a certain food processing equipment is abnormal, this detection result is input into the multi-modal feature fusion layer together with the corresponding timestamp and the unique identifier of the device (such as the device number). In the multi-modal feature fusion layer, the detection result is fused with data of other modalities (such as image information in the video stream, physiological data of personnel collected by the intelligent morning inspection instrument, etc.) to generate a spatio-temporal aligned joint data representation, providing a more comprehensive and accurate data basis for subsequent analysis.

[0042] Embodiment 4: This embodiment elaborates in detail the specific implementation details of the bidirectional long short-term memory behavior recognition model combined with the residual network in the multi-modal data acquisition module.

[0043] Replace the standard convolutional layer with a depthwise separable residual block. The depthwise separable residual block can effectively reduce the number of model parameters, improve the computational efficiency, and at the same time maintain the performance of the model. In the traditional convolutional layer, the convolution operation processes both the channels and the space of the input feature map simultaneously, resulting in a large amount of computation. While the depthwise separable residual block decomposes the convolution operation into a depthwise convolution and a pointwise convolution. The depthwise convolution only performs convolution on the spatial dimension, and the pointwise convolution performs convolution on the channel dimension, which can greatly reduce the amount of computation. For example, when processing image data in a video stream, using the depthwise separable residual block can speed up the processing speed of the model without losing too much accuracy.

[0044] Construct a bidirectional gated recurrent unit (Bi-GRU) chain structure. The bidirectional gated recurrent unit can learn the forward and backward information of the time series simultaneously and aggregate the hidden states of different time steps through skip connections. When processing a video stream, each frame image in the video can be regarded as the input of a time step, and the bidirectional gated recurrent unit can capture the correlation information between different time steps. Introduce a temporal attention mechanism to dynamically adjust the feature weights of each frame. In a video, the frames in some key operation stages are more important for judging whether the personnel's behavior is compliant. Through the temporal attention mechanism, the video segments in these key operation stages can be focused on. Let the input video frame feature sequence be , and calculate the weight of each frame through the temporal attention mechanism , and the weighted feature representation is .

[0045] Input the output behavior feature vector and the physiological data of the intelligent morning inspection device into the cross-modal contrast learning layer. The intelligent morning inspection device can collect physiological data such as the body temperature and heart rate of personnel, and there may be a certain correlation between these physiological data and the behavior of personnel. Through the cross-modal contrast learning layer, the correlation modeling between personnel behavior and physiological state can be enhanced. For example, if it is found that an operator's behavior is abnormal during food processing and at the same time their physiological data (such as too high body temperature) is also abnormal, then the relationship between the two can be further analyzed to determine whether there is a potential food safety risk.

[0046] Example 5: This example specifically illustrates the process of using a spatio-temporal graph convolutional network to perform temporal modeling on operation behaviors in the behavior recognition module.

[0047] Define the topological structure of human body key points as the nodes of the spatio-temporal graph, and the joint movement trajectory as the edge attribute. Human body key points can be obtained through human pose estimation technology, such as key points like the head, hands, feet, etc. The connection relationships between these key points constitute the topological structure of the spatio-temporal graph, while the movement trajectories of joints at different times serve as the attributes of the edges. For example, in food processing operations, the movement trajectory of the operator's hand can reflect their operation actions. By taking the hand key points as nodes and the hand joint movement trajectory as the edge attribute, a spatio-temporal graph is constructed.

[0048] Simultaneously capture spatial joint relationships and temporal movement patterns through hierarchical graph convolution operations. Hierarchical graph convolution operations can perform convolution calculations on the spatio-temporal graph at different levels, thereby extracting spatio-temporal features at different scales. In each layer of graph convolution operation, new node features and edge features are obtained through convolution calculations between the convolution kernel and the features of nodes and edges. For example, in the first layer of graph convolution operation, local spatial relationships between adjacent joints and movement patterns within a short time can be captured; in higher-layer graph convolution operations, more global spatial relationships and long-term movement patterns can be captured, generating multi-level spatio-temporal features.

[0049] Adopt a dynamic graph structure learning algorithm to optimize the node connection weights. Different behavior types may correspond to different joint movement patterns and spatial relationships. Through the dynamic graph structure learning algorithm, the topological structure can be adaptively adjusted according to the behavior type. For example, in the behaviors of cutting vegetables and stir-frying, the movement patterns of the hand joints and their spatial relationships with other joints are different. The dynamic graph structure learning algorithm can adjust the node connection weights based on these differences, enabling the model to better adapt to different behavior types.

[0050] Input the final spatio-temporal features into a multi-classifier. The multi-classifier can output the compliance score of the operation steps and the violation type label based on the input spatio-temporal features. For example, the multi-classifier can determine whether a certain operation step complies with food safety regulations. If not, it can also output specific violation types, such as "not washing hands as required" and "wrong ingredient processing order", providing clear information for subsequent risk prevention and control and supervision.

[0051] Example 6: This example details the specific implementation method of the dynamic inspection module in the system, including the process of constructing an adaptive inspection path generation strategy based on the reinforcement learning framework and generating a candidate inspection node sequence through Monte Carlo tree search.

[0052] Build an adaptive inspection path generation strategy based on the reinforcement learning framework. First, define the state space as the joint encoding vector of the risk prediction result, the equipment distribution map, and the inspector's location. The risk prediction result reflects the degree of food safety risks existing in different regions and equipment; the equipment distribution map shows the distribution of various equipment in the food processing site; the inspector's location records the current location of the inspector. Jointly encode this information as the state input of the reinforcement learning model. Design a composite reward function, including risk coverage reward, path efficiency reward, and inspection intensity balance reward. The risk coverage reward encourages the inspector to prioritize the inspection of high-risk regions and equipment to improve the prevention and control effect of food safety risks; the path efficiency reward prompts the inspector to choose a more efficient inspection path to reduce inspection time and costs; the inspection intensity balance reward ensures that the inspection intensity of the inspector on different regions and equipment is relatively balanced, avoiding over-inspecting some regions while neglecting others. Let the risk coverage reward be , the path efficiency reward be , the inspection intensity balance reward be , and the composite reward function , where is the weight coefficient, which is adjusted according to the actual situation.

[0053] Train the policy network using the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm. The TD3 algorithm can effectively reduce the estimation error of the policy gradient and improve the stability of training. Stabilize the training process through the target network smoothing technique. The target network can be updated regularly, and its parameters have a certain delay with the parameters of the policy network, thereby reducing the fluctuations in the training process. Build a curriculum learning mechanism. Initially, limit the inspection area range and gradually expand it to full-scenario multi-object collaborative inspection. In the initial stage of training, due to the limited understanding of the environment by the model, if full-scenario inspection is directly carried out, it may lead to poor inspection effects. Through the curriculum learning mechanism, first let the model learn and train in a smaller area. As the model performance improves, gradually expand the inspection area range, and finally achieve full-scenario multi-object collaborative inspection.

[0054] Generate candidate inspection node sequences through Monte Carlo tree search. Construct a decision tree for the inspection path, where each node represents a device or area to be inspected, and the edge weights are dynamically updated by the risk prediction values. The higher the risk prediction value of a path branch, the greater the edge weight, indicating a higher potential return for that path branch. The Upper Confidence Bound (UCB) algorithm is used to balance exploration and exploitation, preferentially accessing path branches with high potential returns. The UCB algorithm can find a balance between exploring new paths and exploiting existing experience, avoiding the model from falling into local optimal solutions. Introduce a parallel simulation mechanism to synchronously generate multiple candidate paths and evaluate the comprehensive scores of the paths through a graph neural network. The graph neural network can analyze the structure and features of the paths and evaluate the comprehensive scores of each path, including aspects such as risk coverage and path efficiency. Input the optimal path and the sub-optimal path into the integrated optimization layer to generate a dynamic inspection plan that takes into account both efficiency and risk coverage. In this way, while ensuring the inspection effect, the inspection efficiency can be improved and the inspection resources can be reasonably allocated.

[0055] It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device.

[0056] Although the embodiments of the present invention have been shown and described, it will be understood by those of ordinary skill in the art that various changes, modifications, substitutions and variations can be made therein without departing from the principles and spirit of the present invention, and the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An AI-driven food safety behavior analysis system, characterized in that, Including: A specification parsing module, which is used to parse food safety specification texts, implement entity extraction and relationship mining through a Transformer model based on bidirectional encoder representations, generate structured rule entries, and dynamically adjust the priority of inspection items in combination with the frequent item set mining algorithm; A multi-modal data acquisition module, which integrates an improved object detection model and a bidirectional long short-term memory behavior recognition model combined with a residual network, and acquires multi-modal data through video streams, intelligent morning check instruments, and intelligent sample retention cabinets; A behavior recognition module, which uses a spatio-temporal graph convolutional network to perform temporal modeling on the operation behaviors in the video stream, extracts the key frame sequence of the behavior segment based on a multi-scale feature pyramid, and fuses the long-term and short-term behavior dependencies through a temporal self-attention mechanism to output a compliance evaluation vector; A risk prediction module, which is a hierarchical prediction model based on multi-task learning. It inputs the structured rule entries, multi-modal data, and compliance evaluation vector into a shared feature encoding layer, and generates a joint prediction result of microbial contamination risk, operation violation risk, and equipment anomaly risk through a task-specific gating network.

2. The AI-driven food safety behavior analysis system according to claim 1, wherein The above-mentioned specification parsing module realizes entity extraction and relationship mining through a Transformer model based on bidirectional encoder representations, including: Constructing a hierarchical encoder for the specification text, using a bidirectional long short-term memory network to extract the context semantic features of the text, and generating a preliminary entity embedding representation; Performing cross-paragraph correlation analysis on the specification clauses through a multi-head self-attention mechanism to identify the membership relationship, constraint relationship, and conditional dependence relationship between entities; introducing a contrastive learning strategy to optimize the entity embedding, using the entities within the same clause as positive sample pairs and the entities in different clauses as negative sample pairs to maximize the semantic similarity of the positive sample pairs; Inputting the extracted entities and relationships into a rule graph construction layer, modeling the multi-hop rule logic based on a hypergraph structure, and generating a dynamically updatable basic rule library.

3. The AI-driven food safety behavior analysis system according to claim 2, wherein The above-mentioned dynamic adjustment of the priority of inspection items in combination with the frequent item set mining algorithm includes: Extracting the high-frequency co-occurrence patterns of violation operations from historical inspection records and constructing a transaction database of violation item sets; Mining multi-order frequent item sets through the FP-Growth algorithm, and using a weight decay mechanism to weaken the influence of obsolete violation patterns; Calculating the priority score of inspection items based on the support and lift of item sets, and dynamically generating a priority queue in combination with the real-time risk prediction results.

4. The AI-driven food safety behavior analysis system according to claim 1, wherein The above-mentioned multi-modal data acquisition module integrates an improved object detection model, including: Constructing a heterogeneous object detection framework for food safety scenarios, defining multi-level detection targets for equipment status, operation tools, and ingredient categories; using a deformable convolutional network to adapt to detection objects of different scales, and optimizing the detection accuracy of small targets through a feature alignment loss function; Binding the detection results with timestamps and device identifiers and inputting them into a multi-modal feature fusion layer to generate a spatio-temporally aligned joint data representation.

5. The AI-driven food safety behavior analysis system according to claim 4, wherein The above-mentioned bidirectional long short-term memory behavior recognition model combined with a residual network includes: Designing a depthwise separable residual block to replace the standard convolutional layer; Constructing a chain structure of bidirectional gated recurrent units, aggregating the hidden states of different time steps through skip connections; introducing a temporal attention mechanism to dynamically adjust the feature weights of each frame, and focusing on the video segments in the key operation stages; The output behavioral feature vector and the physiological data of the intelligent morning health check device are input into the cross-modal contrast learning layer to enhance the correlation modeling between the personnel's behavior and physiological state.

6. The AI-driven food safety behavior analysis system according to claim 1, wherein The behavior recognition module uses a spatio-temporal graph convolutional network to perform temporal modeling on the operation behavior, including: Defining the topological structure of human key points as spatio-temporal graph nodes and the joint movement trajectory as edge attributes; Simultaneously capturing the spatial joint relationship and the temporal movement pattern through hierarchical graph convolution operations to generate multi-level spatio-temporal features; Adopting a dynamic graph structure learning algorithm to optimize the node connection weights and adaptively adjust the topological structure according to the behavior type; Inputting the final spatio-temporal features into a multi-classifier to output the compliance score of the operation steps and the violation type label.

7. The AI-driven food safety behavior analysis system according to claim 1, wherein The system further includes: A dynamic inspection module that constructs an adaptive inspection path generation strategy based on a reinforcement learning framework. The risk prediction result and the historical inspection record are input into the policy network. Candidate inspection node sequences are generated through Monte Carlo tree search, and the node relevance is optimized based on a graph neural network to output a dynamic inspection route.

8. The AI-driven food safety behavior analysis system according to claim 1, wherein The dynamic inspection module constructs an adaptive inspection path generation strategy based on a reinforcement learning framework, including: Defining the state space as a joint encoding vector of the risk prediction result, the device distribution map, and the inspector's position; designing a composite reward function that includes a risk coverage reward, a path efficiency reward, and an inspection intensity balance reward; Using the double-delayed policy gradient algorithm to train the policy network and stabilizing the training process through the target network smoothing technique; constructing a curriculum learning mechanism that initially limits the inspection area range and gradually expands to full-scene multi-object collaborative inspection.

9. The AI-driven food safety behavior analysis system according to claim 8, wherein The generation of candidate inspection node sequences through Monte Carlo tree search includes: Constructing a decision tree for the inspection path, where each node represents a device or area to be inspected, and the edge weights are dynamically updated by the risk prediction value; using the upper confidence bound algorithm to balance exploration and exploitation and preferentially accessing the path branches with high potential rewards; Introducing a parallel simulation mechanism to simultaneously generate multiple candidate paths and evaluate the comprehensive scores of the paths through a graph neural network; Inputting the optimal path and the sub-optimal path into the integrated optimization layer to generate a dynamic inspection plan that takes into account both efficiency and risk coverage.

10. The AI-driven food safety behavior analysis system according to claim 1, characterized in that The video stream includes kitchen operation monitoring, food ingredient storage monitoring, and tableware cleaning monitoring.

Images