Bag collecting bag turning operation identification method, device and equipment and storage medium

Abstract

The invention relates to the technical field of intelligent logistics, in particular to a collection bag turning operation identification method, device and equipment and a storage medium, and the method comprises the steps: firstly obtaining a collection bag labeling video frame set and a human body labeling video frame set, and calling a collection bag detection model to carry out collection bag detection on the collection bag labeling video frame set; calling a human body posture detection model to carry out human body posture detection on the human body labeling video frame set to obtain a human body detection result set, carrying out timestamp association processing on the set packaging bag detection result set and the human body detection result set, and carrying out operation motion track generation processing on the obtained joint detection result set. The method comprises the steps of obtaining an interaction graph feature set, calling a package turning action recognition model to carry out package turning action recognition and screening on the interaction graph feature set, and finally calling an anomaly detection model to carry out package turning anomaly detection judgment on an obtained target package turning recognition result set to obtain a package turning anomaly detection result set so as to improve the package turning recognition accuracy of the set package.

Description

Technical Field

[0001] This invention relates to the field of intelligent logistics technology, and in particular to a method, apparatus, equipment and storage medium for recognizing bag flipping operations. Background Technology

[0002] In logistics sorting centers and other operational scenarios, the bag flipping operation is a key process to ensure material sorting efficiency and bag recycling. Its core purpose is to completely empty the residual materials inside the bag by turning the bag inside out, while providing the basic conditions for subsequent operations such as liner replacement.

[0003] Currently, compliance supervision of bag-turning operations mainly relies on manual on-site inspections or post-event video review. These methods suffer from high labor costs and low efficiency, and are easily missed due to fluctuations in human attention. Furthermore, existing video analysis-based supervision systems are mostly built on simple target detection or behavior classification technologies. Their technical architecture can only achieve basic positioning of the bags and operators and coarse-grained behavior identification, making it difficult to accurately capture fine operational features such as the bag's turning angle and hand movement trajectory during the bag-turning operation, thus failing to meet the need for accurate determination of compliance of bag-turning operations. Summary of the Invention

[0004] In order to overcome the shortcomings of the prior art, the present invention aims to provide a method, device, equipment and storage medium for recognizing bag turning operations, which aims to improve the accuracy of bag turning recognition and reduce the false alarm rate through multimodal feature fusion.

[0005] The first aspect of this invention provides a method for recognizing bag-folding operations, comprising: acquiring a set of labeled video frames of bags and a set of labeled video frames of humans; calling a pre-trained bag detection model to perform bag detection on the set of labeled video frames of bags, obtaining a set of bag detection results; calling a pre-trained human posture detection model to perform human posture detection on the set of labeled video frames of humans, obtaining a set of human detection results; using a temporal matching algorithm to perform timestamp association processing on the set of bag detection results and the set of human detection results, obtaining a joint detection result set; performing operation motion trajectory generation processing on the joint detection result set, obtaining an interaction graph feature set; calling a pre-trained bag-folding action recognition model to perform bag-folding action recognition and filtering on the interaction graph feature set, obtaining a target bag-folding recognition result set; and calling a pre-trained anomaly detection model to perform bag-folding anomaly detection judgment on the target bag-folding recognition result set, obtaining a bag-folding anomaly detection result set.

[0006] Optionally, in a first implementation of the first aspect of the present invention, obtaining the set of labeled video frames for bags and the set of labeled video frames for humans includes: obtaining a video to be tested; using a video frame extraction tool to extract the video to be tested frame by frame in chronological order to obtain multiple video frames to be tested; using an annotation tool to annotate each of the video frames to be tested to obtain multiple labeled video frames for bags and multiple labeled video frames for humans; sorting and integrating each labeled video frame for bags based on the chronological order to obtain the set of labeled video frames for bags; and sorting and integrating each labeled video frame for humans based on the chronological order to obtain the set of labeled video frames for humans.

[0007] Optionally, in a second implementation of the first aspect of the present invention, the bag detection model includes a backbone feature extraction module, a first feature fusion module, and a head detection module, wherein the backbone feature extraction module, the first feature fusion module, and the head detection module are connected sequentially; the step of calling the pre-trained bag detection model to perform bag detection on the bag-annotated video frame set to obtain a bag detection result set includes: performing multi-scale image feature extraction on the bag-annotated video frame set based on the backbone feature extraction module to obtain a multi-scale bag image feature set; performing feature fusion enhancement processing on the multi-scale bag image feature set based on the first feature fusion module to obtain a fused bag image feature set; and performing bag bounding box coordinate and category probability prediction on the fused bag image feature set based on the head detection module to obtain the bag detection result set.

[0008] Optionally, in a third implementation of the first aspect of the present invention, the human pose detection model includes a multi-resolution convolution module, a second feature fusion module, and a keypoint prediction module, wherein the multi-resolution convolution module, the second feature fusion module, and the keypoint prediction module are connected sequentially; the step of calling the pre-trained human pose detection model to perform human pose detection on the human-annotated video frame set to obtain a human detection result set includes: extracting multi-scale human features from the human-annotated video frame set based on the multi-resolution convolution module to obtain a multi-scale human image feature set; performing multi-scale feature fusion processing on the multi-scale human image feature set based on the second feature fusion module to obtain a fused human image feature set; and performing human keypoint confidence heatmap prediction on the fused human image feature set based on the keypoint prediction module to obtain the human detection result set.

[0009] Optionally, in a fourth implementation of the first aspect of the present invention, the step of performing motion trajectory generation processing on the joint detection result set to obtain an interaction graph feature set includes: obtaining the bag bounding box coordinates and human key point coordinate sets corresponding to each video frame based on the joint detection result set; determining the bag opening region coordinate range and bag bottom region coordinate range corresponding to each video frame based on the bag bounding box coordinates; performing Euclidean distance calculation processing based on the human key point coordinate set, bag opening region coordinate range, and bag bottom region coordinate range corresponding to each video frame to obtain multiple node association results; constructing interaction graph features corresponding to each video frame based on each node association result using a trajectory encoder; and integrating multiple interaction graph features based on the time sequence to obtain the interaction graph feature set.

[0010] Optionally, in a fifth implementation of the first aspect of the present invention, the bag-flipping action recognition model includes a dual-path feature extraction module, a dual-path feature fusion module, a bag-flipping action recognition module, and a filtering output module. The dual-path feature extraction module, the dual-path feature fusion module, the bag-flipping action recognition module, and the filtering output module are connected sequentially. The dual-path feature extraction module includes a spatial feature extraction submodule and a temporal feature extraction submodule, which are connected to each other. The step of calling the pre-trained bag-flipping action recognition model to perform bag-flipping action recognition and filtering on the interaction graph feature set to obtain a target bag-flipping recognition result set includes: based on the spatial feature extraction submodule, analyzing the interaction graph features... The process involves: extracting spatial detail features to obtain a spatial detail feature set; extracting temporal change features from the interaction graph feature set using the temporal feature extraction submodule to obtain a temporal change feature set; merging the spatial detail feature set and the temporal change feature set using the dual-path feature fusion module to obtain a fused feature set; predicting the probability of bag rummaging using the fused feature set using the bag rummaging action recognition module to obtain an initial bag rummaging recognition result set, which includes a bag rummaging action probability dataset; and finally, using the filtering output module, filtering the initial bag rummaging recognition result set based on the bag rummaging action probability dataset to obtain the target bag rummaging recognition result set.

[0011] Optionally, in a sixth implementation of the first aspect of the present invention, the anomaly detection model includes a feature encoding module, an Euclidean distance calculation module, and an anomaly detection module, wherein the feature encoding module, the Euclidean distance calculation module, and the anomaly detection module are connected sequentially; the step of calling the pre-trained anomaly detection model to perform anomaly detection judgment on the target bag-crossing recognition result set to obtain the bag-crossing anomaly detection result set includes: obtaining an interaction graph feature set corresponding to the target bag-crossing recognition result set; inputting the interaction graph feature set into the feature encoding module for deep encoding processing to obtain a high-dimensional feature vector set; performing Euclidean distance calculation processing on the high-dimensional feature vector set according to a preset compliance prototype vector through the Euclidean distance calculation module to obtain multiple feature distance values; and comparing each of the feature distance values ​​with a preset anomaly threshold through the anomaly detection module to obtain the bag-crossing anomaly detection result set.

[0012] A second aspect of the present invention provides a bag-flipping operation recognition device, comprising: a data acquisition module for acquiring a set of labeled video frames of bags and a set of labeled video frames of humans; a bag detection module for calling a pre-trained bag detection model to perform bag detection on the set of labeled video frames of bags, obtaining a set of bag detection results; a human detection module for calling a pre-trained human posture detection model to perform human posture detection on the set of labeled video frames of humans, obtaining a set of human detection results; a result association module for performing timestamp association processing on the set of bag detection results and the set of human detection results using a time-series matching algorithm, obtaining a joint detection result set; a trajectory generation module for performing operation motion trajectory generation processing on the joint detection result set, obtaining an interaction graph feature set; a bag-flipping action recognition module for calling a pre-trained bag-flipping action recognition model to perform bag-flipping action recognition and filtering on the interaction graph feature set, obtaining a target bag-flipping recognition result set; and a bag-flipping anomaly detection module for calling a pre-trained anomaly detection model to perform bag-flipping anomaly detection judgment on the target bag-flipping recognition result set, obtaining a bag-flipping anomaly detection result set.

[0013] A third aspect of the present invention provides a bag-turning operation recognition device, the bag-turning operation recognition device comprising: a memory and at least one processor, the memory storing instructions; at least one processor calling the instructions in the memory to cause the bag-turning operation recognition device to perform each step of the bag-turning operation recognition method described in any of the preceding claims.

[0014] A fourth aspect of the present invention provides a computer-readable storage medium storing instructions that, when executed by a processor, implement the steps of the bag-flipping operation identification method described in any of the preceding claims.

[0015] In the technical solution of this invention, firstly, a set of video frames labeled with bags and a set of video frames labeled with human figures are acquired. A pre-trained bag detection model is then used to detect bags in the set of video frames labeled with bags, resulting in a bag detection result set. Next, a pre-trained human pose detection model is used to detect human poses in the set of video frames labeled with human figures, resulting in a human detection result set. The bag detection result set and the human detection result set are then linked by timestamps to obtain a joint detection result set. The joint detection result set is then processed to generate operation motion trajectories, resulting in an interaction graph feature set. A pre-trained bag-flipping action recognition model is then used to recognize and filter bag-flipping actions in the interaction graph feature set, resulting in a target bag-flipping recognition result set. Finally, a pre-trained anomaly detection model is used to detect and judge bag-flipping anomalies in the target bag-flipping recognition result set, resulting in a bag-flipping anomaly detection result set. The aim is to improve the accuracy of bag-flipping recognition and reduce the false alarm rate through multi-modal feature fusion. Attached Figure Description

[0016] Figure 1 This is a first flowchart of the bag-turning operation identification method provided in an embodiment of the present invention; Figure 2 This is a second flowchart of the bag-flipping operation recognition method provided in an embodiment of the present invention; Figure 3 This is a third flowchart of the bag-flipping operation recognition method provided in the embodiments of the present invention; Figure 4 This is a fourth flowchart of the bag-flipping operation recognition method provided in the embodiments of the present invention; Figure 5 This is a fifth flowchart of the bag-flipping operation identification method provided in the embodiments of the present invention; Figure 6 This is a sixth flowchart of the bag-flipping operation identification method provided in the embodiments of the present invention; Figure 7 This is the seventh flowchart of the bag-turning operation identification method provided in the embodiments of the present invention; Figure 8 This is a schematic diagram of the structure of the bag-flipping operation recognition device provided in an embodiment of the present invention; Figure 9 This is a schematic diagram of the structure of the bag flipping operation identification device provided in an embodiment of the present invention. Detailed Implementation

[0017] This invention provides a method, apparatus, device, and storage medium for identifying bag-turning operations. In this invention, the terms "first," "second," "third," "fourth," etc. (if applicable) in the specification, claims, and accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms "comprising" or "having" and any variations thereof are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0018] For ease of understanding, the specific process of the embodiments of the present invention is described below. Please refer to [link / reference]. Figure 1 One embodiment of the bag-turning operation identification method of the present invention includes: 101. Obtain the video frame set with bag annotations and the video frame set with human body annotations; In this embodiment, the process begins by collecting videos of bag-turning operations in actual scenarios such as logistics sorting centers. Using a video frame extraction tool, the collected videos are analyzed frame by frame in chronological order to obtain a continuous and temporally complete sequence of video frames to be tested. Then, an annotation tool is used to annotate two types of targets: for bag targets, the bounding box position and bag type are labeled; for human targets, the coordinates of key pose points such as hands and key point confidence are labeled. The annotated bag and human annotated video frames are then organized and integrated according to the chronological order of the original videos, ultimately forming structured sets of annotated bag and human video frames. These two types of video frame sets, through annotation and temporal integration, provide high-quality data support for the subsequent inference operations of the bag detection model and the human pose detection model, ensuring that the models can accurately extract the core feature information of the bags and the human body. Simultaneously, the temporally coherent frame set structure ensures the accuracy of subsequent timestamp association processing, avoiding feature association errors caused by frame sequence disorder.

[0019] 102. Call the pre-trained bag detection model to perform bag detection on the set of video frames labeled with bag information, and obtain a set of bag detection results; In this embodiment, the bag detection model is built on the YOLOv10 architecture. The lightweight architecture and efficient feature extraction capabilities of the YOLOv10 model can ensure the real-time performance of the detection process and adapt to the application requirements of dynamic operation scenarios such as logistics sorting centers. The training dataset of the bag detection model comes from multi-time period and multi-condition bag operation videos collected in the actual operation scenarios of logistics sorting centers. After extracting video frames and labeling bag targets, a bag-specific training set is constructed. The samples cover bag targets of different sizes, different placement angles, and different operation backgrounds. At the same time, some bag samples under complex lighting and occlusion scenarios are added to ensure the scene adaptability and sample diversity of the dataset. The model has completed the initialization configuration of the backbone feature extraction network, neck feature fusion network, and head detection network in advance. During detection, the video frames labeled with bags are input into the model frame by frame in chronological order. The model, relying on its own architecture, completes the accurate identification of the bag target and the extraction of key information, and outputs the bag detection information corresponding to each frame. Then, combined with the timestamps of the original video frames, the detection results of all frames are systematically organized and integrated, and finally a structured bag detection result set is formed, which includes the bounding box coordinates of the bag in each frame, the category confidence, and the corresponding frame timestamp. The structured detection result set can provide standardized core data for the subsequent timestamp association processing of the bag detection result set and the human body detection result set.

[0020] 103. Call the pre-trained human pose detection model to perform human pose detection on the human body labeled video frame set to obtain a human body detection result set; In this embodiment, the human posture detection model is built on the HRNet architecture. The HRNet architecture has excellent multi-resolution feature adaptation capabilities, which can accurately capture detailed human posture features. It is especially suitable for the accurate positioning of key posture points such as hands in logistics sorting scenarios, ensuring the accuracy and stability of posture detection. The key posture points are limited to the hands and related action parts, specifically including the wrist joint, palm base, index fingertip, and middle fingertip. The elbow joint key point is collected simultaneously as an auxiliary tool. These points can accurately depict the interaction between the hand and the bag, such as grasping, flipping, and releasing, as well as the arm linkage posture. The training dataset of the human posture detection model comes from human bag-flipping operation videos collected in the actual operation scenario of the logistics sorting center. After frame extraction and key point annotation, a special sample set is formed. At the same time, a subset of hand postures from the human posture dataset is introduced for supplementary training. The samples are adapted and optimized for the characteristics of the sorting scenario. The samples cover bag-flipping postures of different operators, different operating angles, and different lighting conditions. The model has completed the initialization configuration of multi-resolution convolutional layers, feature fusion modules, and key point prediction layers in advance, and has reliable posture feature extraction and key point recognition capabilities. During the detection process, the set of annotated human body video frames is input into the model frame by frame in chronological order. Leveraging the core features of the HRNet architecture, the model extracts key features of human posture and locates key points, outputting a confidence heatmap of the key human posture points for each frame. Subsequently, combined with the timestamp information of the original video frames, the detection results of all frames are standardized, organized, and integrated, ultimately forming a structured human detection result set containing the confidence heatmap of the key human posture points for each frame and the corresponding frame timestamps. The confidence heatmap includes a set of human key point coordinates. This structured human detection result set provides standardized and high-precision basic data for subsequent timestamp association processing with the bag detection result set, ensuring temporal consistency and association accuracy between the two types of detection results, and providing reliable support for subsequent bag-opening operation motion trajectory generation and action recognition processes.

[0021] 104. Use a time-series matching algorithm to perform timestamp association processing on the bag detection result set and the human body detection result set to obtain a joint detection result set; In this embodiment, a joint detection result set is obtained by performing timestamp association processing on the bag detection result set and the human body detection result set. The core is to rely on a temporal matching algorithm, such as a hash matching algorithm, to use the timestamp of each video frame as a unified association benchmark to perform temporal alignment and matching integration of the two types of detection results. Specifically, based on the unique timestamp identifier generated when the original video frame is extracted, the bag detection result set and the human body detection result set are traversed separately. The bag detection information and human posture detection information corresponding to the same timestamp are matched frame by frame to ensure that the bag bounding box, category confidence, and other data in the same video frame form a one-to-one correspondence with the human body key point coordinates, confidence, and other data, eliminating the temporal misalignment problem between the two types of results. At the same time, abnormal detection data corresponding to invalid timestamps are removed, and the matched valid data are standardized and integrated to finally form a structured joint detection result set containing the timestamp of each frame, the corresponding bag detection parameters, and the human posture detection parameters. By accurately associating the two types of detection data with timestamps, the temporal consistency and information fusion of the two types of detection data are achieved. The joint detection results after temporal alignment can effectively avoid feature association errors caused by data misalignment, ensuring the accuracy of subsequent operation trajectory generation and action recognition. At the same time, the structured data format can improve the computational efficiency of subsequent processing steps, ensuring the temporal coherence and logical rigor of the entire bag-flipping operation recognition process.

[0022] 105. Perform motion trajectory generation processing on the joint detection result set to obtain an interaction graph feature set; In this embodiment, the joint detection result set is processed to generate an interaction graph feature set. The core is to extract interactive target features from the joint detection result set and achieve quantitative representation and sequence integration of the operation trajectory through topological graph construction. By constructing the topological graph, discrete hand and bag detection data are transformed into spatially related graph structure features, realizing a quantitative representation of the core interaction relationship between the hand and the bag during the bag-flipping operation. This effectively captures the fine spatial details and dynamic motion trajectory of the bag-flipping operation. Simultaneously, the structured graph feature format has good algorithm compatibility, significantly improving the efficiency and accuracy of subsequent model inference, effectively avoiding action recognition bias caused by the lack of correlation in discrete data, and adapting to the fine operation recognition needs in complex operational scenarios of logistics sorting centers.

[0023] 106. Call the pre-trained bag-flipping action recognition model to perform bag-flipping action recognition and filtering on the interaction graph feature set to obtain the target bag-flipping recognition result set; In this embodiment, a bag-flipping action recognition model is constructed based on the SlowFast-Plus dual-path network architecture. This model has been pre-trained and optimized, and its dual-path structure can be specifically adapted to the feature extraction requirements of bag-flipping operations. During processing, the interaction graph feature set is input into the model in a temporal sequence, and feature extraction and processing are carried out in parallel on both paths. The Slow path adopts a low frame rate sampling strategy, combined with the built-in spatial attention module, which can accurately focus on the core interaction areas of bag-flipping operations such as the bag opening and hands, and strengthen the capture and representation of features such as the spatial position relationship and morphological details of key parts. At the same time, the Fast path extracts the temporal change features of the operation simultaneously, and the two complement each other. The dual-path features are integrated through the built-in feature fusion mechanism of the model. Combined with the bag-flipping action feature patterns learned in pre-training, the fused features are used to predict the probability of bag-flipping actions and perform preliminary recognition, generating an initial recognition result containing the probability of bag-flipping actions in each frame, integrity score, and corresponding temporal information. Subsequently, the initial results are filtered through a model screening mechanism based on a preset probability threshold. Invalid bag-flipping action results with a probability lower than the threshold are removed, while valid bag-flipping operation records with high confidence are retained. After standardized integration, a structured target bag-flipping recognition result set is formed. The dual-path architecture takes into account both spatial details and temporal features, ensuring a comprehensive representation of bag-flipping actions and improving recognition accuracy.

[0024] 107. Call the pre-trained anomaly detection model to perform anomaly detection on the target bag-turning recognition result set, and obtain the bag-turning anomaly detection result set.

[0025] In this embodiment, a pre-trained anomaly detection model is invoked to perform anomaly detection on the target bag-turning identification result set to obtain an anomaly detection result set. The core of this approach is based on the compliant bag-turning operation feature benchmark learned during the model's pre-training phase, enabling quantitative judgment of the compliance of valid bag-turning operations and anomaly screening. The pre-trained anomaly detection model uses a compliance prototype vector as the judgment benchmark, possessing feature difference capture capabilities and generalization performance. It can identify minor violations such as incomplete bag turning or residual material not being emptied, effectively avoiding the subjectivity and high false negative rate of traditional manual supervision. The entire detection process is automated and batch-processed, significantly reducing the manpower management costs of logistics sorting centers and improving the overall efficiency of bag-turning operation compliance detection.

[0026] After anomaly detection is completed, the initially generated bag-turning anomaly detection result set needs to undergo further processing to enhance the interpretability and engineering adaptability of the results. Specifically, the spatial attention weight distribution of the bag-turning action recognition model is extracted, and the weight values ​​are converted into corresponding heatmap color gradients through a feature mapping algorithm. These gradients are then accurately superimposed onto the abnormal frame images in the bag-turning anomaly detection result set to highlight key interaction areas such as hands and bag openings corresponding to abnormal operations, providing an intuitive basis for manual review and problem tracing. For normal operation frames in the bag-turning anomaly detection result set, a spatial attention heatmap is also generated simultaneously to construct a visual comparison system between normal and abnormal operations, helping operators quickly distinguish operational differences and locate the root cause of deviations. In the later stages of processing, the structured review information in the bag-turning anomaly detection result set, including the judgment conclusions of each operation, deviation distance values, corresponding frame timestamps, and operation details, is linked and integrated with the file storage path of the visual heatmap. It is then strictly converted into standardized data messages in JSON format according to preset data specifications to eliminate data format barriers between different systems and ensure that the detection results can be smoothly transmitted, stored, and displayed on the front end across platforms.

[0027] Please see Figure 2 In this embodiment of the invention, obtaining the set of video frames labeled with bag annotations and the set of video frames labeled with human body annotations includes: 201. Obtain the video to be tested, and use a video frame extraction tool to extract the video frame by frame in chronological order to obtain multiple video frames to be tested; 202. Each of the video frames to be tested is annotated using an annotation tool to obtain multiple annotated video frames of bags and multiple annotated video frames of people; 203. Sort and integrate the labeled video frames of each of the bags according to the time sequence to obtain the labeled video frame set of the bags; 204. Sort and integrate each of the human body labeled video frames according to the time sequence to obtain the human body labeled video frame set.

[0028] In this embodiment, video frame extraction tools such as OpenCV and FFmpeg, which have efficient video parsing capabilities, can be used. Annotation tools such as LabelImg, LabelMe, VGG Image Annotator, and CVAT can be used. During the annotation process, the bounding box coordinates and category attributes of the bag target are marked, and the coordinate information of key pose points such as the hands of the human target are marked, ensuring that the annotated data accurately corresponds to the core detection target and meets the basic data requirements for subsequent model inference. Each bag-annotated video frame is sorted and integrated according to the time sequence of the video to be detected, resulting in a bag-annotated video frame set. Simultaneously, each human-annotated video frame is sorted and integrated according to the same time sequence, resulting in a human-annotated video frame set. The sorting and integration process strictly follows the temporal logic of the frames, eliminating subsequent process deviations caused by frame sequence disorder. Completing the sorting and integration of the two types of frame sets in chronological order ensures the temporal consistency between the bag-annotated video frame set and the human-annotated video frame set, establishing a precise temporal benchmark for the subsequent timestamp association processing of the two types of detection results, and effectively avoiding feature association errors and detection deviations caused by frame sequence disorder.

[0029] Please see Figure 3 In this embodiment of the invention, the bag detection model includes a backbone feature extraction module, a first feature fusion module, and a head detection module, which are connected sequentially. The step of calling the pre-trained bag detection model to perform bag detection on the bag-annotated video frame set, resulting in a bag detection result set, includes: 301. Based on the backbone feature extraction module, perform multi-scale image feature extraction on the set of labeled video frames of the bag to obtain a multi-scale bag image feature set; 302. Based on the first feature fusion module, perform feature fusion enhancement processing on the multi-scale bag image feature set to obtain a fused bag image feature set; 303. Based on the head detection module, predict the bounding box coordinates and category probability of the fused bag image feature set to obtain the bag detection result set.

[0030] In this embodiment, the bag detection model is built on the YOLOv10 architecture. The YOLOv10 architecture gives the model efficient feature extraction and inference capabilities. With anchor frame settings adapted to the size of the bag, it can accurately adapt to the shape characteristics of the bag in the logistics sorting scenario, improve the detection accuracy and speed of bag targets in complex backgrounds, and reduce missed detections, false detections, and bounding box positioning deviations. The bag detection model includes a backbone feature extraction module, a first feature fusion module, and a head detection module. The three are connected in sequence to form a complete inference link. The initial configuration of each module has been completed in the model building stage. At the same time, basic parameters such as input image resolution, anchor frame size adapted to common bag sizes, and confidence threshold are set. After training and optimization with training set data, it has reliable bag detection capabilities. When the pre-trained model is used to detect bag-annotated video frames, the backbone feature extraction module first performs multi-scale image feature extraction on the input bag-annotated video frames, relying on the feature extraction capabilities that have been initialized and enhanced through training. This captures bag image features at different levels, forming a multi-scale bag image feature set. Subsequently, the first feature fusion module performs fusion enhancement processing on the multi-scale feature set, eliminating hierarchical differences between features at different scales, enhancing feature representation capabilities, and outputting a fused bag image feature set. Finally, the head detection module uses the fused feature set as a basis for detection. By combining the preset anchor frame size and confidence threshold, the bounding box coordinate regression and category probability prediction of the bag are completed. Low-confidence detection results are filtered based on the confidence threshold, and the effective results are integrated frame by frame to obtain a bag detection result set containing the bounding box coordinates, category confidence, and time sequence information of each frame. The structured detection result set provides standardized data support for subsequent timestamp association processing with the human body detection result set, ensuring the accuracy of the time sequence alignment of the two types of results, laying a solid foundation for the entire bag-flipping operation recognition process, and indirectly improving the overall reliability of subsequent action recognition and anomaly detection.

[0031] Please see Figure 4 In this embodiment of the invention, the human pose detection model includes a multi-resolution convolution module, a second feature fusion module, and a keypoint prediction module, which are sequentially connected. The step of calling the pre-trained human pose detection model to perform human pose detection on the annotated video frame set to obtain a human detection result set includes: 401. Based on the multi-resolution convolution module, perform multi-scale human feature extraction on the human body labeled video frame set to obtain a multi-scale human body image feature set; 402. Perform multi-scale feature fusion processing on the multi-scale human image feature set based on the second feature fusion module to obtain a fused human image feature set; 403. Based on the key point prediction module, perform human key point confidence heatmap prediction on the fused human image feature set to obtain the human detection result set.

[0032] In this embodiment, the human pose detection model is built on the HRNet architecture. The multi-resolution feature processing capability of the HRNet architecture can accurately capture the subtle details of human pose, especially suitable for the positioning requirements of key operation points such as hands in logistics sorting scenarios. It improves the accuracy and stability of human pose detection under complex operation backgrounds, and reduces key point positioning deviations and missed detections. The human pose detection model includes a multi-resolution convolution module, a second feature fusion module, and a key point prediction module. The three are connected in sequence to form a complete inference link. The initial configuration of each module has been completed in the model building stage. At the same time, the model input resolution is set to be consistent with the resolution of the training set images, and reasonable core parameters such as the number of feature map channels and the size of the convolution kernel are configured. After optimization training on a human pose-specific training set, it has reliable human key point detection capabilities. When the pre-trained model is used to perform human pose detection on a set of annotated human video frames, the multi-resolution convolution module first performs multi-scale human feature extraction on the input set of annotated human video frames, generating human feature maps at different scales to form a multi-scale human image feature set, based on the feature extraction capabilities that have been initialized and enhanced through training. Subsequently, the second feature fusion module performs multi-scale feature fusion processing on the multi-scale human image feature set, strengthening the information complementarity between features of different resolutions, eliminating the feature representation shortcomings caused by scale differences, and outputting a fused human image feature set. Finally, the keypoint prediction module completes the prediction of human keypoint confidence heatmap based on the fused human image feature set, extracts the coordinates of effective human keypoints and corresponding confidence information through peak location of the heatmap, filters out invalid data with low confidence, and integrates the effective detection results of each frame in time sequence to obtain a human detection result set containing the coordinates, confidence, and time sequence information of human keypoints in each frame.

[0033] Please see Figure 5 In this embodiment of the invention, the step of performing motion trajectory generation processing on the joint detection result set to obtain an interaction graph feature set includes: 501. Based on the joint detection result set, obtain the set of bag bounding box coordinates and human key point coordinates corresponding to each video frame; 502. Determine the coordinate range of the bag opening area and the coordinate range of the bag bottom area corresponding to each video frame based on the coordinates of the bounding box of each bag; 503. Based on the coordinate set of human key points, the coordinate range of the bag opening region, and the coordinate range of the bag bottom region corresponding to each video frame, Euclidean distance calculation is performed to obtain the association results of multiple nodes; 504. An interaction graph feature corresponding to each video frame is constructed based on the association result of each node using a trajectory encoder; 505. Based on the time sequence, integrate multiple interaction graph features to obtain the interaction graph feature set.

[0034] In this embodiment, the coordinates of the bag bounding box and the coordinates of human key points corresponding to each video frame are obtained based on the joint detection result set. The focus is on extracting the hand key point coordinates from the human key point coordinate set. At the same time, based on the bag bounding box coordinates, the coordinate range of the bag opening area and the bag bottom area are clearly marked with the intersection of the diagonals of the bounding box as the dividing criterion. The extracted hand key points and bag key areas are respectively used as the core nodes of the interaction graph. Each node is assigned corresponding coordinate features and category features. Then, Euclidean distance calculation is performed on any two nodes, and a reasonable distance threshold is set. The setting of the distance threshold needs to be determined by a combination of factors such as the physical constraints of the actual operation scenario, statistical analysis of the labeled dataset, model verification optimization, and video acquisition equipment parameters. When the Euclidean distance between two nodes is less than the threshold, an edge connecting the two nodes is constructed and the reciprocal of the distance is used as the weight of the edge, thereby forming the node association result. Then, based on the association results of each node, the topology of the single-frame hand-bag interaction graph is constructed. A trajectory encoder is then used to enhance the temporal dimension of the features of the single-frame interaction graph. The trajectory encoder adopts a lightweight hierarchical structure design, which includes an input adaptation layer, a temporal feature extraction layer, a stacked fully connected mapping layer, and a feature concatenation output layer. The input adaptation layer is used to complete the format regularization and dimensional alignment of the hand key point coordinate sequence. The temporal feature extraction layer is used to quantize and calculate temporal features such as trajectory displacement and velocity change rate and complete the initial encoding. The stacked fully connected mapping layer is used to enhance the feature representation capability through two fully connected networks combined with nonlinear activation functions to achieve accurate mapping of temporal features to fixed-dimensional high-dimensional vectors. The feature concatenation output layer is used for the orderly integration and output of trajectory vectors of multiple hand key points. In this process, the trajectory encoder extracts the coordinate sequence of hand key points from single-frame hand-bag interaction graphs across multiple consecutive frames. It then transforms the coordinate sequence of the same hand key point in consecutive frames into a motion trajectory. By calculating the temporal features such as displacement and velocity change rate of the trajectory, these temporal features are mapped into fixed-dimensional vectors via a fully connected layer. The trajectory feature vectors of multiple hand key points are concatenated to form the operation motion trajectory feature. Subsequently, this temporal trajectory feature is integrated into the interaction graph feature of the corresponding single frame, forming a single-frame interaction graph feature that combines spatial topological association and temporal motion features. Finally, based on the temporal order of the original video, the enhanced interaction graph features corresponding to all video frames are systematically integrated to obtain an interaction graph feature set. This structured interaction graph feature set has good algorithm compatibility, smoothly adapting to the inference needs of various deep learning models and meeting the recognition needs of fine bag-turning operations in complex operational scenarios of logistics sorting centers.

[0035] Please see Figure 6 In this embodiment of the invention, the bag-flipping action recognition model includes a dual-path feature extraction module, a dual-path feature fusion module, a bag-flipping action recognition module, and a filtering output module. The dual-path feature extraction module, the dual-path feature fusion module, the bag-flipping action recognition module, and the filtering output module are connected sequentially. The dual-path feature extraction module includes a spatial feature extraction submodule and a temporal feature extraction submodule, which are connected to each other. The step of calling the pre-trained bag-flipping action recognition model to perform bag-flipping action recognition and filtering on the interaction graph feature set to obtain a target bag-flipping recognition result set includes: 601. Based on the spatial feature extraction submodule, perform spatial detail feature extraction processing on the interaction graph feature set to obtain a spatial detail feature set; 602. Based on the temporal feature extraction submodule, perform temporal change feature extraction processing on the interaction graph feature set to obtain a temporal change feature set; 603. Based on the dual-path feature fusion module, channel splicing and fusion processing is performed on the spatial detail feature set and the temporal change feature set to obtain a fused feature set; 604. Based on the bag-flipping action recognition module, perform bag-flipping probability prediction on the fused feature set to obtain an initial bag-flipping recognition result set, wherein the initial bag-flipping recognition result set includes a bag-flipping action probability dataset; 605. Based on the filtering output module, the initial bag-flipping recognition result set is subjected to high-probability result filtering processing according to the bag-flipping action probability dataset to obtain the target bag-flipping recognition result set.

[0036] In this embodiment, the bag-flipping action recognition model is based on a customized SlowFast-Plus architecture, which includes a dual-path feature extraction module, a dual-path feature fusion module, a bag-flipping action recognition module, and a filtering output module. These four modules are connected in sequence to form a complete inference link. The spatial feature extraction sub-module and the temporal feature extraction sub-module of the dual-path feature extraction module work in parallel and complement each other. The parameter configuration and optimization of each module have been completed in the model pre-training stage, which can adapt to the feature extraction and recognition requirements of bag-flipping operations. When using this pre-trained model to identify and filter bag-flipping actions using the interaction graph feature set, the interaction graph feature set output by the trajectory encoder is first synchronously input into the dual-path feature extraction module. The spatial feature extraction submodule corresponds to the Slow path, configured with a low frame rate sampling strategy and an integrated spatial attention module. It focuses on core interaction areas such as the bag opening and hands, accurately extracting spatial detail features from the interaction graph feature set to form a spatial detail feature set. The temporal feature extraction submodule corresponds to the Fast path, configured with a high frame rate sampling strategy and an integrated temporal convolution module. This enhances the modeling of temporal changes such as bag-flipping direction and action rhythm, while also integrating temporal information such as displacement and velocity change rate from the motion trajectory features to extract a temporal change feature set. This dual-path architecture and targeted module design achieve accurate capture of spatial details and temporal dynamic features. The spatial attention module of the Slow path effectively avoids interference from complex background environments, focusing on core interaction areas to improve feature recognition. The temporal convolution module of the Fast path, combined with motion trajectory features, accurately depicts the dynamic evolution of the bag-flipping action. The fusion of these two modules significantly improves the comprehensiveness of feature representation.

[0037] In this embodiment, the spatial detail feature set and the temporal change feature set are processed by channel splicing and weight allocation based on the dual-path feature fusion module. This balances the representation weights of spatial and temporal features, eliminates differences in feature dimensions, enhances feature complementarity, and generates a fused feature set that combines spatial details and temporal dynamics. Subsequently, the bag-flipping action recognition module receives the fused feature set, completes feature mapping and inference through the built-in fully connected layer, and simultaneously outputs two core results: one is the probability value of the bag-flipping action, used to determine whether it is a valid bag-flipping operation, i.e., the bag-flipping action probability dataset; the other is the integrity score of the bag-flipping action, used to quantify the standardization of the bag-flipping operation. Together, these two constitute the initial bag-flipping recognition result set. Finally, the output filtering module uses a preset action probability threshold as the judgment standard. The preset action probability threshold needs to be scientifically set by comprehensively considering multiple factors such as the probability distribution characteristics of the training dataset, model verification evaluation indicators, actual operation scenario requirements, and cross-validation optimization results. Based on the action probability threshold, the initial bag-flipping recognition result set is processed to filter high-probability results, eliminating invalid bag-flipping action records with a bag-flipping action probability data below the threshold, and retaining high-confidence valid bag-flipping operation data and corresponding integrity scores. After time-series processing and structured integration, the target bag-flipping recognition result set is obtained, effectively eliminating invalid interference data, ensuring the purity and reliability of the target bag-flipping recognition result set, and providing high-quality input for the subsequent anomaly detection process.

[0038] Please see Figure 7 In this embodiment of the invention, the anomaly detection model includes a feature encoding module, an Euclidean distance calculation module, and an anomaly detection module, which are sequentially connected. The step of calling the pre-trained anomaly detection model to perform anomaly detection judgment on the target bag-turning recognition result set to obtain a bag-turning anomaly detection result set includes: 701. Obtain the interaction graph feature set corresponding to the target bag-flipping recognition result set; 702. Input the interaction graph feature set into the feature encoding module for deep encoding processing to obtain a high-dimensional feature vector set; 703. Through the Euclidean distance calculation module, Euclidean distance calculation is performed on the high-dimensional feature vector set according to the preset compliant prototype vector to obtain multiple feature distance values; 704. The anomaly detection module compares each feature distance value with a preset anomaly threshold to obtain the bag-turning anomaly detection result set.

[0039] In this embodiment, the anomaly detection model is a prototype-based network architecture, comprising a feature encoding module, an Euclidean distance calculation module, and an anomaly detection module. These three modules are sequentially connected to form a complete inference chain. The model pre-training stage has completed the construction and parameter optimization of compliance prototype vectors, providing a core reference for anomaly judgment. When calling this pre-trained model to perform anomaly detection on the target bag-turning recognition result set, the interaction graph feature set corresponding to the target bag-turning recognition result set is first obtained. This interaction graph feature set is then input into the feature encoding module and processed using the same deep encoding method as in the model training stage to obtain a high-dimensional feature vector set. Specifically, during the model training stage, videos of compliant bag-turning operations in the workshop are collected. Frames are extracted and labeled to form a pure compliance sample set. After preprocessing, hand-bag interaction graph features and motion trajectory features are extracted to construct a compliance sample feature set. The compliance sample feature set is then deep encoded by the feature encoding module to obtain a high-dimensional compliance feature vector. Finally, the average of all high-dimensional compliance feature vectors is taken to generate a compliance prototype vector representing the feature center of all compliant operations, which is then pre-set in the model. Based on the Euclidean distance calculation module, using the preset compliance prototype vector as a benchmark, the Euclidean distance between each vector in the high-dimensional feature vector set and the compliance prototype vector is calculated one by one. This yields the feature distance value that characterizes the degree of deviation between each operation to be detected and the compliance benchmark. Using the prototype vector constructed from pure compliance samples as the judgment benchmark, the core features of compliant bag-turning operations can be accurately anchored, avoiding interference from mixed samples on the benchmark, improving the accuracy of anomaly detection, and effectively identifying minor violations such as bags not being completely turned over or residual materials not being emptied. Furthermore, the quantitative calculation of Euclidean distance makes the degree of operational deviation traceable, which is more objective and scientific than traditional subjective judgment. Finally, the anomaly detection module compares each feature distance value with a preset anomaly threshold. The preset anomaly threshold needs to be set based on the statistical distribution of compliant sample distances, model verification evaluation indicators, and actual operational scenario control requirements. If the distance value is less than the anomaly threshold, the corresponding bag-turning operation is judged to be normal; if the distance value is greater than or equal to the anomaly threshold, the operation is judged to have deviated. The judgment results, feature distance values, corresponding frame timestamps, and associated feature information of each operation are recorded simultaneously. After structured integration, a bag-turning anomaly detection result set is obtained. The automated detection process significantly reduces the cost of manual management in the workshop and improves the detection efficiency of batch operations.

[0040] The above describes the method for recognizing the bag-turning operation in an embodiment of the present invention. The following describes the device for recognizing the bag-turning operation in an embodiment of the present invention. Please refer to [link / reference]. Figure 8 One embodiment of the bag-flipping operation recognition device of the present invention includes: Data acquisition module 801: used to acquire the video frame set of bag annotation and human body annotation; Bag detection module 802: used to call a pre-trained bag detection model to detect bags in the bag-annotated video frame set and obtain a bag detection result set; Human detection module 803: used to call a pre-trained human pose detection model to perform human pose detection on the human labeled video frame set, and obtain a human detection result set; Result association module 804: used to perform timestamp association processing on the bag detection result set and the human body detection result set using a time-series matching algorithm to obtain a joint detection result set; Trajectory generation module 805: used to perform motion trajectory generation processing on the joint detection result set to obtain an interaction graph feature set; Bag-flipping action recognition module 806: used to call a pre-trained bag-flipping action recognition model to perform bag-flipping action recognition and filtering on the interaction graph feature set, and obtain a target bag-flipping recognition result set; The bag-turning anomaly detection module 807 is used to call a pre-trained anomaly detection model to perform bag-turning anomaly detection judgment on the target bag-turning recognition result set, and obtain the bag-turning anomaly detection result set.

[0041] Based on the same ideas as the methods in the above embodiments, the apparatus provided in this application can implement the methods in the above embodiments.

[0042] above Figure 8 The bag-flipping operation recognition device in this embodiment of the invention is described in detail from the perspective of modular functional entities. The bag-flipping operation recognition device in this embodiment of the invention is described in detail from the perspective of hardware processing.

[0043] Figure 9 This is a schematic diagram of the structure of a bag-flipping operation recognition device 900 provided in an embodiment of the present invention. The bag-flipping operation recognition device 900 can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 910 (e.g., one or more processors) and a memory 920, and one or more storage media 930 (e.g., one or more mass storage devices) storing application programs 933 or data 932. The memory 920 and storage media 930 can be temporary or persistent storage. The program stored in the storage media 930 may include one or more modules (not shown in the diagram), each module may include a series of instruction operations on the bag-flipping operation recognition device 900. Furthermore, the processor 910 may be configured to communicate with the storage media 930 and execute the series of instruction operations in the storage media 930 on the bag-flipping operation recognition device 900 to implement the steps of the bag-flipping operation recognition method provided in the above-described method embodiments.

[0044] The bag-flipping and identification device 900 may also include one or more power supplies 940, one or more wired or wireless network interfaces 950, one or more input / output interfaces 960, and / or one or more operating systems 931, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art will understand that... Figure 9 The illustrated structure of the bag flipping operation identification device does not constitute a limitation on the bag flipping operation identification device. It may include more or fewer components than illustrated, or combine certain components, or have different component arrangements.

[0045] The present invention also provides a computer-readable storage medium, which can be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium, wherein the computer-readable storage medium stores instructions that, when executed on a computer, cause the computer to perform the steps of the bag-flipping operation identification method.

[0046] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the system, device, or unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0047] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0048] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for recognizing the bag-turning operation of a collection bag, characterized in that, include: Obtain the video frame set with bag annotations and the video frame set with human body annotations; The pre-trained bag detection model is invoked to detect bags in the set of video frames labeled with bags, and a set of bag detection results is obtained. The pre-trained human pose detection model is invoked to perform human pose detection on the set of human-annotated video frames, and a human detection result set is obtained. A time-series matching algorithm is used to perform timestamp association processing on the bag detection result set and the human body detection result set to obtain a joint detection result set. The joint detection result set is processed to generate an operation motion trajectory, resulting in an interaction graph feature set; The pre-trained bag-flipping action recognition model is invoked to perform bag-flipping action recognition and filtering on the interaction graph feature set to obtain the target bag-flipping recognition result set; The pre-trained anomaly detection model is invoked to perform anomaly detection on the target bag-turning identification result set, thereby obtaining the bag-turning anomaly detection result set.

2. The method for identifying bag-turning operations according to claim 1, characterized in that, The acquisition of the bag-annotated video frame set and the human body-annotated video frame set includes: The video to be tested is acquired, and a video frame extraction tool is used to extract the video frame by frame in chronological order to obtain multiple video frames to be tested. Each of the video frames to be tested was annotated using an annotation tool to obtain multiple annotated video frames of bags and multiple annotated video frames of people. Based on the time sequence, each of the labeled video frames for the collection bag is sorted and integrated to obtain the set of labeled video frames for the collection bag; Based on the time sequence, each of the human body labeled video frames is sorted and integrated to obtain the human body labeled video frame set.

3. The method for identifying bag-turning operations according to claim 1, characterized in that, The bag detection model includes a backbone feature extraction module, a first feature fusion module, and a head detection module, which are connected in sequence. The pre-trained bag detection model is invoked to detect bags in the set of video frames labeled with bags, resulting in a set of bag detection results, including: Based on the backbone feature extraction module, multi-scale image feature extraction is performed on the set of labeled video frames of the bag to obtain a multi-scale bag image feature set. Based on the first feature fusion module, feature fusion enhancement processing is performed on the multi-scale bag image feature set to obtain a fused bag image feature set. Based on the head detection module, the bounding box coordinates and category probabilities of the fused bag image feature set are predicted to obtain the bag detection result set.

4. The method for identifying the bag-turning operation of a collection bag according to claim 1, characterized in that, The human pose detection model includes a multi-resolution convolution module, a second feature fusion module, and a keypoint prediction module, which are sequentially connected. The pre-trained human pose detection model is used to perform human pose detection on the annotated video frame set to obtain a human detection result set, including: Based on the multi-resolution convolution module, multi-scale human features are extracted from the human body labeled video frame set to obtain a multi-scale human image feature set. Based on the second feature fusion module, the multi-scale human image feature set is subjected to multi-scale feature fusion processing to obtain a fused human image feature set. Based on the key point prediction module, the confidence heatmap prediction of human key points is performed on the fused human image feature set to obtain the human detection result set.

5. The method for identifying the bag-turning operation of a collection bag according to claim 2, characterized in that, The step of performing motion trajectory generation processing on the joint detection result set to obtain an interaction graph feature set includes: Based on the joint detection result set, obtain the set of bag bounding box coordinates and human key point coordinates corresponding to each video frame; Based on the coordinates of the bounding box of each bag, determine the coordinate range of the bag opening area and the coordinate range of the bag bottom area corresponding to each video frame; Euclidean distance calculation is performed based on the human key point coordinate set, bag opening area coordinate range and bag bottom area coordinate range corresponding to each video frame to obtain multiple node association results; An interaction graph feature corresponding to each video frame is constructed using a trajectory encoder based on the association results of each node. The interaction graph feature set is obtained by integrating multiple interaction graph features based on the time sequence.

6. The method for identifying bag-turning operations according to claim 1, characterized in that, The bag-flipping action recognition model includes a dual-path feature extraction module, a dual-path feature fusion module, a bag-flipping action recognition module, and a filtering output module. The dual-path feature extraction module, the dual-path feature fusion module, the bag-flipping action recognition module, and the filtering output module are connected in sequence. The dual-path feature extraction module includes a spatial feature extraction submodule and a temporal feature extraction submodule, which are connected to each other. The pre-trained bag-flipping action recognition model is invoked to perform bag-flipping action recognition and filtering on the interaction graph feature set, resulting in a target bag-flipping recognition result set, including: Based on the spatial feature extraction submodule, spatial detail feature extraction processing is performed on the interaction graph feature set to obtain a spatial detail feature set; Based on the temporal feature extraction submodule, the interaction graph feature set is processed to extract temporal change features to obtain a temporal change feature set. Based on the dual-path feature fusion module, the spatial detail feature set and the temporal change feature set are subjected to channel splicing and fusion processing to obtain a fused feature set; Based on the bag-flipping action recognition module, the bag-flipping probability is predicted on the fused feature set to obtain an initial bag-flipping recognition result set, which includes a bag-flipping action probability dataset. Based on the filtering output module, the initial bag-flipping recognition result set is filtered for high probability results according to the bag-flipping action probability dataset to obtain the target bag-flipping recognition result set.

7. The method for identifying the bag-turning operation of a collection bag according to claim 6, characterized in that, The anomaly detection model includes a feature encoding module, an Euclidean distance calculation module, and an anomaly detection module, which are connected in sequence. The pre-trained anomaly detection model is invoked to perform anomaly detection on the target bag-crossing recognition result set, resulting in an anomaly detection result set, including: Obtain the interaction graph feature set corresponding to the target bag-flipping recognition result set; The interaction graph feature set is input into the feature encoding module for deep encoding processing to obtain a high-dimensional feature vector set. The Euclidean distance calculation module performs Euclidean distance calculation on the high-dimensional feature vector set based on the preset compliant prototype vector to obtain multiple feature distance values. The anomaly detection module compares each feature distance value with a preset anomaly threshold to obtain the bag-turning anomaly detection result set.

8. A bag-flipping operation identification device, characterized in that, include: Data acquisition module: used to acquire video frame sets of bag annotations and human body annotations; Bag detection module: used to call a pre-trained bag detection model to detect bags in the set of video frames labeled with bags, and obtain a set of bag detection results; Human detection module: used to call a pre-trained human pose detection model to perform human pose detection on the set of human-annotated video frames, and obtain a human detection result set; Result association module: used to perform timestamp association processing on the bag detection result set and the human body detection result set using a time-series matching algorithm to obtain a joint detection result set; Trajectory generation module: used to process the joint detection result set to generate motion trajectories and obtain an interaction graph feature set; Bag-flipping action recognition module: used to call a pre-trained bag-flipping action recognition model to perform bag-flipping action recognition and filtering on the interaction graph feature set, and obtain the target bag-flipping recognition result set; Bag-turning anomaly detection module: used to call a pre-trained anomaly detection model to perform bag-turning anomaly detection judgment on the target bag-turning recognition result set, and obtain bag-turning anomaly detection result set.

9. A bag-flipping operation identification device, characterized in that, The bag-flipping operation identification device includes: a memory and at least one processor, wherein the memory stores instructions; At least one of the processors invokes the instructions in the memory to cause the bag-turning operation identification device to perform the various steps of the bag-turning operation identification method as described in any one of claims 1-7.

10. A computer-readable storage medium storing instructions thereon, characterized in that, When the instructions are executed by the processor, they implement each step of the bag-flipping operation recognition method as described in any one of claims 1-7.

Images