A method, device, equipment, and medium for detecting theft based on an AI visual model.

Abstract

This disclosure relates to a method, apparatus, device, and medium for detecting theft behavior based on an AI visual model. The method includes: acquiring images of a monitored area using an image acquisition device to generate video stream data; generating a behavior detection box for a person based on the position of the target in the corresponding image frame; determining the number of behavior detection frames based on initialized model parameters; obtaining multi-frame behavior detection maps corresponding to the target in the corresponding number of frames from the video stream data; cropping the target's region in the multi-frame behavior detection maps based on the behavior detection boxes to generate multi-frame target region maps; generating multiple theft behavior confidence scores corresponding to the multi-frame target region maps using a theft behavior detection model; and determining the behavior judgment result corresponding to the target based on the multiple theft behavior confidence scores. This improves the intelligence level of theft monitoring system by introducing a dual verification mechanism for judging human posture and classifying actions in video surveillance.

Description

Technical Field

[0001] This disclosure relates to the field of image recognition technology, and more specifically, to a method, apparatus, device, and medium for detecting theft based on an AI visual model. Background Technology

[0002] Theft is a frequent occurrence in the retail industry, causing significant profit losses for businesses. Although supermarkets have installed surveillance cameras, detecting theft relies on employees spending extended periods of time watching the monitors. This not only increases the workload for employees but also increases the likelihood of missing opportunities to detect theft. Therefore, there is an urgent need to develop an intelligent theft detection system.

[0003] Currently, most existing video surveillance systems use methods based on video classification or single-frame image classification to identify theft. Video-based classification methods mainly extract a video segment, consider the temporal information between frames, and infer theft behavior by analyzing changes in movement. However, these methods require significant computational resources. Furthermore, when there are many human targets in the monitored image, the accuracy of theft detection drops considerably. Image-based theft detection methods process each frame, detecting and further analyzing pedestrians in each frame. This method, using target detection and tracking technology, can effectively locate people in the field of view, but it does not fully utilize the temporal information between frames. Moreover, theft detection methods that only use target detection technology lack information about the interaction between people and shelves, leading to a higher false recognition rate. Summary of the Invention

[0004] The purpose of this disclosure is to provide a method, apparatus, device, and medium for detecting theft based on an AI visual model.

[0005] To achieve the above objectives, the first aspect of this disclosure provides a theft behavior detection method based on an AI visual model, the method comprising:

[0006] In response to receiving user input of initialization model parameters, the theft behavior detection model parameter configuration is initialized according to the initialization model parameters, and when it is determined that the theft behavior detection model initialization is complete, images in the monitored area are acquired based on the image acquisition device to generate video stream data;

[0007] When the video stream data is multi-target detected by the independent tracking model in the theft behavior detection model, and it is determined that there is a human target in at least one image frame corresponding to the video stream data, a behavior detection box of the human target is generated according to the position of the human target in the corresponding image frame.

[0008] The number of frames for detecting human behavior is determined based on the initialization model parameters, and based on the number of frames for detecting human behavior, a multi-frame behavior detection map corresponding to the human target is obtained from the video stream data for the corresponding number of frames.

[0009] The character regions of the target characters in the multi-frame behavior detection maps are cropped based on the behavior detection bounding boxes to generate multi-frame character region maps. The behavior classification model in the theft behavior detection model is then used to detect theft behavior of the target characters in the multi-frame character region maps, generating the multi-frame character region maps. Figure 1 Confidence level of one-to-one correspondence between multiple theft behaviors;

[0010] Based on the confidence levels of the multiple theft behaviors, a behavior determination result corresponding to the target person is determined. The behavior determination result is used to indicate whether the target person has engaged in theft behavior within the time domain corresponding to the video stream data.

[0011] Optionally, in some embodiments, determining the behavior judgment result corresponding to the target person based on the confidence levels of the multiple theft behaviors includes:

[0012] Based on the initialization model parameters, determine the user-defined single-frame confidence threshold and time series proportion threshold;

[0013] Based on the confidence levels of the multiple theft behaviors, a first number of behavior detection maps with a confidence level greater than the single-frame confidence threshold are determined from the multi-frame behavior detection maps;

[0014] If, based on the second quantity and the first quantity of the multi-frame behavior detection map, it is determined that the proportion of the first quantity to the second quantity is greater than the time series proportion threshold, the behavior determination result is determined to be theft behavior determination result;

[0015] If the proportion of the first quantity to the second quantity is less than or equal to the time series proportion threshold, the behavior determination result is determined to be a non-theft behavior determination result.

[0016] Optionally, in some embodiments, the method further includes:

[0017] If the behavior determination result is determined to be the theft behavior determination result, the behavior determination result is sent to the user. In response to the behavior determination confirmation feedback from the user, the multi-frame human region map is used as a positive sample image.

[0018] If the behavior determination result is determined to be the non-theft behavior determination result, the multi-frame character area map is used as a negative sample image;

[0019] The behavior classification model is retrained based on the positive and negative sample images to update the model parameters.

[0020] Optionally, in some embodiments, the theft behavior detection is performed on the behavior of the target person in the multi-frame person region map using the behavior classification model in the theft behavior detection model, and the multi-frame person region map is generated. Figure 1 The confidence levels of multiple theft behaviors corresponding to one include:

[0021] The human body frame and hand key points of the target person in the first person region map are identified, and a human behavior recognition box including the hand key points is generated. The first person region map is any frame of the multi-frame person region map.

[0022] The behavior classification model is used to identify the behavior category of the human target in the human behavior recognition box, and a first behavior feature corresponding to the first human region map is generated.

[0023] The first behavioral feature is compared with the set theft behavior features to determine the confidence level of the first theft behavior corresponding to the first person area map.

[0024] Optionally, in some embodiments, determining the behavior judgment result corresponding to the target person based on the confidence levels of the multiple theft behaviors includes:

[0025] If the behavior determination result is determined to be theft behavior determination result based on the first theft behavior confidence, the hand behavior in the first person region map is analyzed according to the independent tracking model to generate the hand posture features of the person target in the first person region map.

[0026] If, based on the hand gesture features, the hand gesture of the target person in the first person area map is determined to be a theft gesture, then the behavior determination result is determined to be the theft behavior determination result.

[0027] If, based on the hand posture features, the hand posture of the target person in the first person area map is determined to be a non-stealing posture, then the behavior determination result is determined to be a non-stealing behavior determination result.

[0028] Optionally, in some embodiments, the method further includes:

[0029] Based on the hand posture features, determine the first distance between the palms of the human target in the human behavior recognition box;

[0030] If the first distance is greater than a set distance threshold, the behavior determination result is determined to be the theft behavior determination result;

[0031] If the first distance is less than or equal to the set distance threshold, the behavior determination result is determined to be the non-theft behavior determination result.

[0032] Optionally, in some embodiments, the first frame behavior detection map includes at least one of the following: bounding box coordinates, category label, and tracking ID, and the first frame behavior detection map is any behavior detection map among the multi-frame behavior detection maps.

[0033] According to a second aspect of this disclosure, a theft detection device based on an AI visual model is provided, the device comprising:

[0034] The first generation module is used to respond to the initialization model parameters input by the user, initialize the configuration of the theft behavior detection model parameters according to the initialization model parameters, and, when it is determined that the initialization of the theft behavior detection model is completed, acquire images in the monitored area based on the image acquisition device to generate video stream data.

[0035] The second generation module is used to generate a behavior detection box for the person target based on the position of the person target in the corresponding image frame when the video stream data is determined to contain a person target by performing multi-target detection on the video stream data through the independent tracking model in the theft behavior detection model.

[0036] The acquisition module is used to determine the number of human behavior detection frames according to the initialization model parameters, and based on the number of human behavior detection frames, to acquire the multi-frame behavior detection map corresponding to the human target in the corresponding number of frames from the video stream data;

[0037] The third generation module is used to crop the person regions of the person target pairs in the multi-frame behavior detection map according to the behavior detection box, generate a multi-frame person region map, and perform theft behavior detection on the behavior of the person target in the multi-frame person region map using the behavior classification model in the theft behavior detection model to generate the multi-frame person region map. Figure 1 Confidence level of one-to-one correspondence between multiple theft behaviors;

[0038] The execution module is used to determine the behavior judgment result corresponding to the target person based on the confidence levels of the multiple theft behaviors. The behavior judgment result is used to indicate whether the target person has engaged in theft behavior within the time domain corresponding to the video stream data.

[0039] According to a third aspect of this disclosure, an electronic device is provided, comprising:

[0040] A memory on which computer programs are stored;

[0041] A processor for executing the computer program in the memory to implement the steps of the method according to any one of the first aspects of this disclosure.

[0042] According to a fourth aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps of the method described in any of the first aspects of this disclosure.

[0043] The above technical solution, in response to receiving user-inputted initialization model parameters, initializes the theft behavior detection model parameter configuration according to the initialization model parameters. Once the theft behavior detection model initialization is complete, it acquires images within the monitored area using an image acquisition device, generating video stream data. Then, using the independent tracking model in the theft behavior detection model to perform multi-target detection on the video stream data, if it determines that a human target exists in at least one image frame corresponding to the video stream data, it generates a human target behavior detection box based on the human target's position in the corresponding image frame. It determines the number of human behavior detection frames based on the initialization model parameters, and based on the number of human behavior detection frames, it obtains multi-frame behavior detection maps corresponding to the human target in the corresponding number of frames from the video stream data. It then crops the human region of the human target pair in the multi-frame behavior detection map based on the behavior detection boxes, generating a multi-frame human region map. Finally, it uses the behavior classification model in the theft behavior detection model to detect theft behavior of the human target in the multi-frame human region map, generating multi-frame human region maps. Figure 1 The system assigns confidence scores to multiple theft behaviors. Based on these scores, it determines the behavioral assessment result for the target person. This result indicates whether the target person engaged in theft within the corresponding time domain of the video stream data. By introducing a dual verification mechanism, the system tracks and locates each pedestrian in the video surveillance, and accurately determines the occurrence of theft through human posture judgment and action classification, thereby improving the intelligence level of the theft monitoring system.

[0044] Other features and advantages of this disclosure will be described in detail in the following detailed description section. Attached Figure Description

[0045] The accompanying drawings are provided to further illustrate the present disclosure and form part of the specification. They are used together with the following detailed description to explain the present disclosure, but do not constitute a limitation thereof. In the drawings:

[0046] Figure 1 This is a flowchart illustrating a theft detection method based on an AI visual model, according to an exemplary embodiment.

[0047] Figure 2 This is a block diagram illustrating a theft detection device based on an AI visual model, according to an exemplary embodiment.

[0048] Figure 3 This is a block diagram illustrating an electronic device according to an exemplary embodiment. Detailed Implementation

[0049] The specific embodiments of this disclosure will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit this disclosure.

[0050] In related technologies, the identification and detection of theft suffers from problems such as high false alarm rates, inability to continuously track theft, and lack of historical behavior analysis. Specifically, these problems can be attributed to the following reasons:

[0051] The lack of a dual verification mechanism is a significant issue. Current technology analyzes the entire video frame; however, surveillance footage often contains numerous human targets with complex behaviors. The lack of human detection, tracking, and further granular classification of human behavior leads to a high false alarm rate and makes it impossible to identify specific acts of theft.

[0052] Classification models are prone to overfitting. Although theft detection is a binary classification task, the behavioral variations between frames are minimal. The imbalanced distribution of positive and negative samples in the collected dataset leads to model overfitting, making it unable to detect absurd theft in real-world scenarios.

[0053] The model lacks adaptive learning capabilities. Current classification models based on convolutional neural networks, such as YOLO, are often trained on offline datasets. However, when these pre-trained models are deployed to real-world scenarios, new samples are not used to expand the database for iterative model updates, resulting in poor generalization to new samples.

[0054] In view of this, the purpose of this application is to provide a theft detection method based on an AI visual model, overcoming the shortcomings of existing video classification and action classification technologies, and enabling real-time and efficient detection of theft in video streams. By introducing a dual verification mechanism, each pedestrian in the video surveillance is tracked and located, and the occurrence of theft is accurately determined through human posture judgment and action classification, thereby improving the intelligence level of the monitoring system. Specifically, the objectives of this application include: intelligent monitoring: combining target tracking and classification algorithms to analyze the behavior of each human body in video surveillance; accurate identification: analyzing the interaction between people and shelves through human posture algorithms to improve the accuracy of theft behavior identification; continuous learning: adopting a data feedback mechanism so that after actual deployment, the algorithm can effectively utilize samples collected from real-world scenarios to iteratively update the classification model and improve model performance.

[0055] Figure 1 This is a flowchart illustrating a theft detection method based on an AI visual model, according to an exemplary embodiment. Figure 1 As shown, the method includes the following steps:

[0056] Step S101: In response to receiving the user-inputted initialization model parameters, the theft detection model parameter configuration is initialized according to the initialization model parameters. After determining that the theft detection model initialization is complete, images in the monitored area are acquired based on the image acquisition device to generate video stream data.

[0057] For example, this application raises the following key points:

[0058] (1) This application innovatively proposes a detection-classification dual-model cascade architecture, in which the classification model uses EfficientNet-B3 as the feature extraction backbone network, combined with depthwise separable convolution and compound scaling strategies, which significantly reduces computational complexity while ensuring recognition accuracy. By optimizing the model structure, reducing the number of parameters, and improving the inference process, the system can achieve real-time analysis of multiple video streams with limited computing resources. The loss function adopts Focal loss, and the weights are set according to the ratio of positive and negative samples in the dataset to deal with the class imbalance problem. A label smoothing strategy is used for model training. In addition, in order to alleviate the model overfitting problem, an R-Drop consistency regularization training strategy is introduced. By performing two independent forward propagations on the same input sample and using bidirectional KL divergence loss to force the consistency of the output distribution, the generalization ability and robustness of the model in complex monitoring scenarios are significantly improved. This framework effectively solves the problem of overfitting and insufficient generalization of traditional single models in behavior recognition tasks, and provides a more reliable inference basis for real-time behavior analysis.

[0059] (2) This application designs an intelligent behavior determination algorithm based on continuous multi-frame historical data. By maintaining a circular buffer for each tracked target, the frequency of high-confidence behaviors within a specific time window is statistically analyzed. A dual judgment standard of confidence threshold and time proportion threshold is adopted. Only when the confidence of a single frame exceeds the preset threshold and the proportion of high-confidence behaviors in the continuous frame sequence reaches a specified proportion is the behavior determined to be abnormal. This time-series analysis method effectively avoids instantaneous misjudgment and greatly improves the accuracy and reliability of the system in real-world scenarios.

[0060] (3) This application achieves deep integration of target tracking and behavior recognition. By assigning a unique identifier to each detected target and maintaining individualized historical records, the system can perform coherent analysis of the behavior of the same target at different time points. To further reduce false alarms, the YOLO (You Only Look Once) posture determination algorithm is used to further analyze the extended human body region based on the classification model. If the posture analysis includes "taking goods from the shelf", "standing", "moving", and the distance between the two palms. If there are "taking goods from the shelf" and "standing" actions, the distance between the palms is determined. If the distance is less than the set threshold, the posture algorithm determines it as theft. If the posture determination result and the classification model determination result are consistent and both indicate theft, the final result is determined as theft. This fusion processing not only improves the accuracy of behavior determination, but also provides rich temporal and posture data support for subsequent behavior pattern analysis and early warning, and builds a more intelligent and comprehensive monitoring and analysis system.

[0061] (4) Adaptive data feedback and continuous model optimization closed-loop learning system: This application constructs a complete automatic data collection and model self-optimization mechanism. The system can automatically classify and store positive and negative samples according to real-time analysis results, forming a virtuous cycle of "data collection-model training-performance improvement". Due to the lack of positive samples, the Gemini flash visual language model is used, and the prompt "Change the behavior of the person in the image toshop-lifting" is used to edit the negative samples. The edited image is then used as a positive sample containing theft behavior. Through bounded cropping and continuous numbering management, the system continuously accumulates real-world scene data while providing high-quality training samples for model iterative optimization, enabling the system to have continuous learning and environmental adaptability.

[0062] For example, in this embodiment, system initialization and model loading are required before detecting theft. The dual-model loading module loads a target detection model specifically for real-time detection and tracking of people in videos; and a behavior classification model specifically for analyzing behavioral features of cropped person regions. The two models use different network structures and training data, forming a cascaded recognition architecture. The parameter configuration module sets the historical buffer length EVENT_N=7 frames to store continuous time-series behavior data; configures the behavior judgment threshold EVENT_RATIO=0.5 as a statistical condition for triggering alarms; and sets the single-frame confidence threshold CONF_THO=0.85 to filter low-confidence classification results. Data backflow initialization creates positive and negative sample storage directory structures: cropped_persons / yes and cropped_persons / no; initializes the sample counter, numbering it continuously based on the existing sample quantity; and sets the backflow switch BackFlow=True to control whether the automatic sample collection function is enabled.

[0063] Optionally, in some embodiments, the first frame behavior detection map includes at least one of the following: bounding box coordinates, category label, and tracking ID, and the first frame behavior detection map is any behavior detection map among multiple frame behavior detection maps.

[0064] Step S102: After performing multi-target detection on the video stream data through the independent tracking model in the theft behavior detection model and determining that there is a human target in at least one image frame corresponding to the video stream data, a behavior detection box for the human target is generated based on the position of the human target in the corresponding image frame.

[0065] For example, this embodiment processes the video stream and performs person detection. Video input parsing supports multiple video formats: mp4, avi, mov, and mkv; it automatically acquires metadata information such as frame rate and resolution; and creates corresponding output video files while preserving the original video parameters. Real-time person detection and tracking: The YOLO detection and tracking model is used to perform multi-target detection on each frame, identifying targets classified as "person"; the persist=True parameter is used to achieve cross-frame target tracking, assigning a unique ID to each detected person; a detection confidence threshold of 0.7 is set to filter unreliable detection results; and the output includes bounding box coordinates, category labels, and tracking IDs.

[0066] Step S103: Determine the number of frames for detecting human behavior based on the initial model parameters, and obtain the multi-frame behavior detection map corresponding to the human target in the corresponding number of frames from the video stream data based on the number of frames for detecting human behavior.

[0067] For example, in this embodiment, based on the number of task behavior detection frames set by the user, and using the currently detected image frame as a basis, a multi-frame behavior detection map corresponding to the number of frames before and after the currently detected image frame is obtained from the video stream data. Then, the human features are detected using this multi-frame behavior detection map.

[0068] Step S104: Based on the behavior detection bounding box, crop the person region of the person target pair in the multi-frame behavior detection map to generate a multi-frame person region map. Then, use the behavior classification model in the theft behavior detection model to detect theft behavior of the person target in the multi-frame person region map and generate a multi-frame person region map. Figure 1 Confidence level of one corresponding to multiple theft behaviors.

[0069] In some embodiments, person region processing and behavior classification include: Region of Interest Extraction: The person region in the original frame is cropped according to the detection box coordinates; a boundary expansion algorithm is applied to ensure that it does not exceed the image boundary; an expanded region containing the person and the surrounding environment is generated to provide richer contextual information. Behavioral Feature Analysis: The cropped person region is input into the behavior classification model; the model outputs two classification results: "yes" (suspicious behavior) or "no" (normal behavior); it also outputs a classification confidence score, indicating the reliability of the model's judgment.

[0070] In some embodiments, historical data management and statistical analysis are performed on the video stream data. Time-series data storage: Two circular buffers are maintained for each tracking ID, storing the behavior classification labels and corresponding classification confidence scores of the most recent 7 frames respectively; a first-in, first-out (FIFO) strategy is adopted to keep the buffer length fixed at EVENT_N; Multi-frame statistical analysis algorithm: Calculate the number of high-confidence suspicious behaviors; Apply dual threshold judgment. Single-frame confidence threshold: conf > 0.7, and time-series proportion threshold: high_conf_yes_count > 0.65 × EVENT_N.

[0071] Optionally, in some embodiments, the step of determining the behavior judgment result corresponding to the target person based on the confidence levels of multiple theft behaviors includes:

[0072] Based on the initialized model parameters, determine the user-defined single-frame confidence threshold and time series proportion threshold;

[0073] Based on the confidence levels of multiple theft behaviors, determine the first number of behavior detection maps whose confidence levels are greater than the single-frame confidence threshold from the multi-frame behavior detection maps;

[0074] Based on the second and first quantities of the multi-frame behavior detection map, if the proportion of the first quantity to the second quantity is greater than the time series proportion threshold, the behavior determination result is determined to be theft behavior determination result;

[0075] If the proportion of the first quantity to the second quantity is less than or equal to the time series proportion threshold, the behavior determination result is determined to be a non-theft behavior determination result.

[0076] Optionally, in some embodiments, the method further includes:

[0077] If the behavior determination result is determined to be theft, the behavior determination result is sent to the user. In response to the user's behavior determination confirmation feedback, the multi-frame human area map is used as positive sample image.

[0078] If the behavior determination result is determined to be non-theft behavior, the multi-frame human region map will be used as negative sample images.

[0079] The behavior classification model is retrained using positive and negative sample images to update its model parameters.

[0080] Optionally, in some embodiments, the step is: to detect theft behavior of the target person in the multi-frame person region map by using the behavior classification model in the theft behavior detection model, and to generate multi-frame person region maps. Figure 1 The confidence levels of multiple theft behaviors corresponding to one include:

[0081] The human frame and hand key points of the target person in the first person region map are identified, and a human behavior recognition box including the hand key points is generated. The first person region map is any frame of the multi-frame person region map.

[0082] The behavior category of the human target in the human behavior recognition box is identified by the behavior classification model, and the first behavior feature corresponding to the first human region map is generated.

[0083] The confidence level of the first theft behavior is determined by comparing the first behavioral feature with the set theft behavior features.

[0084] Optionally, in some embodiments, the step of determining the behavior judgment result corresponding to the target person based on the confidence levels of multiple theft behaviors includes:

[0085] If the behavior judgment result is determined to be theft behavior judgment result based on the confidence of the first theft behavior, the hand behavior in the first person area map is analyzed by the independent tracking model to generate the hand posture features of the person target in the first person area map.

[0086] If, based on the hand gesture characteristics, the hand gesture of the target in the first person area map is determined to be a theft gesture, then the behavior determination result is determined to be the theft behavior determination result.

[0087] If, based on the hand posture characteristics, the hand posture of the target person in the first person area map is determined to be a non-stealing posture, then the behavior determination result is determined to be a non-stealing behavior determination result.

[0088] Step S105: Based on the confidence levels of multiple theft behaviors, determine the behavior judgment result corresponding to the target person. The behavior judgment result is used to indicate whether the target person has engaged in theft behavior within the time domain corresponding to the video stream data.

[0089] For example, the behavior determination and output are as follows: Behavior state determination: when a threshold condition is met, the classification is "Lifting" (theft). To further reduce false positives, the YOLO pose determination algorithm is used to further analyze the pose of the extended human body region. If the pose includes "Take" (taking goods from the shelf), the distance between the hands is determined. If it is less than a set threshold, the pose algorithm determines it as theft. If both the classification model and the pose algorithm determine it as theft, the final human behavior result is determined to be theft. Therefore, a red bounding box is displayed for the detected human body region, labeled "Lifting" and with a confidence level. This cropped human body region can be optionally saved to the positive sample library.

[0090] When the threshold condition is not met, it is judged as "Normal" (normal behavior), a green bounding box is displayed, labeled "Normal" and the confidence level is indicated, and it can be optionally saved to the negative sample library.

[0091] Visual output generation: The detection box, label and confidence information are overlaid on the original video frame to generate a new video file containing the analysis results. The analysis process is displayed in real time and supports manual monitoring.

[0092] Optionally, in some embodiments, the method further includes:

[0093] Based on hand posture features, determine the first distance between the palms of the target person in the human behavior recognition box;

[0094] If the distance is greater than a set distance threshold, the behavior determination result is determined to be theft behavior.

[0095] If the first distance is less than or equal to a set distance threshold, the behavior is determined to be a non-theft behavior.

[0096] For example, this embodiment performs data feedback and model optimization, and automatic sample collection: sample images are automatically classified and stored according to the behavior judgment results. Positive samples are named in the format: lifting_yes_date_number.jpg; negative samples are named in the format: lifting_no_date_number.jpg. Model iterative optimization: the collected samples are used for subsequent retraining of the classification model, continuously improving the model's recognition accuracy in specific scenarios, forming a closed-loop optimization of data collection → model training → performance improvement.

[0097] The above technical solution, in response to receiving user-inputted initialization model parameters, initializes the theft behavior detection model parameter configuration according to the initialization model parameters. Once the theft behavior detection model initialization is complete, it acquires images within the monitored area using an image acquisition device, generating video stream data. Then, using the independent tracking model in the theft behavior detection model to perform multi-target detection on the video stream data, if it determines that a human target exists in at least one image frame corresponding to the video stream data, it generates a human target behavior detection box based on the human target's position in the corresponding image frame. It determines the number of human behavior detection frames based on the initialization model parameters, and based on the number of human behavior detection frames, it obtains multi-frame behavior detection maps corresponding to the human target in the corresponding number of frames from the video stream data. It then crops the human region of the human target pair in the multi-frame behavior detection map based on the behavior detection boxes, generating a multi-frame human region map. Finally, it uses the behavior classification model in the theft behavior detection model to detect theft behavior of the human target in the multi-frame human region map, generating multi-frame human region maps. Figure 1 The system assigns confidence scores to multiple theft behaviors. Based on these scores, it determines the behavioral assessment result for the target person. This result indicates whether the target person engaged in theft within the corresponding time domain of the video stream data. By introducing a dual verification mechanism, the system tracks and locates each pedestrian in the video surveillance, and accurately determines the occurrence of theft through human posture judgment and action classification, thereby improving the intelligence level of the theft monitoring system.

[0098] Figure 2 This is a schematic diagram illustrating a theft detection device based on an AI visual model, according to an exemplary embodiment. Figure 2 As shown, the device 100 includes:

[0099] The first generation module 110 is used to respond to the initialization model parameters input by the user, initialize the configuration of the theft behavior detection model parameters according to the initialization model parameters, and, when it is determined that the theft behavior detection model initialization is completed, acquire images in the monitoring area based on the image acquisition device to generate video stream data.

[0100] The second generation module 120 is used to generate a behavior detection box for the person target based on the position of the person target in the corresponding image frame when the video stream data is determined to contain a person target by performing multi-target detection on the video stream data through the independent tracking model in the theft behavior detection model.

[0101] The acquisition module 130 is used to determine the number of human behavior detection frames based on the initialization model parameters, and to acquire the multi-frame behavior detection map corresponding to the human target in the corresponding number of frames from the video stream data based on the number of human behavior detection frames.

[0102] The third generation module 140 is used to crop the person regions of the person target pairs in the multi-frame behavior detection map based on the behavior detection box, generate a multi-frame person region map, and perform theft behavior detection on the behavior of the person target in the multi-frame person region map using the behavior classification model in the theft behavior detection model, generating a multi-frame person region map. Figure 1 Confidence level of one-to-one correspondence between multiple theft behaviors;

[0103] The execution module 150 is used to determine the behavior judgment result corresponding to the target person based on the confidence levels of multiple theft behaviors. The behavior judgment result is used to indicate whether the target person has engaged in theft behavior within the time domain corresponding to the video stream data.

[0104] Optionally, in some embodiments, the execution module 150 includes:

[0105] The first determination submodule is used to determine the user-defined single-frame confidence threshold and time series proportion threshold based on the initialized model parameters.

[0106] The second determining submodule is used to determine, based on multiple theft behavior confidence levels, a first number of behavior detection maps whose theft behavior confidence levels are greater than a single frame confidence threshold from the multi-frame behavior detection maps;

[0107] The first determination submodule is used to determine the behavior determination result as theft behavior determination result when the proportion of the first quantity to the second quantity is greater than the time series proportion threshold based on the second quantity and the first quantity of the multi-frame behavior detection map.

[0108] The second determination submodule is used to determine the behavior determination result as a non-theft behavior determination result when the proportion of the first quantity to the second quantity is less than or equal to the time series proportion threshold.

[0109] Optionally, in some embodiments, the device further includes an update module, which is used to:

[0110] If the behavior determination result is determined to be theft, the behavior determination result is sent to the user. In response to the user's behavior determination confirmation feedback, the multi-frame human area map is used as positive sample image.

[0111] If the behavior determination result is determined to be non-theft behavior, the multi-frame human region map will be used as negative sample images.

[0112] The behavior classification model is retrained using positive and negative sample images to update its model parameters.

[0113] Optionally, in some embodiments, the third generation module 140 includes:

[0114] The first generation submodule is used to identify the human body frame and hand key points of the human target in the first human region map, and generate a human behavior recognition box including the hand key points. The first human region map is any frame of the multi-frame human region map.

[0115] The second generation submodule is used to identify the behavior category of the human target in the human behavior recognition box through the behavior classification model, and generate the first behavior feature corresponding to the first human region map.

[0116] The third determination submodule is used to compare the first behavioral feature with the set theft behavior feature to determine the confidence level of the first theft behavior corresponding to the first person area map.

[0117] Optionally, in some embodiments, the execution module 150 further includes:

[0118] The third generation submodule is used to perform posture analysis on the hand behavior in the first person region map based on the independent tracking model when the behavior judgment result is determined to be theft behavior judgment result according to the confidence of the first theft behavior, and generate the hand posture features of the person target in the first person region map.

[0119] The third determination submodule is used to determine the behavior determination result as the theft behavior determination result when the hand posture of the target in the first person area map is determined to be a theft posture based on the hand posture characteristics.

[0120] The fourth determination submodule is used to determine the behavior determination result as a non-theft behavior determination result when the hand posture of the target in the first person area map is determined to be a non-theft behavior determination result based on the hand posture characteristics.

[0121] Optionally, in some embodiments, the device 100 further includes a determination module, which is used to:

[0122] Based on hand posture features, determine the first distance between the palms of the target person in the human behavior recognition box;

[0123] If the distance is greater than a set distance threshold, the behavior determination result is determined to be theft behavior.

[0124] If the first distance is less than or equal to a set distance threshold, the behavior is determined to be a non-theft behavior.

[0125] Optionally, in some embodiments, the first frame behavior detection map includes at least one of the following: bounding box coordinates, category label, and tracking ID, and the first frame behavior detection map is any behavior detection map among multiple frame behavior detection maps.

[0126] The above technical solution, in response to receiving user-inputted initialization model parameters, initializes the theft behavior detection model parameter configuration according to the initialization model parameters. Once the theft behavior detection model initialization is complete, it acquires images within the monitored area using an image acquisition device, generating video stream data. Then, using the independent tracking model in the theft behavior detection model to perform multi-target detection on the video stream data, if it determines that a human target exists in at least one image frame corresponding to the video stream data, it generates a human target behavior detection box based on the human target's position in the corresponding image frame. It determines the number of human behavior detection frames based on the initialization model parameters, and based on the number of human behavior detection frames, it obtains multi-frame behavior detection maps corresponding to the human target in the corresponding number of frames from the video stream data. It then crops the human region of the human target pair in the multi-frame behavior detection map based on the behavior detection boxes, generating a multi-frame human region map. Finally, it uses the behavior classification model in the theft behavior detection model to detect theft behavior of the human target in the multi-frame human region map, generating multi-frame human region maps. Figure 1 The system assigns confidence scores to multiple theft behaviors. Based on these scores, it determines the behavioral assessment result for the target person. This result indicates whether the target person engaged in theft within the corresponding time domain of the video stream data. By introducing a dual verification mechanism, the system tracks and locates each pedestrian in the video surveillance, and accurately determines the occurrence of theft through human posture judgment and action classification, thereby improving the intelligence level of the theft monitoring system.

[0127] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0128] Figure 3 This is a block diagram illustrating an electronic device according to an exemplary embodiment. Figure 3As shown, the electronic device 300 may include a processor 301 and a memory 302. The electronic device 300 may also include one or more of a multimedia component 303, an input / output (I / O) interface 304, and a communication component 305.

[0129] The processor 301 controls the overall operation of the electronic device 300 to complete all or part of the steps in the AI ​​vision model-based theft detection method described above. The memory 302 stores various types of data to support the operation of the electronic device 300. This data may include, for example, instructions for any application or method operating on the electronic device 300, and application-related data such as contact data, sent and received messages, images, audio, video, etc. The memory 302 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, disk, or optical disk. Multimedia component 303 may include a screen and an audio component. The screen may be, for example, a touchscreen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signals may be further stored in memory 302 or transmitted via communication component 305. The audio component also includes at least one speaker for outputting audio signals. I / O interface 304 provides an interface between processor 301 and other interface modules, such as a keyboard, mouse, buttons, etc. These buttons may be virtual or physical buttons. Communication component 305 is used for wired or wireless communication between the electronic device 300 and other devices. Wireless communication may include Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, or 4G, or a combination of these. Therefore, the corresponding communication component 305 may include a Wi-Fi module, a Bluetooth module, or an NFC module.

[0130] In an exemplary embodiment, the electronic device 300 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above-described theft detection method based on an AI vision model.

[0131] In another exemplary embodiment, a computer-readable storage medium is also provided, which includes program instructions that, when executed by a processor, implement the steps of the above-described AI vision model-based theft detection method. For example, the computer-readable storage medium may be the memory 302 including the program instructions, which may be executed by the processor 301 of the electronic device 300 to complete the above-described AI vision model-based theft detection method.

[0132] In another exemplary embodiment, a computer program product is also provided, which includes a computer program executable by a processor, which, when executed by the processor, implements the steps of the above-described theft detection method based on an AI visual model.

[0133] The preferred embodiments of this disclosure have been described in detail above with reference to the accompanying drawings. However, this disclosure is not limited to the specific details of the above embodiments. Within the scope of the technical concept of this disclosure, various simple modifications can be made to the technical solutions of this disclosure, and these simple modifications all fall within the protection scope of this disclosure.

[0134] It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction.

[0135] Furthermore, various different embodiments of this disclosure can be combined in any way, as long as they do not violate the spirit of this disclosure, they should also be regarded as the content disclosed in this disclosure.

Claims

1. A method for detecting theft behavior based on an AI visual model, characterized in that, The method includes: In response to receiving user input of initialization model parameters, the theft behavior detection model parameter configuration is initialized according to the initialization model parameters, and when it is determined that the theft behavior detection model initialization is complete, images in the monitored area are acquired based on the image acquisition device to generate video stream data; When the video stream data is multi-target detected by the independent tracking model in the theft behavior detection model, and it is determined that there is a human target in at least one image frame corresponding to the video stream data, a behavior detection box of the human target is generated according to the position of the human target in the corresponding image frame. The number of frames for detecting human behavior is determined based on the initialization model parameters, and based on the number of frames for detecting human behavior, a multi-frame behavior detection map corresponding to the human target is obtained from the video stream data for the corresponding number of frames. Based on the behavior detection box, the person region of the person target pair in the multi-frame behavior detection map is cropped to generate a multi-frame person region map. The behavior classification model in the theft behavior detection model is used to detect theft behavior of the person target in the multi-frame person region map and generate multiple theft behavior confidence scores corresponding to the multi-frame person region map. Based on the confidence levels of the multiple theft behaviors, a behavior determination result corresponding to the target person is determined. The behavior determination result is used to indicate whether the target person has engaged in theft behavior within the time domain corresponding to the video stream data. The theft behavior detection model uses a behavior classification model to detect theft behavior of the target in the multi-frame character region map, generating multiple theft behavior confidence scores corresponding to each of the multi-frame character region map, including: The human body frame and hand key points of the target person in the first person region map are identified, and a human behavior recognition box including the hand key points is generated. The first person region map is any frame of the multi-frame person region map. The behavior classification model is used to identify the behavior category of the human target in the human behavior recognition box, and a first behavior feature corresponding to the first human region map is generated. The first behavioral feature is compared with the set theft behavior features to determine the confidence level of the first theft behavior corresponding to the first person area map; The step of determining the behavior judgment result corresponding to the target person based on the confidence levels of the multiple theft behaviors includes: If the behavior determination result is determined to be theft behavior determination result based on the first theft behavior confidence, the hand behavior in the first person region map is analyzed according to the independent tracking model to generate the hand posture features of the person target in the first person region map. If, based on the hand gesture features, the hand gesture of the target person in the first person area map is determined to be a theft gesture, then the behavior determination result is determined to be the theft behavior determination result. If, based on the hand gesture characteristics, it is determined that the hand gesture of the target person in the first person area map is not a theft gesture, then the behavior determination result is determined to be a non-theft behavior determination result. The method further includes: Based on the hand posture features, determine the first distance between the palms of the human target in the human behavior recognition box; If the first distance is greater than a set distance threshold, the behavior determination result is determined to be the theft behavior determination result; If the first distance is less than or equal to the set distance threshold, the behavior determination result is determined to be the non-theft behavior determination result.

2. The theft detection method based on an AI visual model according to claim 1, characterized in that, The step of determining the behavior judgment result corresponding to the target person based on the confidence levels of the multiple theft behaviors includes: Based on the initialization model parameters, determine the user-defined single-frame confidence threshold and time series proportion threshold; Based on the confidence levels of the multiple theft behaviors, a first number of behavior detection maps with a confidence level greater than the single-frame confidence threshold are determined from the multi-frame behavior detection maps; If, based on the second quantity and the first quantity of the multi-frame behavior detection map, it is determined that the proportion of the first quantity to the second quantity is greater than the time series proportion threshold, the behavior determination result is determined to be theft behavior determination result; If the proportion of the first quantity to the second quantity is less than or equal to the time series proportion threshold, the behavior determination result is determined to be a non-theft behavior determination result.

3. The theft detection method based on an AI visual model according to claim 2, characterized in that, The method further includes: If the behavior determination result is determined to be the theft behavior determination result, the behavior determination result is sent to the user. In response to the behavior determination confirmation feedback from the user, the multi-frame human region map is used as a positive sample image. If the behavior determination result is determined to be the non-theft behavior determination result, the multi-frame character area map is used as a negative sample image; The behavior classification model is retrained based on the positive and negative sample images to update the model parameters.

4. The theft detection method based on an AI visual model according to any one of claims 1-3, characterized in that, The first frame behavior detection map includes at least one of the following: bounding box coordinates, category label, and tracking ID. The first frame behavior detection map is any behavior detection map in the multi-frame behavior detection map.

5. A theft detection device based on an AI visual model, characterized in that, The device includes: The first generation module is used to respond to the initialization model parameters input by the user, initialize the configuration of the theft behavior detection model parameters according to the initialization model parameters, and, when it is determined that the initialization of the theft behavior detection model is completed, acquire images in the monitored area based on the image acquisition device to generate video stream data. The second generation module is used to generate a behavior detection box for the person target based on the position of the person target in the corresponding image frame when the video stream data is determined to contain a person target by performing multi-target detection on the video stream data through the independent tracking model in the theft behavior detection model. The acquisition module is used to determine the number of human behavior detection frames according to the initialization model parameters, and based on the number of human behavior detection frames, to acquire the multi-frame behavior detection map corresponding to the human target in the corresponding number of frames from the video stream data; The third generation module is used to crop the human region of the target pair in the multi-frame behavior detection map according to the behavior detection box, generate a multi-frame human region map, and perform theft behavior detection on the human target in the multi-frame human region map through the behavior classification model in the theft behavior detection model, generating multiple theft behavior confidence scores corresponding to each of the multi-frame human region map; the step of performing theft behavior detection on the human target in the multi-frame human region map through the behavior classification model in the theft behavior detection model, generating multiple theft behavior confidence scores corresponding to each of the multi-frame human region map includes: identifying the human frame and hand key points of the target in the first human region map, generating a human behavior recognition box including the hand key points, wherein the first human region map is any frame of the multi-frame human region map; identifying the behavior category of the target in the human behavior recognition box map through the behavior classification model, generating a first behavior feature corresponding to the first human region map; comparing the first behavior feature with the set theft behavior feature to determine the first theft behavior confidence score corresponding to the first human region map. The step of determining the behavior judgment result corresponding to the target person based on the confidence levels of the multiple theft behaviors includes: when the behavior judgment result is determined to be a theft behavior judgment result based on the first theft behavior confidence level, performing posture analysis on the hand behavior in the first person region map based on the independent tracking model to generate hand posture features of the target person in the first person region map; when the hand posture features are used to determine that the hand posture of the target person in the first person region map is a theft posture, determining the behavior judgment result as the theft behavior judgment result; when the hand posture features are used to determine that the hand posture of the target person in the first person region map is a non-theft posture, determining the behavior judgment result as a non-theft behavior judgment result; determining a first distance between the corresponding palms of the target person in the human behavior recognition box based on the hand posture features; when the first distance is greater than a set distance threshold, determining the behavior judgment result as the theft behavior judgment result; when the first distance is less than or equal to the set distance threshold, determining the behavior judgment result as the non-theft behavior judgment result. The execution module is used to determine the behavior judgment result corresponding to the target person based on the confidence levels of the multiple theft behaviors. The behavior judgment result is used to indicate whether the target person has engaged in theft behavior within the time domain corresponding to the video stream data.

6. An electronic device, characterized in that, include: A memory on which computer programs are stored; A processor for executing the computer program in the memory to implement the steps of the method according to any one of claims 1-4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the method described in any one of claims 1-4.

Images