Human body small target behavior detection method based on target detection

Abstract

The invention discloses a human body small target behavior detection method based on target detection. Comprising the following steps: S1, data collection and preprocessing; s2, using supervised learning to train a head and hand detection model Mh; s3, data enhancement and annotation; s4, training a smoking detection model Ms; and S5, fusing the head-hand detection model Mh and the training smoking detection model Ms, and detecting smoking behaviors. According to the method, distance judgment of a smoke recognition frame and a head and hand recognition frame and a scoring mechanism for multi-frame judgment of the smoking behavior are introduced, the accuracy of smoking behavior detection is improved, data obtained after super-score and data obtained before super-score are combined to train the target detection model, domain differences are effectively reduced, and the accuracy of smoking behavior detection is improved. And the performance of smoking behavior detection in low-resolution images and videos is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision and deep learning technology, and in particular to a method for detecting small human target behavior based on target detection. Background Art

[0002] With the rapid development of computer vision and deep learning technology, image recognition and behavior detection have made significant progress. These advances are mainly reflected in the processing of high-resolution images, such as facial recognition, object detection, and behavior analysis. However, for the processing of low-resolution images, especially some human behavior detection based on target detection, such as smoking, making phone calls, eating snacks, etc., the objects to be detected are often small in the picture, which increases the difficulty of detection and adds a lot of difficulties to subsequent work.

[0003] Low-resolution images have poor quality, which makes it difficult to extract useful features from them. For example, the detection of smoking behavior usually requires the identification of details such as smoke, cigarette butts, and human faces, the detection of phone calls usually requires the identification of details such as mobile phones and human hands, and some other behavior detections often require the identification of very small objects. This information may not be clearly presented in low-resolution images.

[0004] Existing deep learning models, such as convolutional neural networks (CNNs) and recurrent neural networks (RNNs), usually require a large amount of high-quality training data to achieve good performance. However, obtaining a large amount of low-resolution smoking behavior image data is challenging, which limits the application of these models to smoking behavior detection in low-resolution images.

[0005] Existing small target behavior detection methods usually face many challenges. These behaviors, such as making phone calls, eating snacks, smoking, etc., often occupy a small area in the image, and the movements are subtle and difficult to accurately identify. Traditional detection methods usually require high-resolution images to accurately capture these details. However, in many practical applications, such as surveillance cameras in public places, traffic monitoring systems, etc., the acquired images are usually low-resolution due to the resolution limitation of the camera or the long shooting distance.

[0006] In addition, the behaviors of these small targets are diverse and greatly affected by factors such as environment, angle, and occlusion. For example, a phone may be partially blocked when making a call, the movements may be fast when eating snacks, and the smoke may not be obvious when smoking. These factors increase the difficulty of detection.

[0007] Therefore, it becomes particularly important to develop a method that can accurately detect various small target behaviors in low-resolution images. Summary of the invention

[0008] The purpose of the present invention is to overcome the problems existing in the above-mentioned prior art and to provide a method for detecting small target behaviors of the human body based on target detection, which can effectively perform real-time detection of behaviors mainly of small targets under the surveillance camera, and is of great significance to ensuring public health and safety.

[0009] In order to achieve the above object, the present invention provides a method for detecting small human target behavior based on target detection, comprising the following steps:

[0010] S1, data collection and preprocessing;

[0011] S2. Use supervised learning to train the head and hand detection model M h ;

[0012] S3, data enhancement and annotation;

[0013] S4. Training smoking detection model M s ;

[0014] S5, the head and hand detection model M h And train the smoking detection model M s Fusion and detection of smoking behavior.

[0015] Preferably, step S1 specifically includes the following steps:

[0016] S101, using a surveillance camera to shoot a batch of smoking videos that conform to real smoking scenes, and retaining one frame every five frames as a smoking data set;

[0017] S102, eliminating the data that is blurred or blocked in the data set and affects the model training;

[0018] S103, using labelimg to mark the two categories of hand and head as the head and hand detection model M h The dataset S h , and divided into training data and test samples in a ratio of 8:2.

[0019] Preferably, the head-hand detection model M h The basic framework is the YOLOv9 target detection model.

[0020] Preferably, the specific training process of step S2 is expressed as:

[0021] M h =f det1 (I,D)

[0022] Among them, M h represents the trained head and hand detection model, f det1 (·) indicates YOLOv9 object detection model training.

[0023] Preferably, step S3 specifically comprises: converting the data set S h Cut and label the part containing smoke as data set S s , and then use the HAT super-resolution model to enhance the data, which can be expressed as:

[0024]

[0025]

[0026] in, Represents the cropped image, using represents the corresponding labeled data, cut(·) represents the cropping function, SR(·) represents the super-resolution model, Represents the image after super resolution, and uses Indicates the corresponding annotation data.

[0027] Preferably, step S4 specifically comprises: combining the data after super resolution and the data before super resolution to detect the smoking detection model M s For training, the whole training process is:

[0028]

[0029] Among them, M s represents the trained smoking detection model, f det2 (·) indicates the YOLOv9 object detection model training combining the data after super-resolution with the data before super-resolution.

[0030] Preferably, the head-hand detection model M h And train the smoking detection model M s The fusion is specifically to transform the head and hand detection model M h The output of the smoking detection model M s The smoking detection model M s The output of contains confidence signal and distance information.

[0031] Preferably, the smoking behavior detection is implemented by combining distance determination and multi-frame determination.

[0032] Preferably, the distance determination is specifically as follows: calculating the smoke recognition frame B s Identification box B with head and hands h The distance is compared with a preset distance threshold λ. If the distance is less than λ, it is considered that the recognition box B s Identification box B with head and hands h The distance is close enough to determine that the person is smoking. The calculation of the smoke recognition frame Bs Identification box B with head and hands h The distance function is:

[0033] dis(B s ,B h )<λ

[0034] Among them, B s Represents the smoke identification box, B h represents the recognition box of the head and hand, dis(·) represents the distance function, and λ represents the distance threshold which is a hyperparameter.

[0035] Preferably, the multi-frame determination is specifically as follows: calculating the judgment score of the smoking behavior in x consecutive frames, and comparing it with a preset score threshold α; if the number of frames with a score greater than α in x frames exceeds the multi-frame determination threshold δ, it is considered that the target is smoking, which is expressed by the formula:

[0036] Count((n1,n2,…n x ),α)>δ

[0037] Among them, n1,n2,…n x represents the judgment score of smoking behavior in frame x, α represents the judgment score threshold, Count(·) represents the number of values ​​greater than the score threshold, and δ represents the threshold for multi-frame judgment.

[0038] Compared with the prior art, the present invention has the following beneficial effects:

[0039] 1. This paper proposes a small target behavior detection algorithm that can be effectively applied to low-resolution videos. Taking smoking behavior detection as an example, the algorithm adopts a joint framework of two target detection models. During the entire training process, the head and hand detection model M h The output of the smoking detection model M s In addition, the distance between the smoke recognition frame and the head and hand recognition frames, as well as the scoring mechanism for multi-frame judgment of smoking behavior, are introduced to improve the accuracy of smoking behavior detection.

[0040] 2. In the data preprocessing stage, the present invention annotates the head and hands of the person in the low-resolution image and enhances the data using the super-resolution model. In the model training stage, the object detection model is trained by combining the super-resolution data with the pre-super-resolution data. This method promotes the deep fusion and interaction between models, improves the model's ability to capture visual information, and effectively reduces domain differences, significantly improving the performance of smoking behavior detection in low-resolution images and videos. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1A flow chart of a method for detecting small human target behavior based on target detection provided by the present invention;

[0042] Figure 2 The overall network structure diagram provided by the present invention;

[0043] Figure 3 A visualization result diagram of the detection results on the monitored data provided by the present invention. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] The present invention aims to provide a method for detecting small human target behaviors based on target detection in order to solve the problems existing in the prior art. The method needs to have the following characteristics:

[0046] 1. It can effectively extract key features from low-resolution images and identify small changes in motion.

[0047] 2. It has strong robustness and can adapt to different environmental conditions and behavioral changes.

[0048] 3. Achieve good performance even with limited training data, since there may be fewer samples of certain behaviors.

[0049] 4. Capable of multi-category behavior recognition, and simultaneous detection and classification of multiple small target behaviors.

[0050] 5. It has real-time processing capabilities and can respond quickly in various practical application scenarios.

[0051] Such detection methods will have important impacts in multiple fields. In public safety, potential violations can be detected in a timely manner, such as smoking in a no-smoking area. In traffic management, driver distraction can be identified, such as talking on the phone while driving. In retail, customer shopping behavior can be analyzed. In the field of healthcare, patients' daily behavior can be monitored to assist in diagnosis and treatment.

[0052] Developing this advanced small target behavior detection technology is not only a technical challenge, but also an important step to promote the wider application of computer vision and artificial intelligence in practical applications. It will bring new possibilities to smart cities, intelligent monitoring, behavior analysis and other fields, but also requires full consideration of privacy protection and ethical use issues while developing technology. Next, the present invention will be introduced in detail using smoking detection as an example.

[0053] The present invention proposes a method for detecting small human target behaviors based on target detection, such as Figure 1 As shown, the specific steps are as follows:

[0054] Step S1, data collection and preprocessing. Specifically, first use a surveillance camera to shoot a batch of smoking videos that match the actual smoking scene, and retain one frame every five frames as the smoking data set. Eliminate the data that is blurred, blocked, and other data that affect model training, and use labelimg to annotate the two categories of hand and head. This is used as the head and hand detection model M in step S2. h The dataset S h . And divided into training data and test samples in the ratio of 8:2.

[0055] Step S2: Use supervised learning to train the head and hand detection model M h Specifically, the preprocessed and labeled data is used to train the head and hand detection model M h The goal of this model is to accurately identify the head and hands of a person in a new image, where I and D represent the image and annotation respectively, and the YOLOv9 target detection model performs this task. The training process is expressed as,

[0056] M h =f det1 (I,D)

[0057] Among them, M h represents the trained head and hand detection model, f det1 (·) indicates YOLOv9 object detection model training.

[0058] Step S3: Data enhancement and labeling. Specifically, the head and hand image data within the detection frame labeled in step S1 is intercepted, and the part containing smoke is labeled as the data set S s ; Then, the HAT super-resolution model is used to enhance the data to improve the performance of the model. The overall process of data enhancement and annotation is:

[0059]

[0060]

[0061] in, Represents the cropped image, using represents the corresponding labeled data, cut(·) represents the cropping function, SR(·) represents the super-resolution model, Represents the image after super resolution, and uses Indicates the corresponding annotation data.

[0062] Step S4: training the smoking detection model M s Specifically, the data after super-resolution and the data before super-resolution are combined to perform the smoking detection model M s The goal of this model is to accurately identify smoke in new images. The entire training process is:

[0063]

[0064] Among them, M s Indicates the smoking detection model obtained through training, f det2 (·) indicates the YOLOv9 object detection model training that combines super-resolution images with original images.

[0065] Step S5: the head and hand detection model M h And train the smoking detection model M s Fusion and detection of smoking behavior. Specifically, the head and hand detection model M h And train the smoking detection model M s The fusion is specifically to transform the head and hand detection model M h The output of the smoking detection model M s The smoking detection model M s The output includes a confidence signal and distance information. The smoking behavior detection is implemented by combining distance determination and multi-frame determination. The present invention uses the distance determination and multi-frame determination mechanism to improve the accuracy of model recognition.

[0066] The distance determination is specifically as follows: calculating the smoke recognition frame B s Identification box B with head and hands h The distance is compared with a preset distance threshold λ. If the distance is less than λ, it is considered that the recognition box B s Identification box B with head and hands h The distance is close enough to determine that it is possible that the person is smoking, so as to reduce false positives and false negatives as much as possible while ensuring the detection accuracy. The calculation of the smoke recognition frame B s Identification box B with head and hands h The distance function is:

[0067] dis(B s ,B h )<λ

[0068] Among them, B s Represents the smoke identification box, B h represents the recognition box of the head and hand, dis(·) represents the distance function, and λ represents the distance threshold which is a hyperparameter.

[0069] Multi-frame judgment, that is, combining the scoring results of multi-frame detection, to make a more accurate judgment on the smoking process. In this process, not only the detection results of a single frame image are considered, but also the detection results of continuous multi-frame images are combined. Specifically: the judgment score of smoking behavior in continuous x frames is calculated and compared with a preset score threshold α. If the number of frames with a score greater than α exceeds the multi-frame judgment threshold δ within x frames, it is considered that the target is smoking. It can be expressed as:

[0070] Count((n1,n2,…n x ),α)>δ

[0071] Among them, n1,n2,…n x represents the judgment score of smoking behavior in frame x, α represents the judgment score threshold, Count(·) represents the number greater than the score threshold, and δ represents the threshold for multi-frame judgment.

[0072] This threshold δ can also be determined by cross-validation or other optimization methods. This method can effectively reduce false positives and false negatives caused by image noise, environment or other interference factors, thereby improving the accuracy of smoking behavior detection. In this way, smoking behavior detection is finally achieved for low-resolution images.

[0073] The present invention adds distance determination between the identification frame of the cigarette and the identification frames of the head and the hand, and combines multiple frames to determine the score of the smoking behavior, thereby ultimately achieving the detection of smoking behavior on low-resolution images.

[0074] The overall process of the smoking behavior detection of the present invention is as follows:

[0075] F smoke (V t ,M h ,M s )

[0076] Among them, F smoke (·) represents the entire smoking behavior detection algorithm framework, V t Represents the input video.

[0077] Although the present invention is described herein with reference to specific embodiments, it should be understood that these embodiments are merely examples of the principles and applications of the present invention. It should therefore be understood that many modifications may be made to the exemplary embodiments and that other arrangements may be devised without departing from the spirit and scope of the present invention as defined by the appended claims. It should be understood that the various dependent claims and features described herein may be combined in a manner different from that described in the original claims. It should also be understood that features described in conjunction with individual embodiments may be used in other described embodiments.

Claims

1. A method for detecting small human target behavior based on target detection, characterized in that: The following steps are involved: S1, data collection and preprocessing; S2. Use supervised learning to train the head and hand detection model M h ; S3, data enhancement and annotation; S4. Training smoking detection model M s ; S5, the head and hand detection model M h And train the smoking detection model M s Fusion and detection of smoking behavior.

2. According to the method for detecting small human target behaviors based on target detection in claim 1, it is characterized in that: Step S1 specifically includes the following steps: S101, using a surveillance camera to shoot a batch of smoking videos that conform to real smoking scenes, and retaining one frame every five frames as a smoking data set; S102, eliminating the data that is blurred or blocked in the data set and affects the model training; S103, using labelimg to mark the two categories of hand and head as the head and hand detection model M h The dataset S h , and divided into training data and test samples in a ratio of 8:

2.

3. The method for detecting small human target behaviors based on target detection according to claim 2, characterized in that: The head and hand detection model M h The basic framework is the YOLOv9 target detection model.

4. The method for detecting small human target behaviors based on target detection according to claim 3, characterized in that: The specific training process of step S2 is expressed as: M h =f det1 (I,D) Among them, M h represents the trained head and hand detection model, f det1 (·) indicates YOLOv9 object detection model training.

5. The method for detecting small human target behaviors based on target detection according to claim 4, characterized in that: Step S3 specifically includes: converting the data set S h Cut and label the part containing smoke as data set S s , and then use the HAT super-resolution model to enhance the data, which can be expressed as: in, Represents the cropped image, using represents the corresponding labeled data, cut(·) represents the cropping function, SR(·) represents the super-resolution model, Represents the image after super resolution, and uses Indicates the corresponding annotation data.

6. The method for detecting small human target behaviors based on target detection according to claim 5, characterized in that: Step S4 specifically includes: combining the data after super resolution with the data before super resolution to detect the smoking detection model M s For training, the whole training process is: Among them, M s represents the trained smoking detection model, f det2 (·) indicates the YOLOv9 object detection model training combining the data after super-resolution with the data before super-resolution.

7. The method for detecting small human target behaviors based on target detection according to claim 6, characterized in that: The head and hand detection model M h And train the smoking detection model M s The fusion is specifically to transform the head and hand detection model M h The output of the smoking detection model M s The smoking detection model M s The output of contains confidence signal and distance information.

8. The method for detecting small human target behaviors based on target detection according to claim 6, characterized in that: The smoking behavior detection is achieved by combining distance determination and multi-frame determination.

9. The method for detecting small human target behaviors based on target detection according to claim 8, characterized in that: The distance determination is specifically as follows: calculating the smoke recognition frame B s Identification box B with head and hands h The distance is compared with a preset distance threshold λ. If the distance is less than λ, it is considered that the recognition box B s Identification box B with head and hands h The distance is close enough to determine that the person is smoking. The calculation of the smoke recognition frame B s Identification box B with head and hands h The distance function is: dis(B s ,B h )<λ Among them, B s Represents the smoke identification box, B h represents the recognition box of the head and hand, dis(·) represents the distance function, and λ represents the distance threshold which is a hyperparameter.

10. The method for detecting small human target behaviors based on target detection according to claim 9, characterized in that: The multi-frame judgment is specifically as follows: calculating the judgment score of the smoking behavior in x consecutive frames and comparing it with a preset score threshold α. If the number of frames with a score greater than α in x frames exceeds the multi-frame judgment threshold δ, it is considered that the target is smoking, which can be expressed as: Count((n1,n2,…n x ),a)>d Among them, n1,n2,…n x represents the judgment score of smoking behavior in frame x, α represents the judgment score threshold, Count(·) represents the number of values ​​greater than the score threshold, and δ represents the threshold for multi-frame judgment.

Images