Construction method of target behavior detection model
By improving the information fusion technology of the backbone and neck network modules, the problem of low accuracy and real-time performance of the target behavior detection model is solved, and more efficient feature extraction and prediction accuracy is achieved.
Patent Information
- Application Number
- CN202411452113.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-17
- Publication Date
- 2025-01-07
AI Technical Summary
Due to insufficient information fusion, the existing target behavior detection model has low detection accuracy and poor real-time performance.
The improved backbone network module (CBH, C5_1, SPPFS) and neck network module (C5_2) are adopted to increase information fusion through multiple channel superposition and feature fusion, reduce parameter amount and calculation amount, and improve model prediction accuracy.
It improves the accuracy and real-time nature of target behavior detection, enhances the propagation of shallow semantic information in the image, balances the feature extraction of deep and shallow networks, and improves the accuracy of prediction box position.
Smart Images
Figure CN119273899A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of target detection, and in particular to a method for constructing a target behavior detection model. Background Art
[0002] With the continuous development of target detection technology, this technology has been widely used in fields such as autonomous driving and video surveillance. Target detection not only includes the detection of the target itself, but also the detection of the target behavior. In recent years, artificial intelligence technology has risen rapidly, and target detection technology based on deep learning has been widely used in automatic detection of target behavior.
[0003] Currently, existing target behavior related detection models usually use the following methods for detection:
[0004] (1) Using deep neural networks to extract features of target behavior gestures, thereby achieving automatic detection of target behaviors;
[0005] (2) By establishing a target detection model based on a deep learning algorithm, specific targets can be identified, thereby achieving automatic detection of target behavior.
[0006] The above method has the following defects:
[0007] For method (1), due to the factors such as the variable target behavior gestures, the diverse skin colors of human bodies, the different camera shooting angles, and the easy confusion with gestures similar to the target behavior, the recognition misjudgment rate is large and the accuracy of target behavior detection is low;
[0008] For method (2), some specific targets are difficult to identify due to factors such as their small size and easy confusion with similar targets, resulting in a low accuracy rate for target behavior detection. In addition, the established target detection model has a high spatiotemporal complexity, resulting in a low real-time performance for target behavior detection.
[0009] It can be seen that providing a target behavior detection model with high detection accuracy is an urgent problem to be solved. Summary of the invention
[0010] In view of the above analysis, the present invention aims to provide a method for constructing a target behavior detection model to solve the problem that the existing target behavior related detection models have insufficient information fusion, are not conducive to feature extraction, and result in low detection accuracy.
[0011] The present invention provides a method for constructing a target behavior detection model, the method comprising the following steps:
[0012] Building an initial target behavior detection model based on the YOLOv5s model, wherein the initial target behavior detection model includes an input end, a backbone network, a neck network, and a detection end;
[0013] The CBS module of the backbone network is improved to obtain a CBH module, the C3_1 module of the backbone network is improved to obtain a C5_1 module, and the SPPF module of the backbone network is improved to obtain a SPPFS module;
[0014] The C3_2 module of the neck network is improved to obtain the C5_2 module;
[0015] The CBS module, C3_1 module, and SPPF module in the backbone network are replaced by the CBH module, C5_1 module, and SPPFS module, respectively, and the CBS module and C3_2 module in the neck network are replaced by the CBH module and C5_2 module, respectively, so as to construct a target behavior detection model;
[0016] Target behavior images are collected and annotated to construct a data set, and the target behavior detection model is trained based on the data set to obtain a trained target behavior detection model.
[0017] Furthermore, the CBH module includes a convolutional layer, a batch normalization layer, and an NHSwish activation function connected in sequence.
[0018] Furthermore, the expression of the NHSwish activation function is:
[0019]
[0020] Here, x represents the output of the batch normalization layer.
[0021] Furthermore, the C5_1 module includes the first and second segmentation layers, the first to fifth CBH modules, and a feature fusion layer; wherein the input feature map is divided into two paths through the first segmentation layer and then respectively input into the feature fusion layer through the first CBH module and the second CBH module, the output of the second CBH module is also divided into two paths through the second segmentation layer and then respectively input into the feature fusion layer through the third CBH module and the fourth CBH module, and the output of the feature fusion layer is input into the fifth CBH module.
[0022] Furthermore, the C5_2 module includes a segmentation layer, first to fifth CBH modules, and a feature fusion layer; wherein the input feature map is divided into two paths through the segmentation layer and then input into the feature fusion layer through the first CBH module and the second CBH module respectively, the output of the second CBH module is also input into the feature fusion layer through the third CBH module, the output of the third CBH module is also input into the feature fusion layer through the fourth CBH module, and the output of the feature fusion layer is input into the fifth CBH module.
[0023] Furthermore, the SPPFS module includes a segmentation layer, a first and a second CBH module, a first feature fusion layer, and a maximum pooling module; wherein the input feature map is divided into two paths through the segmentation layer and then respectively input into the first feature fusion layer through the maximum pooling module and the first CBH module, and the output of the first feature fusion layer is input into the second CBH module; the maximum pooling module includes a third CBH module, three maximum pooling layers, a second feature fusion layer, and a fourth CBH module connected in sequence, and the output of the third CBH module and the output of the three maximum pooling layers are also jump-connected to the second feature fusion layer.
[0024] Furthermore, the CBH module is used to replace all CBS modules in the backbone network and the neck network of the initial target behavior detection model; the C5_1 module is used to replace all C3_1 modules in the backbone network of the initial target behavior detection model; the SPPFS module is used to replace the SPPF module in the backbone network of the initial target behavior detection model; the C5_2 module is used to replace all C3_2 modules in the neck network of the initial target behavior detection model, thereby constructing a target behavior detection model.
[0025] Furthermore, in the target behavior detection model:
[0026] The output of the third layer C5_1 module in the backbone network, i.e., the p2 unit, is input to the first layer CBH module in the neck network FPN;
[0027] The output of the SPPFS module in the backbone network is input to the sixth feature fusion layer in the neck network PAN;
[0028] The output of the second feature fusion layer in the neck network FPN is input to the first CBH module in the neck network PAN.
[0029] Furthermore, the target behavior images are preprocessed, enhanced, and annotated to construct a dataset, including:
[0030] Eliminate target behavior images with unclear target feature points, and adjust the pixel size and format of the remaining target behavior images to be consistent;
[0031] Performing image enhancement processing on the target behavior image after pixel size adjustment;
[0032] The target behavior images after pixel size adjustment and image enhancement are annotated to construct a dataset, and the dataset is divided into a training set and a test set.
[0033] Furthermore, the image enhancement processing of the target behavior image after pixel size adjustment includes one or more of the following methods:
[0034] Randomly selecting part of the target behavior image and copying it to other images to generate a new target behavior image;
[0035] Performing a two-dimensional linear transformation on the target behavior image to generate a new target behavior image; the two-dimensional linear transformation includes rotation, translation, scaling, shearing, and flipping;
[0036] Randomly selecting two images from the target behavior image, mixing the two images in a certain ratio, and generating a new target behavior image;
[0037] Randomly adjusting the hue, saturation, and brightness of the target behavior image to generate a new target behavior image;
[0038] Randomly cover a part of the target behavior image to generate a new target behavior image.
[0039] Compared with the prior art, the present invention can achieve at least one of the following beneficial effects:
[0040] 1. The NHSwish activation function proposed in the present invention uses a piecewise function to simplify the expression, which improves the calculation efficiency and is suitable for embedded devices with limited resources.
[0041] 2. The C5_1 module and C5_2 module proposed in the present invention are composed of CBH modules through multiple channel superposition, which not only reduces the number of parameters and calculations, but also increases the information fusion between different channels, providing a basis for improving the accuracy of model prediction.
[0042] 3. The SPPFS module proposed in the present invention has a deeper information fusion effect by increasing the information fusion between different channels, which is conducive to more comprehensive extraction of global feature information and provides a basis for improving the accuracy of model prediction.
[0043] 4. The present invention inputs the output of the p2 unit in the backbone network to the neck network FPN and inputs the output of the second feature fusion layer in the FPN to the PAN, which can reduce the attenuation of the feature information of small targets in the multi-layer convolution calculation process, and enables the feature information to be fused in the FPN and PAN, thereby enhancing the propagation of shallow semantic information of the image, balancing the contradiction between the deep network for target category recognition and the shallow network for small target positioning and extracting image features of different scales; the output of the SPPFS module in the backbone network is directly input into the neck network PAN as an information flow, which improves the accuracy of the prediction box position and provides a basis for improving the accuracy of model prediction.
[0044] In the present invention, the above-mentioned technical solutions can also be combined with each other to achieve more preferred combination solutions. Other features and advantages of the present invention will be described in the subsequent description, and some advantages can become obvious from the description, or can be understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained through the contents particularly pointed out in the description and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] The drawings are only for the purpose of illustrating particular embodiments and are not to be considered limiting of the present invention. Like reference symbols denote like components throughout the drawings.
[0046] Figure 1 Flow chart of a method for constructing a target behavior detection model according to an embodiment of the present invention;
[0047] Figure 2 A schematic diagram of the network structure of the initial target behavior detection model according to an embodiment of the present invention;
[0048] Figure 3 It is a schematic diagram of the structure of the CBH, C5_1 and C5_2 modules according to an embodiment of the present invention;
[0049] Figure 4 It is the NHSwish activation function image and its derivative function image according to the embodiment of the present invention;
[0050] Figure 5 This is a schematic diagram of the structure of the SPPFS module according to an embodiment of the present invention;
[0051] Figure 6 Schematic diagram of the network structure of the target behavior detection model according to an embodiment of the present invention. DETAILED DESCRIPTION
[0052] The preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings, wherein the accompanying drawings constitute a part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not used to limit the scope of the present invention.
[0053] A specific embodiment of the present invention discloses a method for constructing a target behavior detection model. Figure 1 As shown, the method comprises the following steps:
[0054] Step S1, constructing an initial target behavior detection model based on the YOLOv5s model, wherein the initial target behavior detection model includes an input end (Input), a backbone network (Backbone), a neck network (Neck) and a detection end (Detect);
[0055] Step S2, improving the CBS module of the backbone network to obtain a CBH module, improving the C3_1 module of the backbone network to obtain a C5_1 module, and improving the SPPF module of the backbone network to obtain an SPPFS module;
[0056] Step S3, improving the C3_2 module of the neck network to obtain a C5_2 module;
[0057] Step S4, replacing the CBS module, C3_1 module, and SPPF module in the backbone network with the CBH module, C5_1 module, and SPPFS module respectively, and replacing the CBS module and C3_2 module in the neck network with the CBH module and C5_2 module respectively, thereby constructing a target behavior detection model;
[0058] Step S5: construct a data set, and train the target behavior detection model based on the data set to obtain a trained target behavior detection model.
[0059] Specifically, in step S1, the initial target behavior detection model is constructed based on the YOLOv5s model, and its network structure diagram is as follows: Figure 2As shown. The input end is used to obtain the image to be detected; the backbone network is used to extract feature information of multiple different scales from the image to obtain feature maps of multiple corresponding scales; the neck network includes a feature pyramid network (FPN) and a path aggregation network (PAN), which are used to fuse the feature maps of multiple corresponding scales, fuse the shallow feature information of the image with the deep feature information, and input them to the detection end; the detection end is used to output target detection results of multiple corresponding scales. Among them, the CBS module includes a convolutional layer (Conv2d), a batch normalization layer (BN), and a SiLu activation function connected in sequence, which are used to extract feature information of multiple different scales of the image. The C3_1 module includes a first convolution module, a first feature fusion layer (Concat), and a first CBS module connected in sequence, which are used to fuse feature information of multiple different scales; wherein, the first convolution module includes a second CBS module and a first bottleneck module connected in parallel, the first bottleneck module includes a third CBS module and a first bottleneck layer (BottleNeck1) connected in sequence, the first bottleneck layer includes a residual module and a second feature fusion layer connected in sequence, the residual module includes a jump connection and a dual CBS module connected in parallel, and the dual CBS module includes a fourth and a fifth CBS module connected in sequence. The C3_2 module includes a second convolution module, a first feature fusion layer, and a first CBS module connected in sequence, which are used to fuse feature information of multiple different scales; the second convolution module includes a second CBS module and a second bottleneck module connected in parallel, the second bottleneck module includes a third CBS module and a second bottleneck layer (BottleNeck2) connected in sequence, and the second bottleneck layer includes a fourth and a fifth CBS module connected in sequence. The SPPF module includes a first CBS module, three maximum pooling layers (MaxPool2d), a feature fusion layer, and a second CBS module connected in sequence, and is used to extract global feature information of the image; wherein the output of the first CBS module and the output of the three maximum pooling layers are also jump-connected to the feature fusion layer.
[0060] Specifically, in step S2, if Figure 3 As shown, the CBH module includes a convolutional layer, a batch normalization layer, and an NHSwish activation function connected in sequence.
[0061] Specifically, the expression of the NHSwish activation function is:
[0062]
[0063] Among them, x represents the output result of the batch normalization layer;
[0064] Its image and its derivative image are as follows Figure 4 shown.
[0065] It can be understood that the NHSwish activation function uses piecewise functions to simplify the expression, improves the computational efficiency, and is suitable for embedded devices with limited resources.
[0066] Furthermore, if Figure 3 As shown, the C5_1 module includes the first and second segmentation layers (Split), the first to fifth CBH modules, and a feature fusion layer; wherein the input feature map is divided into two paths through the first segmentation layer and then respectively input into the feature fusion layer through the first CBH module and the second CBH module, the output of the second CBH module is also divided into two paths through the second segmentation layer and then respectively input into the feature fusion layer through the third CBH module and the fourth CBH module, and the output of the feature fusion layer is input into the fifth CBH module.
[0067] Furthermore, if Figure 5 As shown, the SPPFS module includes a segmentation layer, a first and a second CBH module, a first feature fusion layer, and a maximum pooling module; wherein the input feature map is divided into two paths by the segmentation layer and then respectively input into the first feature fusion layer through the maximum pooling module and the first CBH module, and the output of the first feature fusion layer is input into the second CBH module; the maximum pooling module includes a third CBH module, three maximum pooling layers, a second feature fusion layer, and a fourth CBH module connected in sequence, and the output of the third CBH module and the output of the three maximum pooling layers are also jump-connected to the second feature fusion layer.
[0068] It can be understood that the SPPFS module has a deeper information fusion effect by increasing the information fusion between different channels, which is conducive to extracting global feature information more comprehensively.
[0069] Specifically, in step S3, if Figure 3 As shown, the C5_2 module includes a segmentation layer, first to fifth CBH modules, and a feature fusion layer; wherein the input feature map is divided into two paths through the segmentation layer and then respectively input into the feature fusion layer through the first CBH module and the second CBH module, the output of the second CBH module is also input into the feature fusion layer through the third CBH module, the output of the third CBH module is also input into the feature fusion layer through the fourth CBH module, and the output of the feature fusion layer is input into the fifth CBH module.
[0070] It can be understood that the C5_1 module and the C5_2 module are composed of CBH modules through multiple channel superposition, which not only reduces the number of parameters and calculations, but also increases the information fusion between different channels.
[0071] Specifically, in step S4, the CBH module is used to replace all CBS modules in the backbone network and the neck network of the initial target behavior detection model; the C5_1 module is used to replace all C3_1 modules in the backbone network of the initial target behavior detection model; the SPPFS module is used to replace the SPPF module in the backbone network of the initial target behavior detection model; the C5_2 module is used to replace all C3_2 modules in the neck network of the initial target behavior detection model, thereby constructing a target behavior detection model. The network structure diagram of the target behavior detection model is shown in FIG. Figure 6 shown.
[0072] Furthermore, in the target behavior detection model:
[0073] The output of the third layer C5_1 module in the backbone network, i.e., the p2 unit, is input to the first layer CBH module in the neck network FPN;
[0074] The output of the SPPFS module in the backbone network is input to the sixth feature fusion layer in the neck network PAN;
[0075] The output of the second feature fusion layer in the neck network FPN is input to the first CBH module in the neck network PAN.
[0076] It can be understood that inputting the output of the p2 unit in the backbone network to the neck network FPN and inputting the output of the second feature fusion layer in FPN to PAN can reduce the attenuation of small target feature information in the multi-layer convolution calculation process, enable feature information to be fused in FPN and PAN, enhance the propagation of shallow semantic information of the image, and balance the contradiction between the deep network for target category recognition and the shallow network for small target positioning and image feature extraction of different scales. The output of the SPPFS module in the backbone network is directly input into the neck network PAN as an information flow, which improves the accuracy of the predicted box position.
[0077] Specifically, in step S5, the collecting of target behavior images and annotating them to construct a data set, and training the target behavior detection model based on the data set to obtain a trained target behavior detection model include:
[0078] Step S501, collecting target behavior images, preprocessing the target behavior images, image enhancement and annotation to construct a data set;
[0079] Step S502: training the target behavior detection model based on the data set, performing positive sample expansion during the training process, and obtaining a trained target behavior detection model after the training is completed.
[0080] Specifically, in step S501, target behavior images are collected through monitoring cameras and the Internet; the target behavior images include specific behavior images, specific target images, and specific interaction images.
[0081] Specifically, a specific behavior image refers to a target behavior to be detected, such as making a phone call, driving a car, etc.; a specific target image refers to a target related to the target behavior to be detected, such as a person making a phone call and a phone.
[0082] Furthermore, the preprocessing, image enhancement and annotation of the target behavior image to construct a data set includes:
[0083] Eliminate target behavior images with unclear target feature points, and adjust the pixel size and format of the remaining target behavior images to be consistent;
[0084] Performing image enhancement processing on the target behavior image after pixel size adjustment;
[0085] The target behavior images after pixel size adjustment and image enhancement are annotated to construct a dataset, and the dataset is divided into a training set and a test set.
[0086] It should be noted that among the collected target behavior images, the target feature points of some images may not be obvious, for example, the image itself is blurry, the target occupies a small area in the image, the target itself is not clear enough, etc., and such images are eliminated by manual screening. Since the pixel size and format of the target behavior images collected by the surveillance camera may be different from those collected through the Internet, their pixel size and format are adjusted to be consistent; the minimum pixel size of the remaining images is obtained, and the pixel sizes of all remaining images are adjusted to the minimum pixel size, and the format of all remaining images is adjusted to jpeg.
[0087] Exemplarily, 8400 target behavior images with good lighting conditions were collected through two surveillance cameras, with an image pixel size of 1920×1080 and an image format of jpeg; 4524 target behavior images were collected through the Internet, with an image pixel size of 1280×720 and an image format of bmp; 100 images in which the target feature points were not obvious among the above 12924 images were eliminated; the minimum pixel size of the remaining 12824 images was 1280×720, and the pixel sizes of the above 12824 images were adjusted to 1280×720 and the formats were adjusted to jpeg through image processing software.
[0088] Furthermore, the image enhancement processing of the target behavior image after pixel size adjustment includes one or more of the following methods:
[0089] Randomly selecting part of the target behavior image and copying it to other images to generate a new target behavior image;
[0090] Performing a two-dimensional linear transformation on the target behavior image to generate a new target behavior image; the two-dimensional linear transformation includes rotation, translation, scaling, shearing, and flipping;
[0091] Randomly selecting two images from the target behavior image, mixing the two images in a certain ratio, and generating a new target behavior image;
[0092] Randomly adjusting the hue, saturation, and brightness of the target behavior image to generate a new target behavior image;
[0093] Randomly cover a part of the target behavior image to generate a new target behavior image.
[0094] It should be noted that the shearing includes horizontal shearing and vertical shearing. Horizontal shearing achieves a distortion effect by adding a horizontal displacement on each row of the image, and vertical shearing achieves a distortion effect by adding a vertical displacement on each column of the image.
[0095] It can be understood that through image enhancement, not only the number of samples is increased, but also the background of the image is enriched, so that the target behavior detection model can learn more diverse knowledge during training, providing a basis for improving the robustness and generalization ability of the model.
[0096] Furthermore, a dataset is constructed by annotating each target to be detected in the target behavior image after pixel size adjustment and image enhancement with a rectangular box through the LabelImg annotation tool; the target to be detected includes the target behavior and its related targets; the annotation content includes the category number, the coordinates of the center point of the target to be detected, and the height and width of the annotation box.
[0097] Specifically, the data set includes target behavior images after pixel size adjustment and image enhancement and their corresponding annotation content; the data set is divided into a training set and a test set in a ratio of 8:2.
[0098] It should be noted that the annotation content of each image is generated as a file in txt format with the same name as the image. Therefore, the data set includes a set of target behavior images and a set of corresponding annotation content files.
[0099] Specifically, in step S502, the target behavior detection model is trained based on the data set, positive samples are expanded during the training process, and the trained target behavior detection model is obtained after the training is completed, including:
[0100] Input each sample in the training set of the data set into the target behavior detection model for training, and output the corresponding reasoning graph; the reasoning graph includes prediction boxes, each prediction box includes the coordinates of the center point corresponding to the prediction box, the height and width of the prediction box, the detection target category number, and the confidence score;
[0101] If the prediction box in the inference graph is a positive sample, the corresponding prediction box is added to expand the positive sample;
[0102] The loss function is calculated by the error between the labeled box of the sample and the predicted box of the corresponding reasoning graph. When the loss function reaches the threshold, the trained target behavior detection model is obtained.
[0103] Furthermore, if the prediction box in the reasoning graph is a positive sample, adding a corresponding prediction box to expand the positive sample includes: if the prediction box in the reasoning graph is a positive sample, establishing four adjacent prediction boxes in the four directions of the prediction box above, below, left and right according to the size of the prediction box, selecting two prediction boxes whose center points are closest to the center point of the annotation box corresponding to the prediction box from the four adjacent prediction boxes as newly added prediction boxes; setting the category of the newly added prediction box to be consistent with the prediction box.
[0104] Specifically, a positive sample refers to a sample in which the error between the predicted box in the inference graph and the actual labeled box of the sample is less than the threshold, indicating that the area intersection ratio of the predicted box and the labeled box is greater than the threshold and the category of the predicted box is consistent with the category of the labeled box, otherwise it is a negative sample.
[0105] It can be understood that by increasing the number of matching prediction boxes, a real annotation box can be predicted by multiple prediction boxes, thereby increasing the number of positive samples and reducing the impact caused by the imbalance of positive and negative samples.
[0106] Furthermore, the loss function includes detection target confidence loss, detection target category loss, and prediction box loss; the expression of the loss function is as follows:
[0107] Loss = λ obj BCE obj +λ cls BCE cls +λ box FIoU,
[0108] Among them, BCE obj To detect the target confidence loss, BCE cls is the target category loss, FIoU is the prediction box loss, and λ obj , cls , box is the weight coefficient.
[0109] Preferably, λ objSet to 1.0, λ cls Set to 0.5, λ box Set to 0.05.
[0110] Specifically, the detection target confidence loss represents the error in prediction accuracy, that is, whether there is a target in the prediction box. The loss function expression of the detection target confidence loss is as follows:
[0111]
[0112]
[0113] Among them, N is the total number of samples, m is the number of positive samples, Nm is the number of negative samples, and y i =1 means there is a target in the prediction box, y i =0 means that there is no target in the prediction box, p1 is the probability that there is a target in the prediction box, and α is the coefficient.
[0114] It can be understood that negative samples correspond to the image background, and positive samples correspond to the actual target area in the image. Usually, there are more negative samples and fewer positive samples, which easily leads to the loss value of negative samples overwhelming the loss value of positive samples, thus reducing the convergence speed of the model and the accuracy of target detection. obj The loss function can dynamically adjust the loss values of positive and negative samples, thereby improving the speed of model convergence and the accuracy of target detection.
[0115] Specifically, the detection target category loss represents the error between the category of the prediction box and the category of the annotation box. The loss function expression of the detection target category loss is as follows:
[0116]
[0117] Among them, N p is the number of positive samples, n is the number of categories, C i =1 means the predicted category belongs to C i Category, C i =0 means the predicted category does not belong to C i Category, β i C is the positive sample i The proportion of the category, γ is the control coefficient, p i Indicates that the prediction is C i The probability of the class.
[0118] Understandably, through β i The loss values of different categories can be adjusted dynamically to balance the imbalance in the number of samples between different categories.
[0119] Specifically, the prediction box loss represents the position error between the prediction box and the sample annotation box. The loss function expression of the prediction box loss is as follows:
[0120]
[0121] Among them, B pre Indicates the area occupied by the prediction box, B gt represents the area occupied by the annotation box, IoU is the intersection-over-union ratio of the predicted box and the annotation box, ρ 2 (B pre , B gt ) represents the distance between the center point of the prediction box and the center point of the annotation box, w pre is the width of the prediction box, w gt is the width of the annotation box, ρ 2 (w pre , w gt ) represents the square of the difference between the width of the prediction box and the width of the annotation box, h pre is the height of the prediction box, h gt is the height of the annotation box, ρ 2 (h pre ,h gt ) represents the square of the height difference between the predicted box and the labeled box, c w 2 Represents the square of the minimum enclosing rectangle width of the prediction box and the annotation box, c h 2 Represents the square of the minimum enclosing rectangle height of the prediction box and the annotation box, c 2 Represents the square of the diagonal length of the minimum circumscribed rectangle of the prediction box and the annotation box. Represents the empty set.
[0122] It can be understood that if the IoU value between the prediction box and the annotation box is small or even 0, the prediction box loss value will fluctuate violently, which is not conducive to the convergence of the model. The FIoU loss function proposed in the present invention improves the accuracy of the aspect ratio by respectively calculating the length ratio and width ratio of the prediction box and the annotation box instead of considering the consistency of the aspect ratio. It can balance the loss value when the prediction box and the annotation box have no intersection, less intersection, and high overlap, avoids the prediction box loss value being too large or too small, and accelerates the convergence of the model.
[0123] Specifically, the number of training rounds is set to 200. In each round of training, the batch size is set to 8, the optimizer is Adam, the initial learning rate is 0.01, the weight decay rate is 0.001, the momentum factor is 0.937, the weight decay factor is 0.0005, the detection target confidence loss threshold is 0.5, the detection target category loss threshold is 0.75, and the prediction box loss threshold is 0.5.
[0124] Performance evaluation:
[0125] Specifically, the target behavior detection model proposed in the present invention and the performance of YOLOv5s are evaluated from three aspects: mean average precision (mAP), precision (P), and recall (R) through ablation experiments. Precision P is used to evaluate the accuracy of target behavior detection, and recall R is used to evaluate the comprehensiveness of target behavior detection. mAP refers to the average area under the PR curve of all categories, which comprehensively considers the precision and recall of the model.
[0126] Specifically, the calculation formulas for average accuracy, precision, and recall are as follows:
[0127]
[0128]
[0129]
[0130]
[0131] Among them, TP represents the number of correct target behavior detection results, FP represents the number of incorrect target behavior detection results, FN represents the number of missed detections where the target behavior was not detected, AP represents the area under the PR curve of the category, and n c is the total number of categories.
[0132] Specifically, the YOLOv5s model parameters trained based on the COCO dataset are used as the pre-training model; as shown in Table 1, ablation experiments are conducted on the five models using the above datasets, each model is trained for 200 rounds, and then the results of the above indicators are obtained:
[0133] Table 1. Ablation comparison experimental results
[0134]
[0135] It can be seen from the above table that compared with YOLOv5s, the target behavior detection model proposed in the present invention has obvious improvements in precision, recall and average precision, which effectively solves the problems of low accuracy and low real-time performance of existing target behavior detection.
[0136] Compared with the prior art, the method for constructing a target behavior detection model provided by the present invention has the following beneficial effects:
[0137] 1. The NHSwish activation function proposed in the present invention uses a piecewise function to simplify the expression, which improves the calculation efficiency and is suitable for embedded devices with limited resources.
[0138] 2. The C5_1 module and C5_2 module proposed in the present invention are composed of CBH modules through multiple channel superposition, which not only reduces the number of parameters and calculations, but also increases the information fusion between different channels, providing a basis for improving the accuracy of model prediction.
[0139] 3. The SPPFS module proposed in the present invention has a deeper information fusion effect by increasing the information fusion between different channels, which is conducive to more comprehensive extraction of global feature information and provides a basis for improving the accuracy of model prediction.
[0140] 4. The present invention inputs the output of the p2 unit in the backbone network to the neck network FPN and inputs the output of the second feature fusion layer in the FPN to the PAN, which can reduce the attenuation of the feature information of small targets in the multi-layer convolution calculation process, and enables the feature information to be fused in the FPN and PAN, thereby enhancing the propagation of shallow semantic information of the image, balancing the contradiction between the deep network for target category recognition and the shallow network for small target positioning and extracting image features of different scales; the output of the SPPFS module in the backbone network is directly input into the neck network PAN as an information flow, which improves the accuracy of the prediction box position and provides a basis for improving the accuracy of model prediction.
[0141] Those skilled in the art will appreciate that all or part of the processes of the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium, wherein the computer-readable storage medium is a disk, an optical disk, a read-only storage memory, or a random access memory, etc.
[0142] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by any technician familiar with the technical field within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention.
Claims
1. A method for constructing a target behavior detection model, characterized in that: The method comprises the following steps: Building an initial target behavior detection model based on the YOLOv5s model, wherein the initial target behavior detection model includes an input end, a backbone network, a neck network, and a detection end; The CBS module of the backbone network is improved to obtain a CBH module, the C3_1 module of the backbone network is improved to obtain a C5_1 module, and the SPPF module of the backbone network is improved to obtain a SPPFS module; The C3_2 module of the neck network is improved to obtain the C5_2 module; The CBS module, C3_1 module, and SPPF module in the backbone network are replaced by the CBH module, C5_1 module, and SPPFS module, respectively, and the CBS module and C3_2 module in the neck network are replaced by the CBH module and C5_2 module, respectively, so as to construct a target behavior detection model; Target behavior images are collected and annotated to construct a data set, and the target behavior detection model is trained based on the data set to obtain a trained target behavior detection model.
2. The method for constructing a target behavior detection model according to claim 1, characterized in that: The CBH module includes a convolutional layer, a batch normalization layer, and an NHSwish activation function connected in sequence.
3. The method for constructing a target behavior detection model according to claim 2, characterized in that: The expression of the NHSwish activation function is: Here, x represents the output of the batch normalization layer.
4. The method for constructing a target behavior detection model according to claim 1, characterized in that: The C5_1 module includes the first and second segmentation layers, the first to fifth CBH modules, and a feature fusion layer; wherein the input feature map is divided into two paths through the first segmentation layer and then respectively input into the feature fusion layer through the first CBH module and the second CBH module, the output of the second CBH module is also divided into two paths through the second segmentation layer and then respectively input into the feature fusion layer through the third CBH module and the fourth CBH module, and the output of the feature fusion layer is input into the fifth CBH module.
5. The method for constructing a target behavior detection model according to claim 1, characterized in that: The C5_2 module includes a segmentation layer, first to fifth CBH modules, and a feature fusion layer; wherein the input feature map is divided into two paths through the segmentation layer and then respectively input into the feature fusion layer through the first CBH module and the second CBH module, the output of the second CBH module is also input into the feature fusion layer through the third CBH module, the output of the third CBH module is also input into the feature fusion layer through the fourth CBH module, and the output of the feature fusion layer is input into the fifth CBH module.
6. The method for constructing a target behavior detection model according to claim 1, characterized in that: The SPPFS module includes a segmentation layer, a first and a second CBH module, a first feature fusion layer, and a maximum pooling module; wherein the input feature map is divided into two paths through the segmentation layer and then respectively input into the first feature fusion layer through the maximum pooling module and the first CBH module, and the output of the first feature fusion layer is input into the second CBH module; the maximum pooling module includes a third CBH module, three maximum pooling layers, a second feature fusion layer, and a fourth CBH module connected in sequence, and the output of the third CBH module and the output of the three maximum pooling layers are also jump-connected to the second feature fusion layer.
7. The method for constructing a target behavior detection model according to claim 1, characterized in that: Replace all CBS modules in the backbone network and neck network of the initial target behavior detection model with CBH modules; replace all C3_1 modules in the backbone network of the initial target behavior detection model with C5_1 modules; replace the SPPF module in the backbone network of the initial target behavior detection model with SPPFS modules; replace all C3_2 modules in the neck network of the initial target behavior detection model with C5_2 modules, thereby constructing a target behavior detection model.
8. The method for constructing a target behavior detection model according to claim 1, characterized in that: In the target behavior detection model: The output of the third layer C5_1 module in the backbone network, i.e., the p2 unit, is input to the first layer CBH module in the neck network FPN; The output of the SPPFS module in the backbone network is input to the sixth feature fusion layer in the neck network PAN; The output of the second feature fusion layer in the neck network FPN is input to the first CBH module in the neck network PAN.
9. The method for constructing a target behavior detection model according to claim 1, characterized in that: Preprocessing, image enhancement, and annotation of target behavior images to build a dataset include: Eliminate target behavior images with unclear target feature points, and adjust the pixel size and format of the remaining target behavior images to be consistent; Performing image enhancement processing on the target behavior image after pixel size adjustment; The target behavior images after pixel size adjustment and image enhancement are annotated to construct a dataset, and the dataset is divided into a training set and a test set.
10. The method for constructing a target behavior detection model according to claim 9, characterized in that: The image enhancement processing of the target behavior image after pixel size adjustment includes one or more of the following methods: Randomly selecting part of the target behavior image and copying it to other images to generate a new target behavior image; Performing a two-dimensional linear transformation on the target behavior image to generate a new target behavior image; the two-dimensional linear transformation includes rotation, translation, scaling, shearing, and flipping; Randomly selecting two images from the target behavior image, mixing the two images in a certain ratio, and generating a new target behavior image; Randomly adjusting the hue, saturation, and brightness of the target behavior image to generate a new target behavior image; Randomly cover a part of the target behavior image to generate a new target behavior image.