A small target detection method of a one-way series YOLOV5 network

Abstract

The application discloses a one-way series YOLOV5 network small target detection method. The method adopts two series YOLOV5 as a detector to obtain the target detection result in a video scene, and detects the smoking behavior of a video stream through two layers of YOLOV5 network. The first YOLOV5 network is responsible for preliminary screening, and detects the input video frame in real time to screen out the class smoking behavior target. The second YOLOV5 network is responsible for fine screening, and detects the input class smoking behavior target again to accurately detect the smoking behavior target existing in the video frame. The application improves the network detection accuracy and detection speed while reducing the volume and parameter quantity of the network, realizes higher target detection accuracy and accuracy through the series YOLOV5 network, improves the false detection and missed detection problems caused by the insufficient target feature extraction capability in the existing network, and improves the detection effect.

Description

Technical Field

[0001] This invention belongs to the field of video detection technology, specifically relating to a method for small target detection using a unidirectional serial YOLOv5 network. Background Technology

[0002] Smoking not only harms individual health, but improper handling of incompletely extinguished cigarette butts can also cause major fire accidents. In public areas and forest scenic areas, smoking can trigger fire suppression systems or cause fires, resulting in significant personal injury and economic losses. Therefore, personnel patrol public areas and forest scenic areas to remind people not to smoke. However, traditional manual patrols are insufficient for large-scale monitoring and real-time detection, leading to serious omissions.

[0003] In recent years, with the continuous development of computer vision technology and the upgrading of supporting hardware, object detection has been widely used in industrial production. Early methods for detecting smoking behavior used smoke sensors, HOG (Homologous Oriented Genesis), background difference, pose recognition, and facial energy value changes to detect smoking behavior. With the development of convolutional neural networks, object detection has gradually moved towards deep learning methods based on deep neural networks. Currently, commonly used object detection algorithms mainly include SSD, YOLO, RCNN, FAST R-CNN, and MASK R-CNN. To further improve the accuracy and precision of smoking behavior detection and mitigate the problem of missed and false detections caused by insufficient feature network extraction capabilities, it is necessary to design a small object detection method using a unidirectional cascaded YOLOv5 network. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of deep learning object detection algorithms in detecting small objects, such as false detection and missed detection. It provides a small object detection method using a unidirectional serial YOLOv5 network, which can avoid the problems of false detection and missed detection of small objects, while improving the detection accuracy of small objects.

[0005] This invention includes the following steps:

[0006] Step A: Input the video stream of the small target to be detected, and extract the video frames frame by frame.

[0007] Step B: Obtain the predicted location det and confidence C1 of the small target class from the first YOLOv5 network.

[0008] Step C: Extend the predicted position det of the small target class to obtain the target class position det1, and extract and generate a preliminary detection image from the video frame based on the target class position det1.

[0009] Step D: Input the preliminary detection image into the second YOLOv5 network to obtain the predicted location det3 and confidence level C2 of the small target.

[0010] Step E: Obtain the target prediction position information det4 based on the predicted position det3 of the small target, and mark the target prediction position information det4 on the video frame image generated in step A.

[0011] Step F: Obtain new target confidence C information using confidence C1 and confidence C2, and mark the target confidence C information on the video frame image generated in step A.

[0012] Furthermore, step B specifically involves:

[0013] Step B-1: Input the video frame into the unidirectional cascaded YOLOv5 network, and extract features through the backbone of the first YOLOv5 network to obtain feature maps at different scales.

[0014] Step B-2: Input the acquired feature maps at different scales into the feature fusion module of the first YOLOv5 network to fuse strong semantic information and strong localization information at different scales.

[0015] Furthermore, step D specifically involves:

[0016] Step D-1: Input the preliminary detection image into the backbone network of the second YOLOV5 network of the unidirectional cascaded YOLOV5 network for feature extraction and obtain feature maps at different scales;

[0017] Step D-2: Input the acquired feature maps at different scales into the feature fusion module of the second YOLOv5 network to fuse strong semantic information and strong localization information at different scales.

[0018] Furthermore, the backbone of the first YOLOv5 network incorporates an attention mechanism to capture cross-channel information, assisting the model in improving detection accuracy and precision; the detection head of the neck network forms a 4-scale output.

[0019] Furthermore, before the acquired feature maps of different scales are fed into the feature fusion module, the feature maps of different scales are input into the Feature Pyramid Network (FPN) to transmit strong semantic information, and the output of the Feature Pyramid Network (FPN) is sent to the Path Aggregation Network (PAN) to transmit the underlying strong localization information.

[0020] Furthermore, the extension described in step C is used to adjust the width and height of the predicted rectangle.

[0021] Furthermore, the backbone of the second YOLOv5 network incorporates an attention mechanism, which is used to capture cross-channel and cross-spatial dimensional information to help the model improve detection accuracy and precision; the detection head of the neck network forms a 2-scale output.

[0022] Technical effects of the present invention:

[0023] Compared to single-stage YOLOv5 network structures, this invention uses a cascaded structure of two YOLOv5 networks. These networks perform target detection on the input video, and the backbone network extracts features from the smoking behavior video by fusing an attention mechanism between the two networks and adjusting the number of detector heads. In this process, the attention mechanism embeds positional information into channel attention, capturing cross-channel information, orientation-aware information, and position-sensitive information, while eliminating interference from other information. The first YOLOv5 network adds a small target detection layer to improve the model's recall, while the second YOLOv5 network reduces the number of target detection layers to decrease network complexity and improve accuracy. Feature pyramids and path aggregation structures are employed to enhance the network's ability to detect targets at different scales. In the two stages of network model prediction, the confidence scores C1 and C2 of the detected smoking behavior are weighted according to the network structure characteristics, and nonlinear calculations are performed to obtain the final target confidence score. This improves the accuracy of small target detection in smoking behavior and reduces false negatives and missed detections caused by insufficient feature extraction capabilities in existing technologies.

[0024] This invention, for the detection of smoking behavior targets, adds novel tobacco targets, such as e-cigarettes, chewing gum cigarettes, and snuff, to the traditional cigarette detection method. Addressing the diversity of targets and the complexity of backgrounds in detection tasks, this invention incorporates more target categories and smoking behavior targets under different complex backgrounds into the network learning process, improving detection accuracy. Considering the small scale of smoking behavior targets and the presence of a large amount of complex interference information in the background, this invention employs a unidirectional serial YOLOv5 network for multi-scale detection. Through initial screening by the first YOLOv5 network and fine screening by the second, the accuracy of determining the category of smoking behavior targets is improved. Simultaneously, the innovative use of weighted processing technology for confidence information can also suppress interference information from abnormal categories in the background during the detection process of smoking behavior targets. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a flowchart of a small target detection method using a unidirectional serial YOLOv5 network according to the present invention;

[0027] Figure 2 This is a flowchart illustrating a small target detection method using a unidirectional serial YOLOv5 network according to the present invention.

[0028] Figure 3 This is a structural diagram of the attention mechanism of the preceding YOLOv5 network in the small target detection method of the unidirectional serial YOLOv5 network of the present invention;

[0029] Figure 4 This is a structural diagram of the preceding YOLOv5 network in the small target detection method of the unidirectional serial YOLOv5 network of the present invention;

[0030] Figure 5 This is a structural diagram of the attention mechanism of the second YOLOv5 network in the small target detection method of the unidirectional serial YOLOv5 network of the present invention.

[0031] Figure 6 This is a structural diagram of the second YOLOv5 network in the small target detection method of the unidirectional serial YOLOv5 network of the present invention;

[0032] Figure 7 This is a detector structure diagram of a small target detection method using a unidirectional serial YOLOv5 network according to the present invention. Detailed Implementation

[0033] The technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0034] Example:

[0035] Please see Figure 1 and Figure 7 This embodiment discloses a small target detection method using a unidirectional serial YOLOv5 network to address the issues of missed and false detections of smoking behavior in public settings. The method specifically includes the following steps:

[0036] Step 1: Input a video stream containing the target of smoking behavior and extract it frame by frame to generate video frames.

[0037] Step 2: Input the video frames generated in Step 1 into a unidirectional cascaded YOLOv5 network. After passing through the backbone network of the previous YOLOv5 network, feature extraction is performed to obtain feature maps at different scales.

[0038] Step 3: Input the feature maps of different scales obtained in Step 2 into the feature fusion module of the previous YOLOv5 network to fuse strong semantic information and strong localization information of different scales.

[0039] Step 4: The previous YOLOv5 network obtains the predicted location (det) and confidence level (C1) of the smoking behavior target.

[0040] Step 5: Using the predicted location information of the smoking-like behavior target obtained in Step 4, perform nonlinear function calculation on the predicted location information to obtain the extended target location information det1.

[0041] Step 6: Based on the target location information det1 obtained in Step 5, extract the target from the corresponding video frame in Step 1 and perform adaptive scaling to generate an image of size w2×h2.

[0042] Step 7: Input the image generated in step 6 into the backbone network of the next YOLOv5 network in the unidirectional cascaded YOLOv5 network for feature extraction to obtain feature maps at different scales.

[0043] Step 8: Input the feature maps of different scales obtained in Step 7 into the feature fusion module of the next YOLOv5 network to fuse strong semantic information and strong localization information of different scales.

[0044] Step 9: The next YOLOv5 network obtains the predicted location det3 and confidence C2 information of the smoking behavior target.

[0045] Step 10: The predicted location information obtained in Step 9 is calculated using a nonlinear function and then used as the target predicted location information det4 of the unidirectional serial YOLOV5 network, which is then marked on the video frame image generated in Step 1.

[0046] Step 11: Perform nonlinear function calculations on the confidence information obtained in Step 4 and Step 9 to generate new target confidence C information.

[0047] Step 12: Mark the confidence information generated in Step 11 as the target confidence information of the unidirectional serial YOLOv5 network onto the video frame image generated in Step 1.

[0048] The unidirectional cascaded YOLOv5 network model of this invention addresses the complexity of smoking behavior detection backgrounds and the unique characteristics of the two YOLOv5 models included in the model. During training, both YOLOv5 models are first trained using a pure cigarette dataset without complex backgrounds, resulting in parameter files. Then, the parameter file of the first YOLOv5 network model is individually fine-tuned using a smoking behavior dataset containing complex backgrounds. Since the model performs initial screening and cropping of the initial video frames before further screening by inputting them into the second YOLOv5 network model, the generated images input into the second YOLOv5 network model do not contain complex backgrounds, thus eliminating the need for separate fine-tuning. Finally, the parameter files of the two YOLOv5 network models are assigned to the overall unidirectional cascaded YOLOv5 network model for large-scale dataset training and parameter adjustment.

[0049] In one embodiment, the video stream in step 1 is real-time streaming of on-site video surveillance data. The unidirectional serial YOLOv5 network is responsible for processing the collected video surveillance data and outputting the corresponding target detection results. By utilizing the unique two-stage target detection of the unidirectional serial YOLOv5 network, accurate detection of smoking behavior targets is achieved from initial screening to fine screening.

[0050] Furthermore, the unidirectional serial YOLOV5 network described in step 2 consists of two serial YOLOV5 networks, each containing an improved backbone network and a neck network, and the two YOLOV5 network structures differ from each other.

[0051] In step 2, the attention mechanism is integrated into the backbone of the previous YOLOv5 network, and the number of detection heads is modified in the neck network.

[0052] During the feature extraction process of inputting the video frames generated in step 1 into the backbone network of the previous YOLOv5 network, an attention mechanism is used to capture cross-channel information, assisting the model in improving detection accuracy and precision. The modification of the number of detection heads involves adding a small target detection layer to the neck network of the previous YOLOv5 network, resulting in a 4-scale output. This 4-scale output fuses shallower, strong localization information with deeper, strong semantic information, enhancing the network's multi-scale detection capabilities.

[0053] In one embodiment, the feature maps of different scales output from steps 2 and 7 are input into the Feature Pyramid Network (FPN) to transmit strong semantic information. The output of the FPN is sent to the Path Aggregation Network (PAN) to transmit strong localization information at the bottom layer. The two networks fuse the multi-scale feature information to generate a multi-scale feature representation.

[0054] In one embodiment, during the process of obtaining the predicted location and confidence information of the smoking behavior target by the previous YOLOv5 network in step 4, interference targets are removed by non-maximum suppression to obtain and retain the predicted location det and confidence C1 information of the smoking behavior target; wherein, the target coordinate information det = (x, y, w, h) obtained by the previous YOLOv5 network detection algorithm, where x, y, w, and h represent the coordinates of the target center point in the image and the width and height of the target's predicted bounding box, respectively.

[0055] Furthermore, the target coordinate information det obtained in step 4 is used to perform nonlinear function calculation in step 5 to obtain det1, that is, while keeping the center point coordinates x and y variables of the target unchanged, the w and h variables are scaled to obtain the variables w1 and h1.

[0056] Furthermore, the target coordinate information det1=(x,y,w1,h1) extended from step 5 is cropped in the video frame generated in step 1, and the cropped image is adaptively enlarged to generate an image with a size of w2×h2.

[0057] In step 7, the attention mechanism is integrated into the backbone of the second YOLOv5 network, and the neck network modifies the number of detection heads.

[0058] During the feature extraction process of the image generated in step 6, which is then input into the backbone network of the subsequent YOLOv5, the attention mechanism is used to capture cross-channel and cross-spatial dimensional information, assisting the model in improving detection accuracy and precision. Modifying the number of detection heads involves reducing the number of detection heads in the neck network of the subsequent YOLOv5 network to form a two-scale output. This two-scale output retains the original 80*80 large-scale target detection layer and 40*40 medium-scale target detection layer, while deleting the original 20*20 small-scale target detection layer, thus reducing the model size and number of parameters while improving the model's detection performance.

[0059] In one embodiment, during the process of obtaining the predicted location and confidence of the smoking behavior target by the second YOLOv5 network in step 9, non-maximum suppression is used to remove interfering targets, thereby obtaining the predicted location det3 and confidence C2 information of the smoking behavior target; wherein, the coordinate information of the smoking behavior target obtained by the second YOLOv5 network detection algorithm is det3=(x3,y3,w3,h3), where x3, y3, w3, and h3 represent the coordinates of the target center point in the image and the width and height of the predicted rectangle of the target, respectively.

[0060] Furthermore, in step 10, the predicted target position information output by the next YOLOV5 network is calculated using a nonlinear function to obtain its position relative to the input video frame of the unidirectional serial YOLOV5 network, and the position information det4 = (x4, y4, w4, h4) is marked on its video frame.

[0061] In one embodiment, the unidirectional serial YOLOv5 network includes two YOLOv5 networks, which output the target prediction confidence in steps 4 and 9 respectively. Based on the network structure characteristics, corresponding weights are assigned, and the final target confidence C information is obtained after nonlinear function calculation.

[0062] The small target detection method using a unidirectional serial YOLOv5 network consists of two specific steps: initial screening and fine screening. This two-stage screening strategy reduces the number of missed and false detections, stabilizes the target detection box accuracy, and improves the accuracy of detecting smoking behavior in small targets. For detailed procedures, please refer to [link / reference]. Figure 2 .

[0063] The inclusion of an attention mechanism in this application allows the network to focus on areas that require more attention, rather than simply applying a global average of attention to the image. The attention mechanism is one way to achieve adaptive attention in a network. Generally, attention mechanisms are mainly divided into channel attention mechanisms and spatial attention mechanisms, as well as combinations of the two. Through extensive experimental analysis, two attention mechanisms are employed for a unidirectional cascaded YOLOv5 network. The SE channel attention mechanism is used for the first YOLOv5 network, focusing on improving the recall rate of the predicted target and inputting a large number of potential targets into the second YOLOv5 network for fine-tuning. The second network uses the CA attention mechanism to accurately predict the target from a large number of generated images, focusing on improving the network's precision and accuracy.

[0064] SE attention mechanism was one of the earliest to introduce attention mechanisms into convolutional deep learning network models, providing a plug-and-play modular approach for convolutional neural networks. SE attention is a typical channel attention mechanism, aiming to assign different weights to different locations in the image from the perspective of the channel domain using a weight matrix, thereby obtaining more important feature information. The main operations are squeezing and activation. The principle of SE attention is to reconstruct feature maps without changing the network structure.

[0065] like Figure 3 After feature extraction from a single image, the generated feature map U has dimensions C×H×W. Performing average pooling or max pooling on the (H,W) dimension of the feature map is essentially a dimensionality reduction process, resulting in a feature map with dimensions C×1×1 after pooling. Each channel C has a corresponding weight value. If global average pooling is used, its generation formula is shown in (1).

[0066]

[0067] Where z c It is the output associated with channel c.

[0068] The vector z after global average pooling is then passed through two fully connected layers. The fully connected layers play a role in adaptive reconstruction. The first fully connected layer has fewer neurons and its activation function is ReLU. The second fully connected layer has the same number of neurons as the number of channels C and its activation function is Sigmoid. After passing through the first fully connected layer, the dimension of vector z becomes C / r×1×1, where r is the decay coefficient used to adjust the parameters of the attention mechanism. The calculation formula for the vector passing through two fully connected layers is shown in equation (2) below:

[0069] s = F ex (z,W)=σ(g(z,W))=σ(W2δ(W1z)) (2)

[0070] W1 and W2 represent fully connected layers.

[0071] The vector z generated in the previous step is processed to obtain the required channel weight values ​​s. Different values ​​in s represent the weight information of different channels, and different weights are assigned to the channels.

[0072] After passing through two fully connected layers, a C×1×1 vector s containing channel weights is obtained. Vector s is then multiplied by the feature map U via a channel multiplication operation to obtain a feature map with readjusted weights. Its generation formula is shown in (3).

[0073]

[0074] in The final feature map has dimensions C×H×W.

[0075] This explains the principle of the SE attention mechanism: it reconstructs feature maps without changing the network structure. This can be understood as follows: when the weight is large, the value of the feature map in that channel increases accordingly, and its impact on the final output also increases; conversely, when the weight is small, the value of the feature map in that channel is smaller, and its impact on the final output also decreases.

[0076] The CA attention mechanism is an improved algorithm proposed to address the issue that the SE attention mechanism only focuses on the inter-channel dependencies and ignores spatial features. This attention mechanism embeds positional information into channel attention, increasing the network's attentionable area, effectively mitigating the positional information loss problem caused by two-dimensional global pooling, and is also flexible and lightweight.

[0077] The CA attention mechanism decomposes channel attention into two 1D feature encoding processes, aggregating features along the width and height spatial directions respectively. This captures long-range dependencies in one spatial direction while preserving precise positional relationships along the other. The generated feature maps are then encoded into a pair of direction-aware and position-sensitive features, which can be complementaryly applied to the input feature map to enhance the representation of the object of interest. For the input feature map X with dimensions C×H×W, pooling is performed using pooling kernels of size (H,1) and (1,W) along the width and height directions, respectively, generating feature maps of size C×H×1 and C×1×W. The output of the c-th channel with height h and the output of the c-th channel with width w can be represented by Equation (4).

[0078]

[0079] The two transformations described above aggregate features along two directions respectively, resulting in a pair of direction-aware feature maps. and These two transformations also allow the attention module to capture long-term dependencies along one spatial direction and preserve precise location information along another spatial direction, which helps the network to more accurately locate the target of interest. For coordinate attention generation, the above transformations are concatted and then transformed using a 1×1 convolution transformation function F1, as shown in formula (5).

[0080] f=δ(F1([z h ,z w ])) (5)

[0081] In the formula, [×,×] represents the concat operation along the spatial dimension, δ is the nonlinear activation function, and f is the intermediate feature map that encodes spatial information in the width and height directions. Here, r represents the reduction rate, which decomposes f into two separate tensors f along the spatial dimension. h ∈R C / r×H and f w ∈R C / r×W Using two additional 1×1 convolution transformations F h and F w f respectively h and f w The transformation is to a tensor with the same number of channels, and its generation structure formula is shown in (6).

[0082]

[0083] Among them, the σ-type sigmoid activation function.

[0084] Then output g h and g wThe feature map Y output by the CA attention mechanism is calculated as shown in Equation (7). The feature map Y is then extended and used as attention weights.

[0085]

[0086] Thus, the SE attention mechanism and CA attention mechanism are embedded in the detection network, which improves the detection performance of the model to a certain extent.

[0087] In this application, the previous YOLOv5 network adds a small target detection layer and the subsequent YOLOv5 network reduces the detection scale.

[0088] The original YOLOv5 network employs multi-scale prediction, using three anchor box sizes to predict large, medium, and small targets in the image. Each size includes three aspect ratios of anchor boxes, resulting in nine anchor box categories for target prediction. The feature fusion module, utilizing the neck network FPN and the PAN structure, enhances semantic representation at different scales. Deeper feature maps carry stronger semantic features, while shallower feature maps carry stronger localization information. In the feature fusion network, FPN transfers deep semantic features to shallower layers, thus enhancing semantic representation across multiple scales; while PAN transfers localization information from shallower layers to deeper layers, enhancing localization capabilities across multiple scales. If the input image size is 640×640, the original YOLOv5 network uses three feature layers: P3 / 8, P4 / 16, and P5 / 12, with corresponding feature map sizes of 80×80, 40×40, and 20×20, respectively, for detecting targets larger than 8×8, 16×16, and 32×32. However, the minimum receptive field for prediction at the P3, P4, and P5 scales is only 8.

[0089] Therefore, considering the multi-scale features of the predicted target in the previous YOLOv5 network structure and the input video frame, such as... Figure 4 The new P2 detection scale fuses the localization information of shallower feature maps with the semantic information of deeper feature maps. This reduces the minimum receptive field while sacrificing some overhead, thereby improving detection accuracy and recall and providing more accurate potential targets for input into the next YOLOv5 network.

[0090] Meanwhile, for the latter YOLOv5 network architecture, such as Figure 5 Because the video frames have undergone initial screening by the previous YOLOv5 network, which predicts potential targets, and adaptive image scaling to enlarge the image size to 640, the images generated by the subsequent YOLOv5 network exhibit characteristics such as a single target category, minimal target scale variation, and relatively concentrated target sizes. Therefore, multi-scale detection cannot leverage its advantages and may even exacerbate the imbalance between positive and negative samples. Thus, as... Figure 6 Reduce the P3 detection layer and only retain the P4 and P5 detection layers.

[0091] Finally, the previous YOLOV5 network structure is as Figure 6 shown. The added attention mechanism and the newly added P2 small target detection layer have been marked in the structure diagram; the latter YOLOV5 network structure is as Figure 6 shown. The added attention mechanism and the retained P4 and P5 detection layers have been marked in the structure diagram.

[0092] In this application, the non - linear function transformation of the target coordinate information of the smoking - like behavior

[0093] After the video frame passes through the previous YOLOV5 network structure, the predicted position det information of the smoking - like behavior target is obtained in step 4, where the predicted position information det=(x,y,w,h), and the x and y parameters represent the coordinates of the target center point in the image, and the w and h parameters represent the width and height of the predicted rectangular box of the target.

[0094] In view of the error between the predicted bounding box by the previous YOLOV5 network and the ground truth, and at the same time to ensure that the generated image input to the latter YOLOV5 network contains the complete target image, it is necessary to perform a non - linear function transformation on the predicted target position information det of the previous YOLOV5 network. Specifically, it mainly adjusts the w and h parameters in the predicted position information while keeping the x and y parameters of the predicted target center point unchanged.

[0095] The specific non - linear function transformation formula is as shown in (8).

[0096]

[0097] Among them, α1 and α2 are transformation factors and their values are both greater than 1. When w>h, the predicted target is a flat shape, corresponding to α1>α2; when w<h, the predicted target is a long and straight shape, corresponding to α1<α2. The specific transformation factors are appropriately adjusted according to the dataset. Finally, the extended target position information det1=(x,y,w1,h1) is obtained.

[0098] In this application, the adaptive scaling method of the smoking - like behavior target

[0099] In object detection, the input images to the network have different sizes. However, according to experimental experience, when the input network size is scaled to the same size, the detection effect is better. Therefore, for the det1 position information generated after the extension of the class target position information in step 5, after the class target is cropped, the generated image sizes are different. Therefore, for better detection effect of the latter YOLOV5, the cropped images are adaptively scaled to a unified size.

[0100] Suppose the image size of the input YOLOv5 network is 640, and the size of the image before adaptive scaling is (w1, h1). Obviously, both w1 and h1 are less than 640. The scaling ratio r is calculated as shown in equation (9).

[0101]

[0102] Then enlarge the image size to w1*r×h1*r. If r=r1, then it may be necessary to fill the height with a black border. If h1*r is not a multiple of 32, then a black border needs to be filled. If r=r2, then it may be necessary to fill the width with a black border. If w1*r is not a multiple of 32, then a black border needs to be filled. The formula for calculating the black border is shown in (10).

[0103]

[0104] Where d represents the width or length of the black border on one side, and [×] is the rounding function. Therefore, if a height-based black border is filled, the image size is 640×h1*r+2d; if a width-based black border is filled, the image size is w1*r+2d×640. In this case, both the length and width are multiples of 32, which meets the network structure's requirement of 5 downsampling operations.

[0105] The generated image after adaptive image scaling has the smallest possible black border size, reducing information redundancy, while also having a uniform size, which facilitates detection by the subsequent YOLOv5 network.

[0106] In this application, the target coordinate information det3 of the smoking behavior is inversely calculated to obtain the image coordinate information det4 relative to the video frame. The target position information det3 predicted by the subsequent YOLOV5 network is the position information relative to the input generated graph of the subsequent YOLOV5 network. However, the prediction box needs to be drawn on the input video frame of the unidirectional serial YOLOV5 network. Therefore, the generated coordinate information det3 needs to be inversely transformed to calculate the position information det4 relative to the input video frame of the serial network, where det4 = (x4, y4, w4, h4). The specific calculation formula is shown in (11).

[0107] Let the predicted target location information output by the previous YOLOv5 network be det = (x, y, w, h), the input generated map size of the next YOLOv5 network be (w2, h2), the predicted target location information output by the next YOLOv5 network be det3 = (x3, y3, w3, h3), and the position information of the predicted target output by the next YOLOv5 network relative to the input video frame of the unidirectional serial YOLOv5 network be det4 = (x4, y4, w4, h4).

[0108]

[0109] Where m1, m2, ρ1, ρ2, l1, and l2 are all adjustable scaling coefficients. The coordinate position information of the predicted target on the input video frame is obtained through the formula, which is convenient for subsequent result display.

[0110] The nonlinear function for calculating the target confidence level of smoking behavior in this application.

[0111] The confidence level of the predicted target output by a unidirectional serial YOLOv5 network after initial screening and fine screening cannot be simply assigned the confidence level predicted by the next YOLOv5 network. Instead, the final confidence level C should be calculated by function calculation in combination with the overall network structure characteristics.

[0112] Assume the confidence level of the target predicted by the previous YOLOv5 network is C1, and the confidence level of the target predicted by the next YOLOv5 network is C2. The specific formula for calculating the final confidence level C is shown in (12).

[0113]

[0114] Where β1, β2, and k are scaling factors. For the subsequent YOLOv5 network, the predicted target needs to be precisely selected; therefore, the value of β2 is usually greater than the value of β1.

[0115] The above description is merely a preferred embodiment of the present invention and should not be construed as limiting the scope of the invention. Those skilled in the art will understand that all or part of the processes for implementing the above embodiments and equivalent variations made in accordance with the claims of the present invention are still within the scope of the invention.

Claims

1. A small target detection method of a one-way serial YOLOV5 network, characterized by The method comprises the following steps: Step A, input a video stream of a small target to be detected, and perform frame-by-frame extraction to generate a video frame; Step B, obtain a predicted position det of a small target and a confidence C1 from a first YOLOV5 network; Step C, extend the predicted position det of the small target to obtain a target position det1, and cut and generate a preliminary detection image from the video frame according to the target position det1; Step D, input the preliminary detection image into a second YOLOV5 network to obtain a predicted position det3 of a small target and a confidence C2; Step E, obtain target prediction position information det4 according to the predicted position det3 of the small target, and mark the target prediction position information det4 on the video frame image generated in step A; Step F, obtain new target confidence C information using the confidence C1 and the confidence C2, and mark the target confidence C information on the video frame image generated in step A; The attention mechanism in the backbone network of the first YOLOV5 network is used to capture cross-channel information and assist the model to improve detection accuracy and precision; and the detection head of the neck network forms 4 scale outputs. The attention mechanism in the backbone network of the second YOLOV5 network is used to capture cross-channel information and cross-spatial dimension information, and assist the model to improve detection accuracy and precision; and the detection head of the neck network forms 2 scale outputs.

2. The small target detection method of the one-way serial YOLOV5 network according to claim 1, characterized in that: The step B is specifically: Step B-1, input the video frame into a single-direction series YOLOV5 network, perform feature extraction through the backbone network of the first YOLOV5 network to obtain different scale feature maps; Step B-2, input the obtained different scale feature maps into a feature fusion module of the first YOLOV5 network to fuse strong semantic information and strong positioning information of different scales.

3. The small target detection method of a one-way serial YOLOV5 network according to claim 1, characterized in that: The step D is specifically: Step D-1, input the preliminary detection image into the backbone network of the second YOLOV5 network of the single-direction series YOLOV5 network to perform feature extraction and obtain different scale feature maps; Step D-2, input the obtained different scale feature maps into a feature fusion module of the second YOLOV5 network to fuse strong semantic information and strong positioning information of different scales.

4. The small target detection method of the one-way serial YOLOV5 network according to claim 2 or 3, characterized in that: Before the obtained different scale feature maps are input into the feature fusion module, the different scale feature maps are input into a feature pyramid network FPN to transfer strong semantic information, and the output of the feature pyramid network FPN is input into a path aggregation network PAN to transfer strong positioning information of a bottom layer.

5. The small target detection method of a one-way serial YOLOV5 network according to claim 1, characterized in that: The extension in step C is used to adjust the width and height of the predicted rectangular frame.

Images