Target detection method and system based on target detection algorithm

Abstract

The invention discloses a target detection method and system based on a target detection algorithm, and belongs to the technical field of target detection, and the method comprises the steps: collecting a target image, carrying out the data marking, and constructing a data set; extracting a plurality of hierarchical features of the target image based on the data set; performing feature alignment and splicing on the plurality of hierarchical features, and extracting global features; fusing the global features with the plurality of hierarchical features, and inputting the fused features into a detection head to complete the construction of a target detection model; and training the constructed target detection model by using the data set, and detecting the target image by using the trained model. According to the method, through multi-layer extraction of the image features, the target in the image is detected by effectively utilizing multiple layers of information, the utilization efficiency of the image features is improved, and rapid and accurate detection of the target and more efficient information exchange among the multiple layers of features are realized.

Description

Technical Field

[0001] The present application relates to the field of target detection, and in particular to a target detection method and system based on a target detection algorithm. Background Art

[0002] As a basic visual task, object detection aims to identify the category of an object and locate the position of the object. Current object detection methods are mainly divided into two categories: two-stage methods and single-stage methods. The single-stage method completes object position estimation and category classification in one calculation, which is fast; the two-stage method first generates candidate regions, and then runs a classifier on these candidate regions to determine whether there is a target object. It is usually more accurate than the single-stage method, but slower.

[0003] After years of development, the YOLO series of models have occupied an important position in the field of target detection. The YOLO algorithm uses a single neural network for end-to-end target detection, and achieves a high detection speed by converting the target detection task into a regression problem. The classic Backbone-Neck-Head structure of the YOLO series extracts features from the original image through the Backbone and fuses the image through the Neck, and finally predicts the category and position of the object in the Head. However, due to the large number of paths in the network and the large number of indirect interactions, there are still problems such as slow speed, cross-layer information exchange and information loss during fusion, especially when dealing with small targets and overlapping targets, the performance is relatively poor, and the utilization efficiency of image feature points is low. Summary of the invention

[0004] The present application aims to provide a target detection method that can solve the problems of fast and accurate detection of targets and more efficient information exchange between multi-layer features.

[0005] In order to achieve the above-mentioned purpose, the technical solution of this application is:

[0006] A target detection method based on a target detection algorithm, comprising:

[0007] Collect target images and annotate data to build a data set;

[0008] Extracting multiple hierarchical features of the target image based on the data set;

[0009] Align and concatenate the multiple hierarchical features, and then extract global features;

[0010] The global feature is fused with multiple hierarchical features, and the fused features are input into the detection head to complete the construction of the target detection model;

[0011] The constructed target detection model is trained using the data set, and the trained model is used to detect the target image.

[0012] Optionally, the extracting multiple hierarchical features of the target image based on the data set includes:

[0013] Convolution operations, batch normalization and activation function operations are used to extract and transform data features;

[0014] Using pooling operations of different scales to acquire multi-scale features of the data;

[0015] According to the feature extraction and conversion and the acquisition of the multi-scale features, the multiple hierarchical features are gradually extracted.

[0016] Optionally, the multiple hierarchical features include: fine-grained features, low-level features, mid-level features and high-level features.

[0017] Optionally, aligning and concatenating the multiple hierarchical features and then extracting the global features comprises: extracting the global features by using 3D convolution, batch normalization, LeakyReLU activation function and dimensionality reduction operations.

[0018] Optionally, the global feature is fused with multiple hierarchical features, and the fused features are input into the detection head to complete the construction of the target detection model, including attributing the low-level features, the mid-level features and the high-level features to local features; using the tanh activation function to process the global features after convolution to obtain overall features; and using the sigmoid activation function to process the global features after convolution to obtain activated features.

[0019] Optionally, fusing the global feature with multiple hierarchical features and inputting the fused features into a detection head to complete the construction of the target detection model includes:

[0020] When performing feature fusion, local features, overall features and activation features are input;

[0021] Obtaining a fusion feature by fusing the local feature, the overall feature and the activation feature through the feature fusion;

[0022] The fusion features are further extracted by using a reparameterization operation.

[0023] Optionally, after further extracting the fused features by using the reparameterization operation, the method further includes: using a ReLU activation function to activate the features using the reparameterization operation.

[0024] Optionally, the constructed target detection model is trained using the data set, and the trained model is used to detect the target image, including: taking the data set as an input sample, randomly extracting samples from the data set as training samples, and using the remaining samples as test samples to select the optimal parameters of the model; and building a feature fusion structure target detection network based on the selection of the optimal parameters to achieve the target detection.

[0025] A target detection system based on a target detection algorithm, characterized in that it includes:

[0026] The C2f module is used to extract and transform the features of the input data through convolution operations, batch normalization, and activation function operations;

[0027] The SPPF module is used to obtain the features of the input data at multiple scales through pooling operations at different scales; and obtain multiple hierarchical features through the C2f module and the SPPF module;

[0028] A global feature extraction module, used for fusing and extracting the aligned multiple hierarchical features and outputting a global feature;

[0029] The feature fusion module is used to fuse and re-parameterize the local features, the overall features and the activation features, and output the re-parameterized fusion features.

[0030] Optionally, the target detection system further includes:

[0031] A backbone network, used to build the C2f module and the SPPF module, and extract the multiple hierarchical features of the target image;

[0032] The neck network is used to align and splice the multiple hierarchical features, and then build the global feature extraction module and the feature fusion module to extract the global features.

[0033] Beneficial effects: The target detection method and system based on the target detection algorithm provided by the present application can realize multi-level extraction of image features, effectively utilize multiple levels of information to detect targets in the image, and at the same time use the global feature extraction and use module to realize more efficient information exchange between multi-layer features, and propose a method for combining global features with local features, thereby improving the utilization efficiency of image feature points and realizing fast and accurate detection of targets.

[0034] In order to make the above features and advantages of the application more obvious and easy to understand, the following embodiments are specifically cited and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1Flow chart of the target detection method of the present application.

[0036] Figure 2 The data format of the images in the dataset used in this application.

[0037] Figure 3 This is the notation format for the anti-vibration hammer in the data set used in this application.

[0038] Figure 4 This is the structure diagram of the C2f module used in feature extraction in this application.

[0039] Figure 5 This is the structure diagram of the SPPF module used in feature extraction in this application.

[0040] Figure 6 This is the network structure diagram of the global feature extraction module proposed in this application.

[0041] Figure 7 This is a network structure diagram of the feature fusion module proposed in this application.

[0042] Figure 8 This is a comparison chart of the detection effects of the model proposed in this application and the single-stage target detection model.

[0043] In the drawings, like reference numerals refer to the same drawing elements. DETAILED DESCRIPTION

[0044] In order to make the purpose and technical solution of the embodiment of the present application clearer, the technical solution of the embodiment of the present application will be clearly and completely described in conjunction with the drawings of the embodiment of the present application. Obviously, the described embodiment is a part of the embodiment of the present application, not all of the embodiments. Based on the described embodiment of the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0045] In the first embodiment of this application, please refer to Figure 1 The present application provides a target detection method based on YOLOv8, and the target detection method based on YOLOv8 includes the following steps: S10~S50.

[0046] S10: Collect target images and perform data annotation to build a data set.

[0047] S20: Extract multiple levels of features of the target image based on the dataset.

[0048] S30: align and concatenate multiple hierarchical features, and then extract global features.

[0049] S40: Fusing the global features with the features at multiple levels, and inputting the fused features into the detection head to complete the construction of the target detection model.

[0050] S50: Use the data set to train the constructed target detection model, and use the trained model to detect the target image.

[0051] The target detection method and system based on the target detection algorithm provided in the present application can realize multi-level extraction of image features and effectively use multiple levels of information to detect targets in the image. At the same time, the global feature extraction and use module is used to realize more efficient information exchange between multiple layers of features, and a method for combining global features with local features is proposed, which improves the utilization efficiency of image feature points and realizes fast and accurate detection of targets.

[0052] In step S10, refer to Figure 1 In step S10, the target image is collected and data is annotated, and the labeled data set is used as an input sample.

[0053] As an example, take the anti-vibration hammer as an example. Figure 2 As shown in the figure, the anti-vibration hammer image is collected and the data is annotated. The label format is [class, x_center, y_center, width, height] to construct the data set X. Among them, class represents the target category, x_center represents the x-axis coordinate of the target center point, y_center represents the y-axis coordinate of the target center point, width represents the target width, and height represents the target height. Figure 3 As shown, this is the notation format of the anti-vibration hammer in the data set used in this application.

[0054] In step S20, please refer to Figure 1 In step S20, in the backbone network, multiple levels of features of the target image are extracted based on the data set, including: S201 to S202.

[0055] S201: Constructing a C2f module and an SPPF module in the backbone network for feature extraction.

[0056] S202: Use the C2f module and the SPPF module to gradually extract multiple levels of features.

[0057] As an example, this application takes the YOLOv8 algorithm in the target detection algorithm as an example, and adopts the backbone network and neck network of the YOLOv8 model. In step S201, the C2f module and the SPPF module are used in the backbone network of the YOLOv8 model for feature extraction; Figure 4The C2f module structure diagram used in feature extraction in this application is shown. The C2f module is a component in the yolo algorithm. It extracts and transforms the features of the input data through convolution operations, batch normalization, and activation function operations. The C2f module input is x and output is out:

[0058] CBS=Conv+BN+Silu Formula (1)

[0059] Bottleneck=conv 1×1 +conv 3×3 +conv 1×1 Formula (2)

[0060]

[0061] out = CBS (concat (f 0 ,f 1 ,f 2 ...f n )) Formula (4)

[0062] In formula (1), CBS represents the operations of convolution operation, batch normalization and activation function, Conv represents convolution operation, BN represents batch normalization, and Silu represents activation function; in formula (2), Bottleneck represents the stacking of 1×1 convolution operation, 3×3 convolution operation and 1×1 convolution operation, conv 1×1 Represents a convolution operation with a convolution kernel of 1×1, conv 3×3 represents a convolution operation with a convolution kernel of 3×3; in formula (3), f0, f1, …, fn represent the intermediate features after repeated Bottleneck operations, split represents a segmentation operation; in formula (4), concat represents a concatenation operation.

[0063] As an example, a SPPF module is constructed in the backbone network for feature extraction, such as Figure 5 The SPPF module structure diagram used in feature extraction of the present invention is shown. SPPF is the spatial pyramid pooling technology in the yolo algorithm, which obtains multi-scale feature information through pooling operations at different scales. The input of the SPPF module is x, and the output is out:

[0064]

[0065] out = conv(concat(f 0 ,f 1 ,f 2 ,f 3 )) Formula (6)

[0066] Among them, f0, f1, f2, and f3 in formula (5) represent the intermediate features after repeated maxpool operations, Conv represents the convolution operation, maxPool represents the maximum pooling operation, and concat in formula (6) represents the concatenation operation.

[0067] In step S202, the C2f module and the SPPF module are used to gradually extract features at multiple levels, including: in the backbone network, convolution operations and the C2f module are used to gradually extract features at different levels.

[0068] As an example, the features at different levels include: fine-grained features F fg , low-level features F s , middle-level features F m and high-level features F l . It can be expressed as:

[0069] F fg =C2f(Conv(Conv(x))) Formula (7)

[0070] F s =C2f(Conv(F fg )) Formula (8)

[0071] F m =C2f(Conv(F s )) Formula (9)

[0072] F l =SPPF(C2f(Conv(F m ))) Formula (10)

[0073] Among them, F in formula (7) fg represents the fine-grained feature, C2f is the C2f module constructed in step S21, and Conv represents the convolution operation; F in formula (8) s represents low-level features; F in formula (9) m represents the middle-level features; F in formula (10) l represents high-level features, and SPPF is the SPPF module constructed in step S22.

[0074] In step S30, please refer to Figure 1 In step S30, in the neck network of the YOLOv8 model, multiple hierarchical features are aligned and concatenated, and then global features are extracted.

[0075] As an example, in the neck network, the fine-grained feature F obtained in step S202 is fg , low-level features F s , middle-level features F m, and high-level features F l Perform feature alignment and output the aligned and spliced ​​features F align for:

[0076]

[0077] Among them, F in formula (11) align_1 represents the fine-grained features after alignment, F align_2 represents the low-level features after alignment, F align_3 represents the mid-level features after alignment, F align_4 represents the high-level features after alignment, avgpool represents the average pooling operation, Conv represents the convolution operation, and interpolate represents the bi-sex interpolation operation; at this time, the aligned and spliced ​​features F align is a feature with four three-dimensional features.

[0078] Furthermore, in the neck network, the global feature extraction module is used to extract features and the fine-grained features F fg , low-level features F s , middle-level features F m , and high-level features F l Perform fusion extraction, such as Figure 6 The following is a network model diagram of the global feature extraction module when the alignment scale is the mid-level feature size. global for:

[0079] F global_4d =3DDBL(concat(unsqueeze(F align ))) Formula (12)

[0080] F global =squeeze(3d-pooling(F global_4d )) Formula (13)

[0081] Where, 3DDBL in formula (12) represents the operation of 3D convolution, batch normalization and LeakyReLU activation function, F align To align the spliced ​​features, F global_4d represents the features in the form of 4D tensors extracted by 3DDBL after inserting the new dimension. Unsqueeze is a dimensionality increase operation, which adds a depth dimension to the four 3D features. Concat represents a concatenation operation, which concatenates the four 3D features into a 4D feature in the depth dimension. In formula (13), 3d-pooling is a 3D pooling operation, and squeeze is a dimensionality reduction operation, which deletes the dimension with a dimension value of 1.

[0082] In step S40, please refer to Figure 1 In step S40, in the neck network, the global features are fused with multiple hierarchical features, and the fused features are input into the detection head to complete the construction of the target detection model.

[0083] Furthermore, step S40 further includes:

[0084] S401: Inputting local feature F into the feature fusion module local_feat , global feature F global_feat and activation feature F global_act .

[0085] S402: Obtain fusion features by fusing local features, global features, and activation features using a feature fusion module.

[0086] S403: Using the re-parameterization operation Rep to further extract the fused features.

[0087] In step S401, the local feature F is input into the feature fusion module. local_feat , overall characteristics F global_feat and activation feature F global_act ,like Figure 7 As shown in FIG. 1 , it is a network model diagram of the feature fusion module proposed in this application, which is expressed by the formula:

[0088]

[0089] F global_feat =tanh(conv(F global )) Formula (15)

[0090] F global_act =sigmoid(conv(F global )) Formula (16)

[0091] Among them, F in formula (14) s represents low-level features, F m represents the middle-level features, F l represents high-level features, conv represents convolution operation, avgpool represents average pooling operation, interpolate represents bi-sex interpolation operation, and F in formula (15) global represents the global feature, tanh is the tanh activation function, and sigmoid is the sigmoid activation function.

[0092] In step S402, the local feature F is obtained by using the feature fusion module. local_feat , overall characteristics F global_feat And the activation feature F global_actThe fused feature F after fusion fusion for:

[0093] F fusion =F global_act *F local_feat +(1-F global_act )*F global_feat Formula (17)

[0094] In step S403, the fused features are further extracted by using a re-parameterization operation Rep, including: fusion Use the reparameterization block RepVGG to perform reparameterization operations to further extract features. The features F out for:

[0095] RepVGG=add(BN(conv 1×1 (x)),BN(conv 3×3 (x)),BN(x)) Formula (18)

[0096] F out =ReLU(RepVGG(ReLU(RepVGG(x)))) Formula (19)

[0097] In formula (18), RepVGG is a reparameterizable reparameterization block, x is the input of the reparameterization block RepVGG, add is the addition function, BN represents batch normalization, conv 1×1 Represents a convolution operation with a convolution kernel of 1×1, conv 3×3 represents a convolution operation with a convolution kernel of 3×3; in formula (19), ReLU is the ReLU activation function operation.

[0098] As an example, through step S20, step S30 and step S40, the target detection model proposed in this application is built.

[0099] In step S50, please refer to Figure 1 In step S50, the constructed target detection model is trained using the data set, and the trained model is used to detect the target image.

[0100] As an example, take the labeled dataset X as input sample and randomly select M from the dataset X. 1 samples are used as training samples, and the remaining M 2 samples are used as test samples to select the optimal parameters of the model; finally, based on the optimal parameters, an improved feature fusion structure target detection network is built to achieve target detection.

[0101] As an example, Figure 8A comparison chart of the detection effects of the model constructed by the target detection method proposed in this application and the single-stage target detection model, taking the anti-vibration hammer target detection provided in this embodiment as an example, where mAP is the mean average precision. It calculates the average of the AP values ​​of all categories, where AP (Average Precision) refers to the average value of the precision at different recall points. mAP@[0.5:0.95] means that mAP is calculated under different intersection-and-union ratio thresholds, specifically from 0.5 to 0.95, with a step size of 0.05. This means that for each category, AP values ​​under multiple different intersection-and-union ratio thresholds (for example, 0.5, 0.55, 0.6,..., 0.95) are calculated, and then the average of these AP values ​​is calculated to obtain the mAP of the category. Finally, the mAP of all categories is averaged to obtain the overall mAP@[0.5:0.95]. Finally, the speed of the model in anti-vibration hammer target detection is obtained through the test set test, and the generalization accuracy is reported. Through Figure 8 It can be seen that the average accuracy of the target detection method and target detection system provided by this application is greater than that of the single-stage target detection model, which is a significant improvement.

[0102] The target detection method and system based on the target detection algorithm provided in the present application can realize multi-level extraction of image features and effectively use multiple levels of information to detect targets in the image. At the same time, the global feature extraction and use module is used to realize more efficient information exchange between multiple layers of features, and a method for combining global features with local features is proposed, which improves the utilization efficiency of image feature points and realizes fast and accurate detection of targets.

[0103] Although the present application has been disclosed as above with the embodiments, it is not intended to limit the present application. Any person with ordinary knowledge in the technical field can make some changes and modifications without departing from the spirit and scope of the present application. Therefore, the scope of protection of the present application shall be determined by the scope of the attached patent application.

Claims

1. A target detection method based on a target detection algorithm, characterized in that: include, Collect target images and annotate data to build a data set; Extracting multiple hierarchical features of the target image based on the data set; Align and concatenate the multiple hierarchical features, and then extract global features; The multiple hierarchical features are fused, and the fused features are input into the detection head to complete the construction of the target detection model; The constructed target detection model is trained using the data set, and the trained model is used to detect the target image.

2. The target detection method based on the target detection algorithm according to claim 1, characterized in that: Extract multiple levels of features of the target image based on the data set, including: Convolution operations, batch normalization and activation function operations are used to extract and transform data features; Using pooling operations of different scales to acquire multi-scale features of the data; According to the feature extraction and conversion and the acquisition of the multi-scale features, the multiple hierarchical features are gradually extracted.

3. The target detection method based on the target detection algorithm according to claim 2, characterized in that: The multiple levels of features include: fine-grained features, low-level features, medium-level features and high-level features.

4. The target detection method based on the target detection algorithm according to claim 3, characterized in that: The multiple hierarchical features are aligned and spliced, and then the global features are extracted, including: using 3D convolution, batch normalization, LeakyReLU activation function and dimensionality reduction operations to extract the global features.

5. The target detection method based on the target detection algorithm according to claim 4, characterized in that: The step of fusing the global features with the multiple hierarchical features and inputting the fused features into the detection head to complete the construction of the target detection model includes attributing the low-level features, the mid-level features and the high-level features to local features; The tanh activation function is used to process the global features after convolution to obtain overall features; the sigmoid activation function is used to process the global features after convolution to obtain activation features.

6. The target detection method based on the target detection algorithm according to claim 5, characterized in that: The global feature is fused with multiple hierarchical features, and the fused features are input into the detection head to complete the construction of the target detection model, including: When performing feature fusion, local features, overall features and activation features are input; Obtaining a fusion feature by fusing the local feature, the overall feature and the activation feature through the feature fusion; The fusion features are further extracted by using a reparameterization operation.

7. The target detection method based on the target detection algorithm according to claim 6, characterized in that: After further extracting the fusion features by using the reparameterization operation, the method further includes: using a ReLU activation function to activate the features using the reparameterization operation.

8. The target detection method based on the target detection algorithm according to claim 7, characterized in that: The target detection model constructed by using the data set is trained, and the target image is detected using the trained model, including: taking the data set as an input sample, randomly extracting samples from the data set as training samples, and using the remaining samples as test samples to select the optimal parameters of the model; and building a feature fusion structure target detection network based on the selection of the optimal parameters to achieve the target detection.

9. A target detection system based on a target detection algorithm, characterized in that: include, The C2f module is used to extract and transform the features of the input data through convolution operations, batch normalization, and activation function operations; The SPPF module is used to obtain the features of the input data at multiple scales through pooling operations at different scales; and obtain multiple hierarchical features through the C2f module and the SPPF module; A global feature extraction module, used for fusing and extracting the aligned multiple hierarchical features and outputting a global feature; The feature fusion module is used to fuse and re-parameterize the local features, the overall features and the activation features, and output the re-parameterized fusion features.

10. The target detection system based on the target detection algorithm according to claim 9, characterized in that: Also includes, A backbone network, used to build the C2f module and the SPPF module, and extract the multiple hierarchical features of the target image; The neck network is used to align and splice the multiple hierarchical features, and then build the global feature extraction module and the feature fusion module to extract the global features.

Images