Target detection method, device, apparatus and storage medium

Abstract

The application discloses a target detection method and device, equipment and a storage medium, and the method comprises the steps of acquiring an image to be detected and an initial data set; performing image preprocessing on the initial data set to obtain a preprocessed data set; constructing an initial target detection model based on a YOLOv7 network, replacing the backbone network of the initial target detection model with an improved ConvNextv2 structure, adding a CA attention mechanism to the neck layer network of the initial target detection model, and replacing the detection head in the head network of the initial target detection model with a decoupled detection head to obtain an improved target detection model; training the improved target detection model through the preprocessed data set to obtain an optimized target detection model; and detecting the image to be detected according to the optimized target detection model to obtain a detection result. The application solves the problems of low small target detection precision and easy interference of model performance, and improves the precision of small target detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of target detection, and particularly relates to a target detection method, device and equipment and a storage medium. BACKGROUND

[0002] Nowadays, with the rapid development of artificial intelligence and the upgrading of computer hardware devices, the target detection algorithm has made a breakthrough, and the related research results have brought many conveniences to people's daily life.

[0003] At present, large and medium-sized target detection has achieved remarkable results, but with the rapid development of intelligent systems and the wide application of portable shooting devices, a large number of small targets exist in videos and images, and only detecting large and medium-sized objects cannot meet the actual needs, and many fields need to obtain key information from small targets. This fully shows that small target detection has great research value and application prospect. However, compared with large and medium-sized target detection, the performance index of the same detection algorithm used for small target detection is often low, and it is difficult to achieve the expected effect. This is because the resolution of small targets is low, the information is limited, and the key features are easily lost in the downsampling process, resulting in serious missing detection and false detection. In addition, the model performance is also disturbed by factors such as light intensity change, image noise, complex background and target occlusion, which further increases the detection difficulty. Therefore, it is necessary to improve the existing target detection algorithm to improve the precision in small target detection. SUMMARY

[0004] The main purpose of the present application is to provide a target detection method, device, equipment and storage medium, which aims to solve the technical problems of low small target detection precision and easy model performance interference in the prior art.

[0005] To achieve the above purpose, the present application provides a target detection method, which comprises the following steps:

[0006] Obtaining an image to be detected and an initial data set;

[0007] Image preprocessing is performed on the initial data set to obtain a preprocessed data set;

[0008] An initial target detection model is constructed based on a YOLOv7 network, a main network of the initial target detection model is replaced with an improved ConvNextv2 structure, a CA attention mechanism is added to a neck layer network of the initial target detection model, and a detection head in a head network of the initial target detection model is replaced with a decoupled detection head, to obtain an improved target detection model, wherein the improved ConvNextv2 structure is constructed based on a ConvNextv2 structure and obtained by replacing a down-sampling layer of the ConvNextv2 structure with an SPD module, and the ConvNextv2 structure includes a global response normalization layer;

[0009] The improved target detection model is trained through the preprocessed data set, to obtain an optimized target detection model.

[0010] The initial data set is preprocessed through image preprocessing, to obtain a preprocessed data set.

[0011] Optionally, the initial data set is preprocessed through image preprocessing, to obtain a preprocessed data set, including:

[0012] The original label of the initial data set is converted into a target label, to obtain a target data set.

[0013] The target data set is expanded through geometric transformation and color transformation, to obtain the preprocessed data set, wherein the geometric transformation includes at least one of flipping, rotating, cropping, deforming, scaling, adding noise and blurring, and the color transformation includes at least one of color transformation, erasing and filling.

[0014] Optionally, the improved target detection model is trained through the preprocessed data set, to obtain an optimized target detection model, including:

[0015] An image in the preprocessed data set is input into the improved ConvNextv2 structure of the improved target detection model for feature extraction, to obtain a reference feature map, wherein the improved ConvNextv2 structure includes an SPD module, a convolution layer and a global response normalization layer, and the convolution layer includes a depth convolution and a one-dimensional convolution.

[0016] The reference feature map is input into the neck layer network of the improved target detection model for encoding, to obtain a target feature map, wherein the neck layer network includes a CA attention mechanism.

[0017] The target feature map is input into the head network of the improved target detection model for detection, to output a bounding box and a classification label.

[0018] Determine a target loss function based on the marked box and the classification label;

[0019] The parameters of the improved target detection model are optimized by the target loss function to obtain an optimized target detection model.

[0020] Optionally, inputting the images in the preprocessed dataset into the improved ConvNextv2 structure of the improved object detection model for feature extraction to obtain a reference feature map includes:

[0021] Performing feature extraction on the images in the preprocessed data set to obtain an original feature map;

[0022] Slicing the original feature map through the SPD module in the improved ConvNextv2 structure to obtain a number of sub-feature maps;

[0023] Connect the plurality of sub-feature maps according to the channel dimension to obtain an intermediate feature map;

[0024] Inputting the intermediate feature map into the convolution layer for depthwise convolution and then performing one-dimensional convolution to obtain a convolution feature map;

[0025] The convolution feature map is input into the global response normalization layer for aggregation processing to obtain a reference feature map.

[0026] Optionally, the step of inputting the convolutional feature map into the global response normalization layer for aggregation processing to obtain a reference feature map includes:

[0027] Performing global feature aggregation on the convolutional feature map through L2 normal pooling to obtain an aggregated feature map;

[0028] Performing feature normalization on the aggregated features to obtain a normalized feature map;

[0029] Perform feature calibration on the normalized feature map to obtain a reference feature map.

[0030] Optionally, inputting the reference feature map into a neck layer network of the improved target detection model for encoding to obtain a target feature map includes:

[0031] Encoding the reference feature map in the horizontal direction and the vertical direction according to the first convolution kernel and the second convolution kernel through the CA attention mechanism to obtain a horizontal aggregate feature map and a vertical aggregate feature map, wherein the spatial ranges of the first convolution kernel and the second convolution kernel are different;

[0032] The horizontal aggregate feature map and the vertical aggregate feature map are spliced ​​together to obtain a spliced ​​tensor;

[0033] calculating the spliced tensor through a one-dimensional convolution transformation function to obtain a horizontal tensor and a vertical tensor;

[0034] calculating the horizontal tensor and the vertical tensor with a reference feature map to obtain a target feature map.

[0035] Optionally, the determining the target loss function according to the marking box and the classification label comprises:

[0036] calculating the marking box with a real box to obtain a position loss function;

[0037] calculating the classification label with a preset label to obtain a classification loss function;

[0038] determining the target loss function according to the position loss function and the classification loss function.

[0039] In addition, to achieve the above object, the application further provides a target detection device, which comprises:

[0040] an acquisition module, configured to acquire a to-be-detected image and an initial data set;

[0041] a processing module, configured to perform image preprocessing on the initial data set to obtain a preprocessed data set;

[0042] a construction module, configured to construct an initial target detection model based on a YOLOv7 network, replace a backbone network of the initial target detection model with an improved ConvNextv2 structure, add a CA attention mechanism in a neck layer network of the initial target detection model, and replace a detection head in a head network of the initial target detection model with a decoupled detection head to obtain an improved target detection model, wherein the improved ConvNextv2 structure is constructed based on a ConvNextv2 structure and obtained by replacing a down-sampling layer of the ConvNextv2 structure with an SPD module, and the ConvNextv2 structure comprises a global response normalization layer;

[0043] a training module, configured to train the improved target detection model through the preprocessed data set to obtain an optimized target detection model;

[0044] a detection module, configured to detect the to-be-detected image according to the optimized target detection model to obtain a detection result.

[0045] In addition, to achieve the above-mentioned purpose, the present invention also proposes a target detection device, which includes: a memory, a processor, and a target detection program stored on the memory and executable on the processor, wherein the target detection program is configured to implement the steps of the target detection method described above.

[0046] In addition, to achieve the above-mentioned purpose, the present invention further proposes a storage medium, on which a target detection program is stored. When the target detection program is executed by a processor, the steps of the target detection method described above are implemented.

[0047] The present invention obtains an image to be detected and an initial data set; performs image preprocessing on the initial data set to obtain a preprocessed data set; constructs an initial target detection model based on a YOLOv7 network, replaces the backbone network of the initial target detection model with an improved ConvNextv2 structure, adds a CA attention mechanism to the neck layer network of the initial target detection model, and replaces the detection head in the head network of the initial target detection model with a decoupling detection head to obtain an improved target detection model; trains the improved target detection model with the preprocessed data set to obtain an optimized target detection model; and detects the image to be detected according to the optimized target detection model to obtain a detection result. In the above manner, the improved ConvNextv2 structure, the CA attention mechanism, and the decoupling detection head are added to the initial target detection model to improve the model structure, and the detection model optimization is completed by training the improved model, thereby solving the current problems of low small target detection accuracy and susceptibility of model performance to interference, and improving the accuracy of small target detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is a structural diagram of a target detection device in a hardware operating environment involved in an embodiment of the present invention;

[0049] Figure 2 Schematic diagram of the flow of the first embodiment of the target detection method of the present invention;

[0050] Figure 3 1 is a structural diagram of an improved target detection model in the second embodiment of the target detection method of the present invention;

[0051] Figure 4 Schematic diagram of the flow of the second embodiment of the target detection method of the present invention;

[0052] Figure 5 Schematic diagram of the slicing operation in the second embodiment of the target detection method of the present invention;

[0053] Figure 6 This is a flowchart of the implementation of the CA attention mechanism in the second embodiment of the target detection method of the present invention;

[0054] Figure 7 SCE module schematic diagram in the second embodiment of the target detection method of the present application;

[0055] Figure 8 DPE module schematic diagram in the second embodiment of the target detection method of the present application;

[0056] Figure 9 Decoupling detection head schematic diagram in the second embodiment of the target detection method of the present application;

[0057] Figure 10 Structure block diagram of the first embodiment of the target detection device of the present application.

[0058] The implementation, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0059] It should be understood that the specific embodiments described herein are merely intended to explain the present application and not to limit the present application.

[0060] Reference Figure 1 , Figure 1 Target detection device structure schematic diagram of the hardware running environment involved in the embodiment scheme of the present application.

[0061] As Figure 1 shown, the target detection device can include a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 can include a display screen, an input unit such as a keyboard, and an optional user interface 1003 can also include a standard wired interface, a wireless interface. The network interface 1004 can optionally include a standard wired interface, a wireless interface (such as a wireless fidelity (Wi-Fi) interface). The memory 1005 can be a high-speed random access memory (RAM), and can also be a stable non-volatile memory (NVM), such as a disk memory. The memory 1005 can also be an independent storage device from the aforementioned processor 1001.

[0062] Those skilled in the art can understand that Figure 1 the structure shown in the foregoing embodiments does not constitute a limitation on the target detection device, and can include more or fewer components than those shown in the drawings, or combine certain components, or different component arrangements.

[0063] As Figure 1 shown, the memory 1005 as a storage medium can include an operating system, a network communication module, a user interface module, and a target detection program.

[0064] In Figure 1 the target detection device shown, the network interface 1004 is mainly used for data communication with a network server; the user interface 1003 is mainly used for data interaction with a user; the processor 1001 and the memory 1005 in the target detection device of the present application can be arranged in the target detection device, and the target detection device calls the target detection program stored in the memory 1005 through the processor 1001, and executes the target detection method provided by the embodiments of the present application.

[0065] The embodiments of the present application provide a target detection method, which refers to Figure 2 , Figure 2 the flowchart of the first embodiment of the target detection method of the present application.

[0066] In this embodiment, the target detection method comprises the following steps:

[0067] Step S10: acquiring an image to be detected and an initial data set.

[0068] It should be noted that the execution subject of the present embodiment is a target detection device, and can also be other devices that can achieve the same or similar functions, which are not limited in the present embodiment, and the present embodiment is described taking the target detection device as an example.

[0069] It can be understood that the image to be detected is an image that needs to be detected, and the object to be detected in the image to be detected is a small target. There are two definitions of small target, one is according to the relative size, and the other is according to the absolute size. The relative size means that the target area is less than 80 pixels in a 256x256 image, that is, less than 0.12% of 256x256, which is a small target. The absolute size means that according to the definition of COCO data set, the target with a size less than 32x32 pixels can be considered as a small target. The small target can be a roadside traffic sign, an external defect of an article, etc., which is not limited in the present embodiment.

[0070] It is worth noting that the data set is a collection composed of data samples, and the initial data set is a collection composed of images to be detected. The initial data set can be a VisDrone2021 data set, which is not limited in the present embodiment.

[0071] Step S20: image preprocessing is performed on the initial data set to obtain a preprocessed data set.

[0072] It should be noted that, since machine learning needs to be learned through a large number of image samples, the initial data set is preprocessed, the data set is expanded, a richer data set image is obtained for model training, and the generalization ability of the model is improved.

[0073] Further, in order to improve the accuracy of the detection model, the step S20 comprises: converting the original label of the initial data set into a target label to obtain a target data set; and expanding the target data set through geometric transformation and color transformation to obtain a preprocessed data set, wherein the geometric transformation comprises at least one of flipping, rotating, cropping, deforming, scaling, adding noise and blurring, and the color transformation comprises at least one of color transformation, erasing and filling.

[0074] It should be noted that the images in the initial data set are all provided with original labels, and the original labels include: the horizontal coordinate of the upper left corner of the annotation box <bbox_left>, the vertical coordinate of the upper left corner of the annotation box <bbox_top>, the width of the annotation box <bbox_width>, the height of the annotation box <bbox_height>, the score <score>, target category <object_category>, truncation rate <truncation>and an occlusion rate <occlusion>and the like, and the present embodiment is not specifically limited thereto.

[0075] It can be understood that the original label of the initial data set is converted into a target label by label format conversion, and the target label includes: category <c>, bounding box center horizontal coordinate <x>, a center vertical coordinate of the bounding box <y>, the bounding box is relatively wide <w>, bounding box relatively high <h>Etc., this embodiment does not impose any specific limitation on this.

[0076] In a specific implementation, the data augmentation includes geometric transformation and color transformation operations. The geometric transformation operations include flipping, rotation, cropping, deformation, scaling, noise addition and blurring. The color transformation operations include color transformation, erasing and filling. Flipping includes horizontal flipping and vertical flipping. Noising is the addition of Gaussian noise, which can improve the robustness and generalization ability of the model. Since the image pixel size in the target data set is very high, the original image can be cropped into 2×2 images, which can increase the data volume while improving the model's detection ability for small targets.

[0077] Step S30: Construct an initial target detection model based on the YOLOv7 network, replace the backbone network of the initial target detection model with an improved ConvNextv2 structure, add a CA attention mechanism to the neck layer network of the initial target detection model, and replace the detection head in the head network of the initial target detection model with a decoupled detection head to obtain an improved target detection model, wherein the improved ConvNextv2 structure is constructed based on the ConvNextv2 structure and the downsampling layer of the ConvNextv2 structure is replaced by the SPD module, and the ConvNextv2 structure includes a global response normalization layer.

[0078] It should be noted that the initial target detection model is structurally improved, that is, the backbone network of the initial target detection model is replaced with an improved ConvNextv2 structure. The backbone network is used for feature extraction, and the backbone network is replaced with an improved ConvNextv2 structure. The improved ConvNextv2 structure is constructed based on the ConvNextv2 structure and the downsampling layer of the ConvNextv2 structure is replaced by the SPD module. The ConvNextv2 structure includes a global response normalization layer, a global response normalization layer (GRN), which is used to increase the contrast and selectivity between channels. The ConvNextv2 structure is based on the ConvNext structure, and the ConvNext structure is a pure convolutional structure.

[0079] like Figure 3 As shown, Figure 3 This is a structural diagram of the improved target detection model in the target detection method of this embodiment. The improved target detection model includes a 2D convolution block CONV2D, a layer normalization Layer Norm, a convolution layer ConvNextv2 Block, an SPD module, a CA attention mechanism, an upsampling layer upsample, a feature aggregation layer ELAN, a feature fusion layer Concat, an SCE module, a DPE module, and a detection head Detect.

[0080] It can be understood that the CA attention mechanism is added in the neck layer network of the initial target detection model, the neck layer network is used for feature fusion, the CA attention module aims to enhance the expression ability of the mobile network to learn features, and it can transform any intermediate feature tensor in the network and output a tensor of the same size.

[0081] It can be understood that the detection head in the head network of the initial target detection model is replaced by a decoupled detection head, the head network includes the detection head, the detection head is used for classification and positioning of the marking box, the decoupled detection head includes a regression detection head and a classification detection head, and the object position and the object category can be processed respectively, the decoupled detection head is used for decoupling, the classification and positioning loss are separated, different layer information is combined for final prediction, and the detection precision is improved.

[0082] Step S40: training the improved target detection model through the preprocessed data set to obtain an optimized target detection model.

[0083] It should be noted that the improved target detection model is trained through the preprocessed data set, the loss function is calculated according to the training result, the model parameters are optimized through the loss function, the model performance is improved, and the optimized target detection model is obtained.

[0084] Step S50: detecting the to-be-detected image according to the optimized target detection model to obtain a detection result.

[0085] According to the optimized target detection model, the to-be-detected image is detected, and the classification label and the positioning box of the to-be-detected object in the detection image are output.

[0086] In this embodiment, the to-be-detected image and the initial data set are obtained; the initial data set is preprocessed to obtain a preprocessed data set; an initial target detection model is constructed based on a YOLOv7 network, the backbone network of the initial target detection model is replaced by an improved ConvNextv2 structure, a CA attention mechanism is added in the neck layer network of the initial target detection model, and a decoupled detection head is used to replace the detection head in the head network of the initial target detection model to obtain an improved target detection model; the improved target detection model is trained through the preprocessed data set to obtain an optimized target detection model; and the to-be-detected image is detected according to the optimized target detection model to obtain a detection result. Through the above manner, the improved ConvNextv2 structure, the CA attention mechanism and the decoupled detection head are added to the initial target detection model to improve the model structure, and the detection model is optimized by training the improved model, thereby solving the problems of low small target detection precision and easy model performance interference, and improving the small target detection precision.

[0087] Reference Figure 4 , Figure 4 The flowchart of the second embodiment of the target detection method is shown in the figure.

[0088] Based on the first embodiment, the step S30 in the target detection method includes:

[0089] Step S401: input the image in the pre-processed data set into the improved ConvNextv2 structure of the improved target detection model for feature extraction to obtain a reference feature map, wherein the improved ConvNextv2 structure includes an SPD module, a convolution layer, and a global response normalization layer, and the convolution layer includes a depth convolution and a one-dimensional convolution.

[0090] It should be noted that feature extraction through the improved ConvNextv2 structure can reduce the parameter quantity and the calculation quantity, while maintaining the spatial information and increasing the perception ability of the model to different scale features.

[0091] Further, the step S401 includes: performing feature extraction on the image in the pre-processed data set to obtain an original feature map; performing slicing operation on the original feature map through the SPD module in the improved ConvNextv2 structure to obtain a plurality of sub-feature maps; connecting the plurality of sub-feature maps in the channel dimension to obtain an intermediate feature map; inputting the intermediate feature map into the convolution layer for depth convolution and then one-dimensional convolution to obtain a convolution feature map; and inputting the convolution feature map into the global response normalization layer for aggregation processing to obtain the reference feature map.

[0092] In a specific implementation, the specific operation of performing slicing operation on the original feature map through the SPD module in the improved ConvNextv2 structure to obtain a plurality of sub-feature maps includes: performing feature extraction on the image in the pre-processed data set to obtain an original feature map with a size of S×S×C1, inputting the original feature map into the SPD module, and performing slicing operation on the original feature map through the SPD module to obtain a series of sub-feature maps, as shown in the following formula 1. Figure X

[0093] f 0，0 = X[0:S:scale, 0:S:scale], f 1，0 = X[1:S:scale, 0:S:scale],..., f scale-1，0 = X[scale-1:S:scale, 0:S:scale]

[0094] f 0，1 = X[0:S:scale, 1:S:scale], f 1，1 ​..., f sale-1，1 = X [scale-1:S:scale, 1:S:scale]

[0095] f 0，scale-1 = X [0:S:scale, scale-1:S:scale], f 1，scale-1 ..., f sale-1，scale-1 = X [scale-1:S:scale, scale-1:S:scale] (Formula 1)

[0096] For the original feature Figure X , the sub-feature maps f x，y can be obtained by i+x, j+y divided by scale X(i, j), so that each sub-feature map is down-sampled by scale to the original feature Figure X . When scale = 2, the original feature Figure X is down-sampled by 2 to obtain four sub-feature maps f 0，0 , f 0，1 , f 1，0 , and f 1，1 , each sub-feature map has a shape of As shown in Figure 5 , Figure 5 is a schematic diagram of the slicing operation in the target detection method of the embodiment.

[0097] In a specific implementation, the specific operation of connecting the plurality of sub-feature maps in the channel dimension to obtain the intermediate feature map includes connecting four sub-feature maps in the channel dimension to obtain the connected feature Figure X ', which has a shape of Then, dimension reduction is performed through a convolution with C2 1x1, and information between channels is fused, where C2 < scale 2 < C1, to obtain the intermediate feature Figure X ", which has a shape of

[0098] It should be noted that the intermediate feature map enters the convolution layer ConvNextv2 Block, which is composed of one 7x7 deep convolution dwconv and two 1x1 convolutions for realizing the inverted bottleneck structure. The deep convolution dwconv can greatly reduce the calculation amount, and the calculation amount A of the standard convolution is as follows:

[0099] A = D k · D k · M · N · D F · D F (Formula 2)

[0100] Wherein, A is the standard convolution calculation amount, D k ×D k is the kernel size of the convolution kernel, M is the number of input channels, N is the number of output channels, D F ×D F is the feature map size.

[0101] And the calculation amount B of the depth convolution is as follows:

[0102] B=D k ·D k ·M·D F ·D F (Formula 3)

[0103] Wherein, B is the calculation amount of the depth convolution, D k ×D k is the kernel size of the convolution kernel, M is the number of input channels, D F ×D F is the feature map size.

[0104] As can be seen from formula 2 and formula 3, the depth convolution dwconv is

[0105] After two 1x1 convolutions realizing the inverted bottleneck structure, the convolution feature map enters the global response normalization layer GRN, enhances the competition between different channels, and increases the contrast and selectivity of the channels.

[0106] Further, the method comprises: inputting the convolution feature map into the global response normalization layer for aggregation processing to obtain a reference feature map; performing global feature aggregation on the convolution feature map through L2-norm pooling to obtain an aggregated feature map; performing feature normalization on the aggregated feature to obtain a normalized feature map; and performing feature calibration on the normalized feature map to obtain the reference feature map.

[0107] In a specific implementation, the GRN layer first completes global feature aggregation through L2-norm pooling, which can effectively suppress noise and prevent overfitting, and improve the model generalization capability. The global feature aggregation is as follows:

[0108] G(X)=X∈R H×W×C →gx ∈R c (Formula 4)

[0109] Wherein, X represents the feature map before input, gx represents the feature map after L2-norm pooling, G represents L2-norm pooling, H represents the height of the feature map, W represents the width of the feature map, and C represents the number of channels of the feature map.

[0110] The L2-norm pooling is specifically as follows:

[0111]

[0112] wherein, u c (i, j) represents a pixel point located in the c-th channel feature Figure X map, i and j represent the horizontal coordinate and the vertical coordinate of the pixel point respectively.

[0113] Finally, the aggregated feature map is obtained, as shown in the following formula 6:

[0114] G(X)=gx={||x1||,||X2||,...,||X C ||}∈R c (Formula 6)

[0115] wherein, gx represents the feature map after L2 norm pooling, that is, the aggregated feature map, X represents the feature, and C represents the number of channels of the feature map.

[0116] The aggregated feature map is normalized, G(X) i =||X i || represents the feature scalar of the i-th channel, and a normalized feature map is obtained, as shown in the following formula 7:

[0117]

[0118] wherein, N represents the normalized feature map, and X represents the feature.

[0119] The normalized feature map is calibrated to obtain a reference feature map, as shown in the following formula 8:

[0120] X i =X i *N(G(X) i )∈R H×W (Formula 8)

[0121] wherein, X i represents the reference feature map, N represents the normalized feature map, G represents the aggregated feature map, H represents the height of the feature map, and W represents the width of the feature map.

[0122] Step S402: inputting the reference feature map into a neck network of the improved target detection model to encode and obtain a target feature map, wherein the neck network comprises a CA attention mechanism.

[0123] It can be understood that the CA attention mechanism can encode the horizontal and vertical position information of the to-be-fused feature into the channel attention, so that the mobile network can pay attention to a large range of position information without bringing too much calculation amount, focus on important information with high weight, ignore irrelevant information with low weight, and constantly adjust the weight to locate the information of interest and send it into the detection head.

[0124] Further, the step S402 comprises: encoding the reference feature map along horizontal and vertical directions respectively by the CA attention mechanism according to first and second convolution kernels, to obtain an aggregated feature map along the horizontal direction and an aggregated feature map along the vertical direction, wherein the first and second convolution kernels have different spatial ranges; splicing the aggregated feature map along the horizontal direction and the aggregated feature map along the vertical direction to obtain a spliced tensor; calculating the spliced tensor by a one-dimensional convolution transformation function to obtain a horizontal tensor and a vertical tensor; and calculating the horizontal tensor and the vertical tensor with the reference feature map to obtain a target feature map.

[0125] In a specific implementation, as shown in FIG. 4, Figure 6 Figure 6 FIG. 4 is a flow chart of a CA attention mechanism in a target detection method according to an embodiment of the present application. The CA attention mechanism encodes each channel along horizontal and vertical coordinates by using first and second convolution kernels (H, 1) and (1, W) respectively, and thus the output of the cth channel with a height of h can be expressed as formula 9 as follows:

[0126]

[0127] wherein, represents the aggregated feature map along the vertical direction of the cth channel with a height of h, and W represents the width of the feature map.

[0128] The output of the cth channel with a width of w is expressed as formula 10 as follows:

[0129]

[0130] wherein, represents the aggregated feature map along the horizontal direction of the cth channel with a width of w, and H represents the height of the feature map.

[0131] The aggregated feature map along the horizontal direction and the aggregated feature map along the vertical direction are spliced to obtain a spliced tensor, which is expressed as formula 11 as follows:

[0132] f = δ (F1 ([z h , z w ]) ) (formula 11)

[0133] wherein, f is the spliced tensor, δ is a nonlinear activation function, represents the aggregated feature map along the vertical direction, represents the aggregated feature map along the horizontal direction, and F1 is a one-dimensional convolution transformation function.

[0134] The spliced tensor is split into a separate horizontal tensor and a vertical tensor along the spatial dimension, which is expressed as formula 12 as follows: ​

[0135] g h = σ(F h (f h )), g w = σ(F w (f w )) (Formula 12)

[0136] wherein g h is a horizontal tensor, g w is a vertical tensor, σ is an activation function, F represents a convolution transformation function, and f is a tensor before splitting.

[0137] The horizontal tensor and the vertical tensor are taken as attention weights, and the output of the last CA module is as follows in Formula 13:

[0138]

[0139] wherein y c (i, j) is a pixel point of the c-th channel feature map y, i and j respectively represent the horizontal coordinate and the vertical coordinate of the pixel point, x represents a feature, g h is a horizontal tensor, and g w is a vertical tensor.

[0140] Step S403: inputting the target feature map into a head network of the improved target detection model for detection, and outputting a label frame and a classification label.

[0141] It should be noted that the label frame is a rectangular frame marking the position of the to-be-detected object in the input image, and the classification label is a label inputting the category information of the to-be-detected object in the image.

[0142] It can be understood that the detection is performed by a decoupled detection head of the head network, and the decoupled detection head can be a TSCODE decoupled detection head.

[0143] Step S404: determining a target loss function according to the label frame and the classification label.

[0144] It should be noted that the function of the loss function is to measure the distance between the predicted information of the neural network and the expected information (label), and the closer the predicted information is to the expected information, the smaller the loss function value is. In this embodiment, the loss includes a classification loss and a position loss, and the target loss function is obtained according to the classification loss and the position loss.

[0145] Further, the target loss function is determined according to the label frame and the classification label, including: calculating the position loss function according to the label frame and the real frame; calculating the classification loss function according to the classification label and a preset label; and determining the target loss function according to the position loss function and the classification loss function.

[0146] It should be noted that the target loss function is determined according to the position loss function and the classification loss function, as shown in the following formula 14:

[0147]

[0148] wherein L is the target loss function, L cls represents the classification loss, L loc represents the position loss, is fused by the SCE module from the feature pyramid P l and P l+1 layer, is fused by the DPE module from the feature pyramid P l-1 , P l and P l+1 layer.

[0149] is fused by the SCE module from the feature pyramid P l and P l+1 layer as shown in the following formula 15:

[0150]

[0151] wherein Concat(·) represents stacking along the channel direction, DConv(·) represents shared down-sampling convolution, P l and P l+1 represent the feature pyramid layer, as shown in Figure 7 , Figure 7 is the schematic diagram of the SCE module in the target detection method of the present embodiment.

[0152] is fused by the DPE module from the feature pyramid P l-1 , P l and P l+1 layer as shown in the following formula 16:

[0153]

[0154] wherein P l-1 , P l and P l+1 represent the feature pyramid layer, μ(·) represents up-sampling, as shown in Figure 8 , Figure 8 is the schematic diagram of the DPE module in the target detection method of the present embodiment.

[0155] as shown in Figure 9 , Figure 9 To decouple the detection head in the target detection method of the embodiment, the improved detection model includes Backbone, Neck and Head. The backbone network Backbone acts as a feature extractor to generate multi-scale feature maps from the input image, then performs deep and shallow feature fusion, and finally inputs into the head for corresponding decoding output. TSCODE receives the feature maps from the output of the three middle levels and generates decoupled feature maps for classification and positioning.

[0156] It is worth noting that the target detection model using the classic decoupled detection head is based on the feature pyramid network of the same layer. The classic decoupled detection head includes regression and classification branches. The loss function of the classic decoupled detection head is as follows:

[0157] L = L cls (F C (P l ), c) + L loc (F r (P l , B) (Equation 17)

[0158] Where F c (·) = {f cls (·), C(·)}, F r = {f loc (·), R(·)} is a feature projection function for classification and positioning, l is the lth layer of the feature pyramid, B is the real box, c is the class label, C(·) and R(·) are the last layers of the regression and classification branches in the classic decoupled detection head, which decode the features into classification scores and bounding box positions.

[0159] Step S405: optimizing the parameters of the improved target detection model through the target loss function to obtain an optimized target detection model.

[0160] It should be noted that the model parameters are updated through the target loss function until the confidence of the model prediction exceeds a preset value, such as 90%, then the training and updating are stopped, and the current model is taken as the optimized target detection model.

[0161] The embodiment inputs the images in the preprocessed data set into the improved ConvNextv2 structure of the improved target detection model for feature extraction, to obtain a reference feature map, wherein the improved ConvNextv2 structure comprises an SPD module, a convolution layer, and a global response normalization layer, and the convolution layer comprises a depth convolution and a one-dimensional convolution; the reference feature map is input into a neck network of the improved target detection model for encoding, to obtain a target feature map, wherein the neck network comprises a CA attention mechanism; the target feature map is input into a head network of the improved target detection model for detection, to output a bounding box and a classification label; a target loss function is determined according to the bounding box and the classification label; the parameters of the improved target detection model are optimized through the target loss function, to obtain an optimized target detection model, in the above manner, the improved target detection model is trained through the preprocessed data set, the loss function is calculated according to the training result, and the model parameters are optimized through the loss function, to improve the model performance and obtain the optimized target detection model.

[0162] Referring to Figure 10 , Figure 10 Figure 1 is a structural block diagram of a first embodiment of a target detection device of the present application.

[0163] As Figure 10 shown, the target detection device proposed in the embodiment of the present application comprises:

[0164] An acquisition module 10 is configured to acquire an image to be detected and an initial data set.

[0165] A processing module 20 is configured to perform image preprocessing on the initial data set, to obtain a preprocessed data set.

[0166] A construction module 30 is configured to construct an initial target detection model based on a YOLOv7 network, replace a backbone network of the initial target detection model with an improved ConvNextv2 structure, add a CA attention mechanism to a neck network of the initial target detection model, and replace a detection head in a head network of the initial target detection model with a decoupled detection head, to obtain an improved target detection model, wherein the improved ConvNextv2 structure is constructed based on a ConvNextv2 structure and obtained by replacing a down-sampling layer of the ConvNextv2 structure with an SPD module, and the ConvNextv2 structure comprises a global response normalization layer.

[0167] A training module 40 is configured to train the improved target detection model through the preprocessed data set, to obtain an optimized target detection model.

[0168] The detection module 50 is configured to detect the to-be-detected image according to the optimized target detection model to obtain a detection result.

[0169] In the embodiment, the to-be-detected image and initial data set are obtained, the initial data set is preprocessed to obtain a preprocessed data set, an initial target detection model is constructed based on a YOLOv7 network, a backbone network of the initial target detection model is replaced with an improved ConvNextv2 structure, a CA attention mechanism is added to a neck network of the initial target detection model, and a detection head in a head network of the initial target detection model is replaced with a decoupled detection head to obtain an improved target detection model, the improved target detection model is trained by using the preprocessed data set to obtain an optimized target detection model, and the to-be-detected image is detected according to the optimized target detection model to obtain a detection result. In the above manner, the improved ConvNextv2 structure, the CA attention mechanism, and the decoupled detection head are added to the initial target detection model to improve the model structure, and the improved model is trained to complete the optimization of the detection model, thereby solving the problem of low detection precision of small targets and easy interference of model performance and improving the detection precision of small targets.

[0170] In addition, the present application also provides a target detection device, which comprises a memory, a processor, and a target detection program stored in the memory and executable on the processor, and the target detection program is configured to implement the steps of the target detection method described above.

[0171] Since the target detection device adopts all the technical solutions of the above-mentioned embodiments, it at least has all the beneficial effects brought by the technical solutions of the above-mentioned embodiments, which will not be repeated here.

[0172] In addition, the present application also provides a storage medium, which stores a target detection program, and the target detection program is executed by a processor to implement the steps of the target detection method described above.

[0173] Since the storage medium adopts all the technical solutions of the above-mentioned embodiments, it at least has all the beneficial effects brought by the technical solutions of the above-mentioned embodiments, which will not be repeated here.

[0174] It should be understood that the above is only for illustration, and does not constitute any limitation on the technical solutions of the present application. In specific applications, those skilled in the art can set it up according to the needs, and the present application does not limit it.

[0175] It should be noted that the above-described workflow is merely illustrative and does not limit the scope of protection of the present application. In actual applications, a person skilled in the art can select part or all of the above-described workflow to achieve the purpose of the present embodiment according to actual needs, which is not limited herein.

[0176] In addition, technical details not described in detail in the present embodiment can be found in the target detection method provided by any embodiment of the present application, which will not be described here.

[0177] In addition, it should be noted that in this document, the terms "comprise", "contain" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or system including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or system. Without more limitations, the element defined by the statement "comprises a" does not exclude the presence of another identical element in the process, method, article or system that includes the element.

[0178] The above-mentioned embodiment numbers of the present application are only for description, not representing the advantages and disadvantages of the embodiments.

[0179] From the above description of the embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be realized by means of software and necessary general hardware platforms, of course, they can also be realized by hardware, but in many cases the former is a better embodiment. Based on such understanding, the technical solutions of the present application or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as a read-only memory (ROM) / RAM, a magnetic disk, an optical disk) and includes a number of instructions to make a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) execute the methods described in various embodiments of the present application.

[0180] The above is only the preferred embodiment of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation, or direct or indirect application in other related technical fields, is also included in the patent protection scope of the present application.< / h> < / w> < / y> < / x> < / c> < / occlusion> < / truncation> < / score>

Claims

1. A target detection method, characterized in that: The method comprises: Obtain the image to be detected and the initial data set; Performing image preprocessing on the initial data set to obtain a preprocessed data set; An initial target detection model is constructed based on the YOLOv7 network, the backbone network of the initial target detection model is replaced with an improved ConvNextv2 structure, a CA attention mechanism is added to the neck layer network of the initial target detection model, and the detection head in the head network of the initial target detection model is replaced with a decoupled detection head, to obtain an improved target detection model, wherein the improved ConvNextv2 structure is constructed based on the ConvNextv2 structure and the downsampling layer of the ConvNextv2 structure is replaced with an SPD module, and the ConvNextv2 structure includes a global response normalization layer; Training the improved target detection model using the preprocessed data set to obtain an optimized target detection model; Detecting the image to be detected according to the optimized target detection model to obtain a detection result; The step of training the improved target detection model using the preprocessed data set to obtain an optimized target detection model includes: Inputting the images in the preprocessed dataset into the improved ConvNextv2 structure of the improved target detection model for feature extraction to obtain a reference feature map, wherein the improved ConvNextv2 structure includes an SPD module, a convolution layer, and a global response normalization layer, and the convolution layer includes depthwise convolution and one-dimensional convolution; Inputting the reference feature map into the neck layer network of the improved target detection model for encoding to obtain a target feature map, wherein the neck layer network includes a CA attention mechanism; Inputting the target feature map into the head network of the improved target detection model for detection, and outputting a marking box and a classification label; Determine a target loss function based on the marked box and the classification label; Optimizing the parameters of the improved target detection model by using the target loss function to obtain an optimized target detection model; The step of inputting the image in the preprocessed dataset into the improved ConvNextv2 structure of the improved target detection model for feature extraction to obtain a reference feature map includes: Performing feature extraction on the images in the preprocessed data set to obtain an original feature map; Slicing the original feature map through the SPD module in the improved ConvNextv2 structure to obtain a number of sub-feature maps; Connect the plurality of sub-feature maps according to the channel dimension to obtain an intermediate feature map; Inputting the intermediate feature map into the convolution layer for depthwise convolution and then performing one-dimensional convolution to obtain a convolution feature map; The convolution feature map is input into the global response normalization layer for aggregation processing to obtain a reference feature map.

2. The method according to claim 1, wherein The performing image preprocessing on the initial data set to obtain a preprocessed data set includes: Converting the original labels of the initial data set into target labels to obtain a target data set; The target data set is augmented by geometric transformation and color transformation to obtain a preprocessed data set, wherein the geometric transformation includes at least one of flipping, rotating, cropping, deforming, scaling, adding noise and blurring, and the color transformation includes at least one of color transformation, erasing and filling.

3. The method according to claim 1, wherein The step of inputting the convolutional feature map into the global response normalization layer for aggregation processing to obtain a reference feature map includes: Performing global feature aggregation on the convolutional feature map through L2 normal pooling to obtain an aggregated feature map; Performing feature normalization on the aggregated features to obtain a normalized feature map; Perform feature calibration on the normalized feature map to obtain a reference feature map.

4. The method according to claim 3, wherein The step of inputting the reference feature map into the neck layer network of the improved target detection model for encoding to obtain a target feature map comprises: Encoding the reference feature map in the horizontal direction and the vertical direction according to the first convolution kernel and the second convolution kernel through the CA attention mechanism to obtain a horizontal aggregate feature map and a vertical aggregate feature map, wherein the spatial ranges of the first convolution kernel and the second convolution kernel are different; The horizontal aggregate feature map and the vertical aggregate feature map are spliced ​​together to obtain a spliced ​​tensor; Calculating the spliced ​​tensor through a one-dimensional convolution transformation function to obtain a horizontal tensor and a vertical tensor; The target feature map is obtained by calculating the horizontal tensor and the vertical tensor and the reference feature map.

5. The method according to claim 1, wherein Determining the target loss function according to the marked box and the classification label includes: Calculate the position loss function based on the marked frame and the real frame; Calculate the classification loss function based on the classification label and the preset label; A target loss function is determined according to the position loss function and the classification loss function.

6. A target detection device, characterized in that: The target detection device comprises: An acquisition module is used to obtain the image to be detected and the initial data set; A processing module, configured to perform image preprocessing on the initial data set to obtain a preprocessed data set; A construction module is used to build an initial target detection model based on the YOLOv7 network, replace the backbone network of the initial target detection model with an improved ConvNextv2 structure, add a CA attention mechanism to the neck layer network of the initial target detection model, and replace the detection head in the head network of the initial target detection model with a decoupled detection head, thereby obtaining an improved target detection model, wherein the improved ConvNextv2 structure is constructed based on the ConvNextv2 structure and the downsampling layer of the ConvNextv2 structure is replaced by an SPD module, and the ConvNextv2 structure includes a global response normalization layer; A training module, configured to train the improved target detection model using the preprocessed data set to obtain an optimized target detection model; A detection module, configured to detect the image to be detected based on the optimized target detection model to obtain a detection result; The step of training the improved target detection model using the preprocessed data set to obtain an optimized target detection model includes: Inputting the images in the preprocessed dataset into the improved ConvNextv2 structure of the improved target detection model for feature extraction to obtain a reference feature map, wherein the improved ConvNextv2 structure includes an SPD module, a convolution layer, and a global response normalization layer, and the convolution layer includes depthwise convolution and one-dimensional convolution; Inputting the reference feature map into the neck layer network of the improved target detection model for encoding to obtain a target feature map, wherein the neck layer network includes a CA attention mechanism; Inputting the target feature map into the head network of the improved target detection model for detection, and outputting a marking box and a classification label; Determine a target loss function based on the marked box and the classification label; Optimizing the parameters of the improved target detection model by using the target loss function to obtain an optimized target detection model; The step of inputting the image in the preprocessed dataset into the improved ConvNextv2 structure of the improved target detection model for feature extraction to obtain a reference feature map includes: Performing feature extraction on the images in the preprocessed data set to obtain an original feature map; Slicing the original feature map through the SPD module in the improved ConvNextv2 structure to obtain a number of sub-feature maps; Connect the plurality of sub-feature maps according to the channel dimension to obtain an intermediate feature map; Inputting the intermediate feature map into the convolution layer for depthwise convolution and then performing one-dimensional convolution to obtain a convolution feature map; The convolution feature map is input into the global response normalization layer for aggregation processing to obtain a reference feature map.

7. A target detection device, characterized in that: The target detection device includes: a memory, a processor, and a target detection program stored in the memory and executable on the processor, wherein the target detection program is configured to implement the target detection method according to any one of claims 1 to 5.

8. A storage medium, characterized in that: The storage medium stores a target detection program, which, when executed by a processor, implements the target detection method according to any one of claims 1 to 5.

Images