A target detection method based on a lightweight target detection convolutional neural network

Abstract

The application discloses a target detection method based on a light-weight target detection convolutional neural network, which comprises the following steps: determining the batch composition mode of images with different sizes according to the width-height distribution of training images in a training image set, and generating a plurality of batches of training images; converting the annotation information after pre-processing the plurality of batches of training images; training an initial light-weight target detection convolutional neural network according to each batch of training images after the conversion of the annotation information and a preset loss function, to obtain a trained light-weight target detection convolutional neural network; obtaining a test image set, and obtaining a target detection result according to the test image set and the trained light-weight target detection convolutional neural network. The whole process from the design of a target detection convolutional neural network structure, data preprocessing to model training is provided, the problem of dense target detection when the size difference of training images is large is solved, and the detection rate of distributed dense, small and basically consistent targets is effectively improved.

Description

Technical Field

[0001] This invention relates to the field of deep learning object detection technology, and in particular to an object detection method based on a lightweight object detection convolutional neural network. Background Technology

[0002] In daily life and industrial production, scenarios requiring target detection are frequently encountered. A typical optical character recognition process includes character detection steps, requiring the detection and classification of individual characters, or classification during detection. Another scenario, such as blood cell counting under a microscope, requires detecting blood cells in an image to obtain the total number of blood cells on the slide. Target detection is a crucial step in various image processing processes, directly affecting the final result of the entire image-related processing. These scenarios share some common characteristics. For example, the size of the targets, such as characters and blood cells, is relatively small compared to the overall image size, and their distribution may be relatively dense. This is particularly evident in characters, where the distance between characters is small. Within the same image, the target sizes are relatively consistent, with most being similar. Therefore, there is an urgent need for a target detection method to address the problem of dense target detection when image sizes vary significantly. Summary of the Invention

[0003] To address the above technical problems, this invention provides a target detection method based on a lightweight target detection convolutional neural network.

[0004] The technical solution adopted by this invention to solve its technical problem is:

[0005] A target detection method based on a lightweight target detection convolutional neural network, the method includes the following steps:

[0006] Step S100: Obtain the training image set, determine the batching method of images of different sizes according to the width and height distribution of the training images in the training image set, and generate multiple batches of training images;

[0007] Step S200: Preprocess the training images from multiple batches, and convert the annotation information of the preprocessed training images from multiple batches.

[0008] Step S300: Train the initial lightweight object detection convolutional neural network with the training images of each batch after the conversion of the annotation information and the preset loss function to obtain the loss value of each batch until the set number of training iterations of the current batch is completed. Evaluate the evaluation index of the initial lightweight object detection convolutional neural network based on the loss value of each batch. Adjust the network parameter values ​​of the initial lightweight object detection convolutional neural network according to the evaluation index until the evaluation index reaches the expected set value, and obtain the trained lightweight object detection convolutional neural network.

[0009] Step S400: Obtain the test image set, and obtain the object detection result based on the test image set and the trained lightweight object detection convolutional neural network.

[0010] Preferably, step S100 includes:

[0011] Step S110: Let all training images be set P(w,h), where w and h are the width and height of the images, and the total number of images is N. During training, a batch training image set P of size b is generated each time. b First, discretize the width and height values ​​of the images in set P according to the step size s. Then, count the number and index of images whose width and height fall within the nth discrete space [s*n, s*(n+1)]. Generate a histogram H(H) of the number and index of the discretized images in set P. w H h ), H w H h These are the width and height of the index histogram, respectively.

[0012] Step S120: Traverse the histogram H. For the i-th discrete space in H, if H i The number of images C i If it equals 0, then H i After deleting all discrete spaces with a value of 0, the remaining effective discrete space has a value of N. h And calculate the mean number C of images in the remaining effective discrete space. a If C a <b, let C a =b;

[0013] Step S130: Traverse the histogram H. For the i-th discrete space in H, if H i The number of images C i Less than C a Then randomly copy H i The image in the image is expanded until H i The number of images C i ≥C a ;

[0014] Step S140: Randomly shuffle the order of the histogram H, from H0 to H10. Nh Each time, b images are randomly selected to form a batch for training.

[0015] Step S150: Repeat step S140 to generate multiple batches of training images until all images in all discrete spaces of H have been extracted for training.

[0016] Preferably, step S200 involves preprocessing multiple batches of training images, including:

[0017] Step S210: Randomly crop the training images in the current batch: Based on the target annotation information of each image in the batch, calculate the minimum bounding rectangle of all labeled targets, and randomly crop between the minimum bounding rectangle and the entire image, that is: Let image P i (w i ,h i The image P contains a set of m labeled targets L(x1,y1,x2,y2,label), and is randomly cropped from the target set L(x1,y1,x2,y2,label). ic (x 1c ,y 1c ,x 2c ,y 2c Operate according to formula (1):

[0018]

[0019] Where random represents taking a random value, label represents the target's label, and x 10 ,x 11 ,x 12 ...,x 1m-1 The x-coordinate and y-coordinate of the top-left corner of the smallest bounding rectangle of the 0th to m-1th labeled targets are represented. 10 ,y 11 ,y 12 ...,y 1m-1 The x-coordinate represents the top-left corner y-coordinate of the smallest bounding rectangle of the 0th to m-1th labeled targets. 20 ,x 21 ,x 22 ...,x 2m-1 The x-coordinate and y-coordinate of the bottom right corner of the smallest bounding rectangle of the 0th to m-1th labeled targets are represented. 20 ,y 21 ,y 22 ...,y 2m-1 This represents the y-coordinate of the bottom right corner of the smallest bounding rectangle of the 0th to m-1th labeled targets;

[0020] Step S220: Add random Gaussian white noise, random brightness, hue, contrast transformation and random grayscale transformation to the training images of the current batch after random cropping;

[0021] Step S230: Calculate the average width and height w of the current batch of images after random cropping. a ,h a ;

[0022] Step S240: Scale and pad the randomly cropped training images of the current batch while maintaining the aspect ratio to w. a ,h a Among them, the filling method is selected as nearest neighbor copy.

[0023] Preferably, step S200 involves converting the annotation information of multiple batches of preprocessed training images, including:

[0024] Step S250: Calculate image P according to formula (2) i (w i ,h i The set of labeled targets L(x1,y1,x2,y2,label) contains m labeled targets, and the average width and height w of all labeled targets are given. L ,h L ;

[0025]

[0026] Where, x 1,i The x-coordinate of the top-left corner of the minimum bounding rectangle of the i-th labeled target is represented by x. 2,i The x-coordinate and y-coordinate of the bottom right corner of the minimum bounding rectangle of the i-th labeled target are represented. 1,i The y-coordinate of the top-left corner of the minimum bounding rectangle of the i-th labeled target is represented by y. 2,i The y-coordinate of the bottom right corner of the minimum bounding rectangle of the i-th labeled target;

[0027] Step S260: Traverse image P i (w i ,h i For all labeled targets, keep the center point of the minimum bounding box of each target unchanged, and use w for the width and height of all labeled targets. L ,h L Replacement, that is, for the labeled target L j (x j1 ,y j1 ,x j2 ,y j2 The transformed labeled target L is calculated using equation (3). j '(x' j1 ,y' j1 ,x'j2 ,y' j2 (label):

[0028]

[0029] Preferably, in step S300, the initial lightweight object detection convolutional neural network is based on the CenterNet object detection network structure, including using the MobinetV3 network as the backbone network, using a single 3×3 convolutional layer and upsampling layer to form a neck feature layer fusion structure, using a single 3×3 convolution and activation function to form the head part network output, and finally the lightweight object detection convolutional neural network generates and outputs the target probability value matrix H, the target center point offset matrix O, and the target width and height size matrix S.

[0030] Preferably, the preset loss function in step S300 is as follows:

[0031] L a =w1*L H +w2*L O +w3*L S (4)

[0032]

[0033]

[0034] Among them, L H For the target probability matrix H loss, L O For the target center point offset matrix O loss, L S Let S be the target width and height matrix loss, w1, w2, w3 be the weights of the target probability matrix loss, the target center point offset matrix loss, and the target width and height matrix loss, respectively, α, γ be the hyperparameters of the loss function, f(x) be the probability predicted by the network that there is a target at that location, and Y be the target width and height matrix loss. i It is a manual marking to indicate whether a target exists.

[0035] Preferably, the lightweight object detection convolutional neural network generates the target probability matrix H in the following ways:

[0036] The difference between the width wL and the height hL of the target is obtained. If the difference is greater than a preset threshold, the lengths of the horizontal and vertical axes of the ellipse under three different positional relationships of the preset prediction box and the annotation box are calculated according to equations (7) to (10), specifically:

[0037]

[0038]

[0039]

[0040] RB elph =RA elph *(w L / h L (10)

[0041] Among them, RA elph RB represents the length of the horizontal axis of the ellipse under the h-th positional relationship between the preset prediction box and the label box. elph w represents the length of the ellipse's vertical axis under the h-th positional relationship between the preset prediction box and the label box. L h represents the width of the target. L This indicates the height of the target, and T represents the set first threshold.

[0042] The target probability matrix H is obtained based on the lengths of the horizontal and vertical axes of the ellipse under the three different positional relationships of the preset prediction box and the label box. Among them, the first positional relationship of the preset prediction box and the label box is that the prediction box and the label box intersect; the second positional relationship of the preset prediction box and the label box is that the prediction box is embedded in the label box; and the third positional relationship of the preset prediction box and the label box is that the label box is embedded in the prediction box.

[0043] Preferably, the lightweight object detection convolutional neural network generates the target probability matrix H in the following ways:

[0044] The difference between the width wL and the height hL of the target is obtained. If the difference is less than a preset threshold, the length of the ellipse's horizontal axis and the length of its vertical axis are calculated according to equations (11) and (12), specifically:

[0045]

[0046]

[0047] Among them, w L h represents the width of the target. L RA represents the height of the target, RB represents the length of the horizontal axis of the ellipse, and t represents the set second threshold.

[0048] The target probability matrix H is obtained based on the lengths of the horizontal and vertical axes of the ellipse.

[0049] The object detection method based on the lightweight object detection convolutional neural network described above provides the entire process from object detection convolutional neural network structure design, data preprocessing to model training. It can effectively solve the problem of dense object detection when the training images have large size differences, and effectively improve the detection rate of densely distributed, small-sized and basically consistent objects. Attached Figure Description

[0050] Figure 1 This is a flowchart of a target detection method based on a lightweight target detection convolutional neural network, provided in an embodiment of the present invention.

[0051] Figure 2 This is an example of the operation of discretizing the width and height distribution of the training image in step S110 of an embodiment of the present invention;

[0052] Figure 3 This is a schematic diagram of a lightweight object detection convolutional neural network structure in one embodiment of the present invention;

[0053] Figure 4 This is an example of the expected output results generated in three cases in one embodiment of the present invention;

[0054] Figure 5 This is a schematic diagram showing the expected target detection results of the existing method.

[0055] Figure 6 This is a schematic diagram of the expected target detection result output of the present invention. Detailed Implementation

[0056] To enable those skilled in the art to better understand the technical solution of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings.

[0057] In one embodiment, such as Figure 1 As shown, a target detection method based on a lightweight target detection convolutional neural network is presented. The method includes the following steps:

[0058] Step S100: Obtain the training image set, determine the batching method of images of different sizes according to the width and height distribution of the training images in the training image set, and generate multiple batches of training images.

[0059] Specifically, the size of the training image set is used to determine how to form batches of images of different sizes.

[0060] In one embodiment, step S100 includes:

[0061] Step S110: Let all training images be set P(w,h), where w and h are the width and height of the images, and the total number of images is N. During training, a batch training image set P of size b is generated each time. bFirst, discretize the width and height values ​​of the images in set P according to the step size s. Then, count the number and index of images whose width and height fall within the nth discrete space [s*n, s*(n+1)]. Generate a histogram H(H) of the number and index of the discretized images in set P. w H h ), H w H h These are the width and height of the index histogram, respectively.

[0062] Step S120: Traverse the histogram H. For the i-th discrete space in H, if H i The number of images C i If it equals 0, then H i After deleting all discrete spaces with a value of 0, the remaining effective discrete space has a value of N. h And calculate the mean number C of images in the remaining effective discrete space. a If C a <b, let C a =b;

[0063] Step S130: Traverse the histogram H. For the i-th discrete space in H, if H i The number of images C i Less than C a Then randomly copy H i The image in the image is expanded until H i The number of images C i ≥C a ;

[0064] Step S140: Randomly shuffle the order of the histogram H, from H0 to H10. Nh Each time, b images are randomly selected to form a batch for training.

[0065] Step S150: Repeat step S140 to generate multiple batches of training images until all images in all discrete spaces of H have been extracted for training.

[0066] Specifically, a training iteration of one epoch is completed when all images in the discrete space have been traversed, extracted, and trained. Furthermore, the step size s in the discrete space can be set to 32 or 64.

[0067] The training set images exhibit significant variations in width and height. In this application scenario, the main factors contributing to these size variations are: first, the number of character lines varies greatly, ranging from one to five lines depending on the product being printed; second, the number of characters per line varies considerably, with some lines containing only a dozen characters while others contain dozens; and third, the character sizes are inconsistent. Even with the same number of character lines and the same characters, the image size can change during printing due to variations in character size settings and spacing. The image batching method proposed in this invention effectively solves the problem of significant variations in training image size.

[0068] Appendix Figure 2 This demonstrates an example of an operation after step S110 discretizes the training images according to their width and height. The numbers in the boxes represent the number of training set images whose width and height fall within this discrete space.

[0069] Step S200: Preprocess the training images from multiple batches, and convert the annotation information of the preprocessed training images from multiple batches.

[0070] In one embodiment, step S200 preprocesses multiple batches of training images, including:

[0071] Step S210: Randomly crop the training images in the current batch: Based on the target annotation information of each image in the batch, calculate the minimum bounding rectangle of all labeled targets, and randomly crop between the minimum bounding rectangle and the entire image, that is: Let image P i (w i ,h i The image P contains a set of m labeled targets L(x1,y1,x2,y2,label), and is randomly cropped from the target set L(x1,y1,x2,y2,label). ic (x 1c ,y 1c ,x 2c ,y 2c Operate according to formula (1):

[0072]

[0073] Where random represents taking a random value, label represents the target's label, and x 10 ,x 11 ,x 12 ...,x 1m-1 The x-coordinate and y-coordinate of the top-left corner of the smallest bounding rectangle of the 0th to m-1th labeled targets are represented. 10 ,y 11 ,y 12 ...,y 1m-1 The x-coordinate represents the top-left corner y-coordinate of the smallest bounding rectangle of the 0th to m-1th labeled targets.20 ,x 21 ,x 22 ...,x 2m-1 The x-coordinate and y-coordinate of the bottom right corner of the smallest bounding rectangle of the 0th to m-1th labeled targets are represented. 20 ,y 21 ,y 22 ...,y 2m-1 This represents the y-coordinate of the bottom right corner of the smallest bounding rectangle of the 0th to m-1th labeled targets;

[0074] Step S220: Add random Gaussian white noise, random brightness, hue, contrast transformation and random grayscale transformation to the training images of the current batch after random cropping;

[0075] Step S230: Calculate the average width and height w of the current batch of images after random cropping. a ,h a ;

[0076] Step S240: Scale and pad the randomly cropped training images of the current batch while maintaining the aspect ratio to w. a ,h a Among them, the filling method is selected as nearest neighbor copy.

[0077] Specifically, in formula (1), label represents the label of the target. For example, when recognizing characters, the label can be the number 1 or the letter a. Random cropping can increase the spatial randomness of the input image and improve the detection effect of the target detection network on targets at different locations.

[0078] Furthermore, an example of step S240 is as follows:

[0079] For example, given a batch of images [(90,120),(60,90)], the calculated average size is (75,105). Each image in the batch needs to be scaled and filled to the (75,105) size while maintaining its aspect ratio. Using formula (13), the scaling ratio in the height direction is 75 / 90, approximately 0.833, and the scaling ratio in the width direction is 105 / 120 = 0.875. Choosing 0.833 as the final scaling ratio, the image height is scaled to 90×0.833, approximately 75, and the image width is scaled to 120×0.833, approximately 100. At this point, the width direction differs from the target by 5 pixels. Two columns of pixels are filled on the left side of the scaled image, with pixel values ​​copied from the first and second columns of the original image (nearest neighbor filling). Three columns of zero pixels are filled on the right side to obtain the required (75,105) input size. The same method is used for images of (60,90).

[0080] S = X target / X org (13)

[0081] In one embodiment, step S200, which involves converting the annotation information of multiple batches of preprocessed training images, includes:

[0082] Step S250: Calculate image P according to formula (2) i (w i ,h i The set of labeled targets L(x1,y1,x2,y2,label) contains m labeled targets, and the average width and height w of all labeled targets are given. L ,h L ;

[0083]

[0084] Where, x 1,i The x-coordinate of the top-left corner of the minimum bounding rectangle of the i-th labeled target is represented by x. 2,i The x-coordinate and y-coordinate of the bottom right corner of the minimum bounding rectangle of the i-th labeled target are represented. 1,i The y-coordinate of the top-left corner of the minimum bounding rectangle of the i-th labeled target is represented by y. 2,i The y-coordinate of the bottom right corner of the minimum bounding rectangle of the i-th labeled target;

[0085] Step S260: Traverse image P i (w i ,h i For all labeled targets, keep the center point of the minimum bounding box of each target unchanged, and use w for the width and height of all labeled targets. L ,h L Replacement, that is, for the labeled target L j (x j1 ,y j1 ,x j2 ,y j2 The transformed labeled target L is calculated using equation (3). j '(x' j1 ,y' j1 ,x' j2 ,y' j2 (label):

[0086]

[0087] Specifically, suppose that the width and height of each target in a training image are calculated from the labeled data of the target objects as [(24,40),(17,39),(25,41),(24,41),(23,38),(17,40),(21,39),(16,40),(21,39),(20,38),] and the average value obtained by rounding down according to Equation (2) is (21,40). The obtained (21,40) is used as the width and height of all targets in the image.

[0088] Replacing the width and height values ​​of all labeled targets with the average width and height can reduce the impact of labeling errors on training accuracy.

[0089] Since this invention is designed for scenarios where targets are densely distributed but their sizes are basically the same, such as character detection scenarios, a single image typically contains 10-64 characters, and each character is basically the same size. When manually annotating, due to the large number of targets and their dense distribution, various annotation errors are inevitable, such as offset annotation boxes, individual box annotation sizes being too large or too small, etc. Using the mean value as a substitute can reduce the above annotation errors.

[0090] Step S300: Train the initial lightweight object detection convolutional neural network according to the training images of each batch and the preset loss function to obtain the loss value of each batch until the training iteration of the current batch is completed by a set number. Evaluate the evaluation index of the initial lightweight object detection convolutional neural network according to the loss value of each batch. Adjust the network parameter values ​​of the initial lightweight object detection convolutional neural network according to the evaluation index until the evaluation index reaches the expected set value, and obtain the trained lightweight object detection convolutional neural network.

[0091] Specifically, multiple batches of training images are generated, and the loss value of each batch is calculated according to the preset loss function calculation method until the training iteration with a set number of epochs is completed. Then, the relevant metrics of the model (recall rate, false positive rate, etc.) are evaluated. The values ​​of w1, w2, and w3 are adjusted according to the evaluation results, and the above operation is repeated until the evaluation metrics reach the expected set values.

[0092] In one embodiment, the initial lightweight object detection convolutional neural network in step S300 is based on the CenterNet object detection network structure, including using the MobinetV3 network as the backbone network, using a single 3×3 convolutional layer and upsampling layer to form a neck feature layer fusion structure, using a single 3×3 convolution and activation function to form the head part network output, and finally the lightweight object detection convolutional neural network generates and outputs the target probability value matrix H, the target center point offset matrix O, and the target width and height size matrix S.

[0093] Specifically, the lightweight object detection convolutional neural network is based on the CenterNet object detection network structure. A typical object detection network structure can be divided into three parts: backbone, neck, and head. The backbone is used to obtain the features of the input image, usually using a network structure for classification to obtain multi-scale feature maps of the input image. The neck is used to fuse features, fusing the feature maps obtained from the backbone to better extract the features of the target. The head is used to further process the output that has fused features at different scales to obtain a specific output. The format of this output is related to the object detection network structure. For example, CenterNet outputs three matrices, which can be represented as the target probability value (heatmap) matrix H, the target center point offset matrix O, and the target width and height dimension matrix S.

[0094] A lightweight design for the object detection network is implemented using three parts: backbone, neck, and head.

[0095] Lightening the backbone achieves the most significant changes in speed and model size, but also has a huge impact on performance. After testing different classification network models, MobinetV3 was selected as the backbone network because of its excellent performance in speed, model size, and feature extraction capabilities, and its network structure can also benefit from general-purpose devices. The neck and head parts were redesigned to minimize the computational cost required by these two parts. The neck feature layer fusion structure is composed of a single 3×3 convolutional layer plus necessary upsampling layers, and the head part network output is composed of a single 3×3 convolution and activation function.

[0096] Furthermore, the structure diagram of the lightweight object detection convolutional neural network is as follows: Figure 3As shown, after passing through the MobinetV3 backbone network, four feature layers with different downsampling ratios are obtained: 4x downsampling, 8x downsampling, 16x downsampling, and 32x downsampling. To ensure that corresponding element-wise addition can be performed after upsampling of the feature layers, a convolution operation is performed on the four feature layers to maintain the same number of feature layers. The Conv layers added after the four feature layers all use 1*1 convolution kernels to perform channel fusion. Upsampling is used to upsample the low-resolution feature map to a suitable resolution and then fuse it with the feature map of the corresponding resolution. In practice, we set the number of feature layers to 32, meaning that the feature map obtained after the four feature maps undergo the Conv operation has 32 channels. For the 32x downsampling feature map, the number of feature layers obtained is 1200. After the Conv operation, the number of feature layers is 32. After the upsampling operation, a result with the same size as the 16x downsampling feature map is obtained. Then, the result is summed element-wise with the 16x downsampling feature map after the Conv layer. The fusion result of the 16x downsampled feature map and the 32x downsampled feature map is obtained. This step is repeated on the 8x downsampled and 4x downsampled feature maps to obtain the final feature map fusion result.

[0097] A Conv operation is performed again on the feature map fusion result. The purpose of this operation is to adapt the number of feature layers in the fusion result to the required number of output feature layers. Here, output1 corresponds to the target probability value (heatmap) matrix H. The number of layers in this matrix is ​​the number of actual classification categories. Here, we are performing single classification, so it is a single-layer feature map. The reason for choosing sigmoid as the activation function is that the output value range of the heatmap is [0,1), and the sigmoid function can achieve this function. output2 corresponds to the target center point offset matrix O. For all targets, there are only two offset directions, namely the X-axis and Y-axis directions. Therefore, this is a two-layer feature map. The offset is relative to the cell. When the target is located at a certain pixel position, this offset is used to fine-tune the target center. The value range is also [0,1), and the sigmoid function is also chosen. output3 corresponds to the width and height dimension matrix S, which corresponds to the width and height dimensions of all targets, respectively. Therefore, this is also a two-layer feature map. The target's width and height are integers, and the exact upper limit of the target's width and height cannot be given. The ReLU activation function can well represent the target's width and height.

[0098] In one embodiment, the preset loss function in step S300 is specifically:

[0099] L a =w1*L H +w2*LO +w3*L S (4)

[0100]

[0101]

[0102] Among them, L H For the target probability matrix H loss, L O For the target center point offset matrix O loss, L S Let S be the target width and height matrix loss, w1, w2, w3 be the weights of the target probability matrix loss, the target center point offset matrix loss, and the target width and height matrix loss, respectively, α, γ be the hyperparameters of the loss function, f(x) be the probability predicted by the network that there is a target at that location, and Y be the target width and height matrix loss. i The presence or absence of a target is manually marked; a value of 1 indicates the presence of a target, while a value of 0 indicates the absence of a target.

[0103] Specifically, for the three outputs of the network, reasonable loss functions are designed respectively: for the target probability value (heatmap) matrix H, the loss L H Since most of the values ​​in the generated target probability matrix are 0, and only the values ​​near the target center are positive (close to 1), to deal with the situation where the number of positive and negative samples differs greatly, FocalLoss of equation (4) is used to calculate the loss of the target probability matrix. FocalLoss can well balance the weights of positive and negative samples; for the target center offset matrix O, the loss L O Target width and height matrix S loss L S The L1 norm of equation (5) is used for calculation; the final loss L is calculated. a This is the weighted sum of the three losses.

[0104] In one embodiment, the lightweight object detection convolutional neural network generates the object probability matrix H in the following manner:

[0105] The difference between the width wL and the height hL of the target is obtained. If the difference is greater than a preset threshold, the lengths of the horizontal and vertical axes of the ellipse under three different positional relationships of the preset prediction box and the annotation box are calculated according to equations (7) to (10), specifically:

[0106]

[0107]

[0108]

[0109] RB elph =RA elph *(wL / h L (10)

[0110] Among them, RA elph RB represents the length of the horizontal axis of the ellipse under the h-th positional relationship between the preset prediction box and the label box. elph w represents the length of the ellipse's vertical axis under the h-th positional relationship between the preset prediction box and the label box. L h represents the width of the target. L This indicates the height of the target, and T represents the set first threshold.

[0111] The target probability matrix H is obtained based on the lengths of the horizontal and vertical axes of the ellipse under the three different positional relationships of the preset prediction box and the label box. Among them, the first positional relationship of the preset prediction box and the label box is that the prediction box and the label box intersect; the second positional relationship of the preset prediction box and the label box is that the prediction box is embedded in the label box; and the third positional relationship of the preset prediction box and the label box is that the label box is embedded in the prediction box.

[0112] Specifically, Figure 4 The diagram shows three different positional relationships between the preset prediction box and the annotation box. The dashed line represents the prediction box and the solid line represents the annotation box.

[0113] In one embodiment, the lightweight object detection convolutional neural network generates the object probability matrix H in the following manner:

[0114] The difference between the width wL and the height hL of the target is obtained. If the difference is less than a preset threshold, the length of the ellipse's horizontal axis and the length of its vertical axis are calculated according to equations (11) and (12), specifically:

[0115]

[0116]

[0117] Among them, w L h represents the width of the target. L RA represents the height of the target, RB represents the length of the horizontal axis of the ellipse, and t represents the set second threshold.

[0118] The target probability matrix H is obtained based on the lengths of the horizontal and vertical axes of the ellipse.

[0119] Specifically, the generation method of the target probability matrix H in the expected output of the lightweight target detection convolutional neural network is redesigned and calculated according to the new formula. Equations (7) to (10) are more suitable for drawing Gaussian ellipses when the target width and height values ​​differ significantly, and it is necessary to calculate the length of the horizontal axis and the vertical axis of the ellipse. Equations (11) and (12) are suitable for the case where the target width and height values ​​differ less. Equations (11) and (12) are different from the ideas of Equations (7) to (10). They directly draw an ellipse at the center point, and make the area of ​​the ellipse and the area of ​​the actual labeled rectangle meet the set requirements. In the formula, RA represents the length of the horizontal axis of the ellipse, RB represents the length of the vertical axis of the ellipse, and t represents the set threshold.

[0120] Appendix Figure 5 The result shown is the original calculation method for the Gaussian circle, without transforming all labeled targets. (Attached) Figure 6 The results of performing the transformation operation on all labeled targets according to equations (7) to (10) and using step S200 are shown. It can be clearly seen that in the appendix Figure 5 When the target's width and height remain relatively constant, the drawn label information can vary significantly, leading to some targets being missed during detection. Figure 6 In this system, the label information generated for each target is consistent, which is more conducive to target detection.

[0121] Step S400: Obtain the test image set, and obtain the object detection result based on the test image set and the trained lightweight object detection convolutional neural network.

[0122] This invention provides a complete workflow from object detection network structure design and data preprocessing to model training. The proposed lightweight object detection network is a redesign of the CenterNet object detection structure, using the more lightweight MobinetV3 network structure, which is more suitable for low-end devices, to extract image features. Simultaneously, only convolutional and upsampling layers are used during feature fusion to reduce computation and improve network detection speed. To address the issue of large differences in training image sizes, a batching method with limited image sizes is used to group images of different sizes together. Since the performance degradation caused by the lightweight design is addressed, data preprocessing methods such as random image cropping, aspect ratio maintenance scaling, color gamut transformation, and adding Gaussian white noise are combined with an optimized expected output generation method to improve the performance of the lightweight object detection network. It achieves a good trade-off between speed and accuracy, is suitable for various small and dense object detection scenarios, and performs well in object detection of images with different input sizes. It also features high accuracy and low processing time, and in practical applications, both speed and accuracy meet the requirements of actual industrial use.

[0123] The above provides a detailed description of a target detection method based on a lightweight target detection convolutional neural network provided by this invention. Specific examples have been used to illustrate the principles and implementation methods of this invention. The descriptions of the embodiments above are merely for the purpose of helping to understand the core ideas of this invention. It should be noted that those skilled in the art can make various improvements and modifications to this invention without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this invention.

Claims

1. A target detection method based on a lightweight target detection convolutional neural network, characterized in that, The method includes the following steps: Step S100: Obtain a training image set, and determine the batching method for images of different sizes based on the width and height distribution of the training images in the training image set, generating multiple batches of training images; Step S100 includes: Step S110: Let all training images be a set. P ( w,h ), w,h Image width and height, total number of images N During training, a number of [items] are generated each time. b batch training image set P b First, set P The width and height values ​​of the image are determined by the step size. s Discretization, the width and height of the statistical image fall on the th... n A discrete space[ s*n,s* ( n+ 1)] The number and index of images within the set are used to generate a collection. P Number of Discretized Images and Index Histogram H ( H w , H h ), H w , H h These are the width and height of the index histogram, respectively. Step S120: Traverse the histogram H ,right H The Middle i In a discrete space, if H i Number of images in C i If it equals 0, then the above H i After deleting all discrete spaces with a value of 0, the remaining effective discrete space is: N h And calculate the average number of images in the remaining effective discrete space. C a ,like C a < b ,make C a =b ; Step S130: Traverse the histogram H ,right H The Middle i In a discrete space, if H i Number of images in C i Less than C a Then copy randomly. H i The image in the image is expanded until... H i Number of images in C i ≥ C a ; Step S140: Randomly shuffle the histogram H The order, from H 0~ H Nh Each time, randomly select b A batch of images is formed for training; Step S150: Repeat step S140 to generate multiple batches of training images until all images in all discrete spaces of H have been extracted for training. Step S200: Preprocess the multiple batches of training images, and convert the annotation information of the preprocessed multiple batches of training images; Step S300: Train the initial lightweight object detection convolutional neural network with the training images of each batch after the conversion of the annotation information and the preset loss function to obtain the loss value of each batch until a set number of training iterations of the current batch are completed. Evaluate the evaluation index of the initial lightweight object detection convolutional neural network based on the loss value of each batch. Adjust the network parameter values ​​of the initial lightweight object detection convolutional neural network according to the evaluation index until the evaluation index reaches the expected set value, and obtain the trained lightweight object detection convolutional neural network. Step S400: Obtain a test image set, and obtain the target detection result based on the test image set and the trained lightweight target detection convolutional neural network.

2. The method according to claim 1, characterized in that, Step S200 involves preprocessing the multiple batches of training images, including: Step S210: Randomly crop the training images in the current batch: Based on the target annotation information of each image in the batch, calculate the minimum bounding rectangle of all labeled targets, and randomly crop between the minimum bounding rectangle and the entire image, that is: Let the image P i ( w i ,h i The number contained in ) is m Annotated target set L ( x 1, y 1, x 2, y 2, label), randomly cropped image P ic ( x 1c ,y 1c ,x 2c ,y 2c Operate according to formula (1): Where random represents taking a random value, and label represents the target's label. This represents the x-coordinate of the top-left corner of the smallest bounding rectangle of the 0th to m-1th labeled targets. This represents the top-left y-coordinate of the smallest bounding rectangle of the 0th to m-1th labeled targets. This represents the x-coordinate of the bottom right corner of the smallest bounding rectangle of the 0th to m-1th labeled target. This represents the y-coordinate of the bottom right corner of the smallest bounding rectangle of the 0th to m-1th labeled targets; Step S220: Add random Gaussian white noise, random brightness, hue, contrast transformation and random grayscale transformation to the training images of the current batch after random cropping; Step S230: Calculate the average width and height of the randomly cropped images in the current batch. w a , h a ; Step S240: Scale and fill the randomly cropped training images of the current batch while maintaining their aspect ratio. w a , h a Among them, the filling method is selected as nearest neighbor copy.

3. The method according to claim 2, characterized in that, Step S200 involves converting the annotation information of the preprocessed batches of training images, including: Step S250: Calculate the image according to formula (2) P i ( w i ,h i The number contained in ) is m Annotated target set L ( x 1, y 1, x 2, y 2, the average width and height of all labeled targets. w L , h L ; in, Let x represent the x-coordinate of the top-left corner of the smallest bounding rectangle of the i-th labeled target. This represents the x-coordinate of the bottom right corner of the minimum bounding rectangle of the i-th labeled target. Let represent the y-coordinate of the top-left corner of the smallest bounding rectangle of the i-th labeled target. The y-coordinate of the bottom right corner of the minimum bounding rectangle of the i-th labeled target; Step S260: Traverse the image P i ( w i ,h i For all labeled targets, keep the center point of the minimum bounding box of each target unchanged, and use the same width and height for all labeled targets. w L , h L Replacement, that is, for the labeled target L j ( x j1 , y j1 , x j2 , y j2 The transformed labeled target is calculated using formula (3). L j ’ ( x ’ j1 , y ’ j1 , x ’ j2 , y ’ j2 (label): 。 4. The method according to claim 3, characterized in that, In step S300, the initial lightweight object detection convolutional neural network is based on the CenterNet object detection network structure. It includes using the MobinetV3 network as the backbone network, using a single 3×3 convolutional layer and an upsampling layer to form a neck feature layer fusion structure, and using a single 3×3 convolution and activation function to form the head part network output. Finally, the lightweight object detection convolutional neural network generates and outputs the target probability matrix H, the target center point offset matrix O, and the target width and height dimension matrix S.

5. The method according to claim 4, characterized in that, The preset loss function in step S300 is as follows: （4） （5） （6） Where LH is the target probability value matrix H loss, LO is the target center point offset matrix O loss, and LS is the target width and height dimension matrix S loss. w 1, w 2, w 3 represents the weights of the target probability value matrix H loss, the target center point offset matrix O loss, and the target width and height dimension matrix S loss, respectively. , Here, f(x) is a hyperparameter of the loss function, and f(x) is the probability that the target is at the location predicted by the network. It is a manual marking to indicate whether a target exists.

6. The method according to claim 4, characterized in that, The lightweight object detection convolutional neural network generates the object probability matrix H in the following ways: The difference between the width wL and the height hL of the target is obtained. If the difference is greater than a preset threshold, the lengths of the horizontal and vertical axes of the ellipse under three different positional relationships of the preset prediction box and the annotation box are calculated according to equations (7) to (10), specifically: (7) (8) (9) (10) in, This represents the length of the horizontal axis of the ellipse under the preset h-th positional relationship between the prediction box and the label box. This represents the length of the ellipse's vertical axis under the preset h-th positional relationship between the prediction bounding box and the label box. w L Indicates the width of the target. h L Indicates the high level of the goal. T This indicates the first threshold value set. The target probability matrix H is obtained based on the lengths of the horizontal and vertical axes of the ellipse under the three different positional relationships of the preset prediction box and the label box. Among them, the first positional relationship of the preset prediction box and the label box is that the prediction box and the label box intersect; the second positional relationship of the preset prediction box and the label box is that the prediction box is embedded in the label box; and the third positional relationship of the preset prediction box and the label box is that the label box is embedded in the prediction box.

7. The method according to claim 4, characterized in that, The lightweight object detection convolutional neural network generates the object probability matrix H in the following ways: The difference between the width wL and the height hL of the target is obtained. If the difference is less than a preset threshold, the length of the horizontal axis and the length of the vertical axis of the ellipse are calculated according to equations (11) and (12), specifically: (11) (12) in, w L Indicates the width of the target. h L Indicates the high level of the goal. RA Represents the length of the horizontal axis of the ellipse RB This represents the length of the ellipse's vertical axis. t This indicates the set second threshold; The target probability matrix H is obtained based on the lengths of the horizontal and vertical axes of the ellipse.

Images