Target recognition method

Abstract

The present application belongs to the field of artificial intelligence and computer vision recognition technology, and relates to a target recognition method, comprising: 1) determining a target to be recognized and obtaining an image data set of the target to be recognized; 2) constructing an overall target recognition positioning network, the overall target recognition positioning network comprising a backbone feature extraction network and a strengthened feature extraction network DFPNet arranged after the backbone feature extraction network; 3) pre-training the overall target recognition positioning network by using a standard data set MSCOCO to generate pre-training weights; 4) training the strengthened feature extraction network DFPNet based on the image data set of the target to be recognized and the pre-training weights to generate final network prediction weights; and 5) recognizing and positioning the target to be recognized by using the final network prediction weights. The present application provides a target recognition method based on a convolutional neural network model which can realize performance optimization of target detection precision.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of artificial intelligence and computer vision recognition, and relates to a target recognition method, in particular to a target recognition method of a strengthened feature extraction network model based on multi-feature bidirectional fusion. BACKGROUND

[0002] With the rapid progress of information technology, the importance of target detection based on machine vision in computer graphics is increasing, which has been integrated into people's daily life. Target detection is a comprehensive research topic of image processing, computer vision, pattern recognition and many other disciplines, and has important theoretical research significance and practical research value in medical, transportation and aerospace fields.

[0003] With the rapid development of graphics processing capability, several common deep learning models capable of detecting specific targets in complex unconstrained environments have been proposed in the field of computer graphics. Compared with the feature matching method which is easily affected by noise, the deep learning model can better complete the target detection task. Among them, the two-stage detection algorithm Fast R-CNN and the single-stage detection algorithm SSD and YOLO series represented target detection algorithm has achieved good results in target recognition detection. However, the accuracy and positioning precision of the currently disclosed target recognition methods for target recognition are uneven. SUMMARY

[0004] In order to solve the above technical problems existing in the background art, the application provides a target recognition method capable of realizing performance optimization of target detection precision.

[0005] In order to achieve the above purpose, the application adopts the following technical scheme:

[0006] A target recognition method, characterized in that the target recognition method comprises the following steps:

[0007] 1) determining a target to be recognized, and acquiring an image data set of the target to be recognized;

[0008] 2) constructing an overall target recognition positioning network, wherein the overall target recognition positioning network comprises a backbone feature extraction network and a strengthened feature extraction network DFPNet placed after the backbone feature extraction network;

[0009] 3) pre-training the overall target recognition positioning network by using a standard data set MSCOCO to generate pre-training weights;

[0010] 4) training the enhanced feature extraction network DFPNet in step 2) based on the image dataset of the target to be identified obtained in step 1) and the pre-training weights generated in step 3) to generate final network prediction weights;

[0011] 5) identifying and positioning the target to be identified using the final network prediction weights generated in step 4).

[0012] Preferably, the image dataset of the target to be identified in step 1) is obtained by shooting the target to be identified at different positions, different angles, different quantities, different sizes, and different light intensities and using a labeling tool to make a label file.

[0013] Preferably, the backbone feature extraction network in step 2) is Swin-Transformer, and the enhanced feature extraction network DFPNet is constructed based on PANet.

[0014] Preferably, the specific implementation of step 3) is as follows:

[0015] 3.1) the standard dataset MSCOCO provides an input image to the backbone feature extraction network;

[0016] 3.2) the backbone feature extraction network extracts features from the input image in step 3.1) and outputs feature maps of three different feature extraction stages to the enhanced feature extraction network DFPNet;

[0017] 3.3) the enhanced feature extraction network DFPNet further extracts features from the three different feature extraction stage feature maps input in step 3.2), and completes target position regression and target category classification in the last ordinary convolution layer until pre-training is completed.

[0018] 3.4) generating pre-training weights according to the pre-training result.

[0019] Preferably, the specific implementation of step 3.2) is as follows: using Swin-Transformer to extract features from the input image in step 3.1), and based on stage2, stage3, and stage4 in Swin-Transformer, outputting feature maps of three different feature extraction stages to the enhanced feature extraction network DFPNet.

[0020] As preferred, the specific implementation of step 3.3) adopted by the present application is: adding the bottleneck structure of the backbone feature extraction network to the enhanced feature extraction network DFPNet, the enhanced feature extraction network DFPNet further extracts features of the feature maps of the three different feature extraction stages through fusing the two paths of bottom-up and top-down, and completes the regression of the target position and the classification of the target category in the variable parameter convolution layer of the last layer until the pre-training is completed.

[0021] As preferred, the specific implementation of step 4) adopted by the present application is:

[0022] 4.1) adjusting the weight parameters of the overall target recognition positioning network based on the pre-training weight obtained in step 3);

[0023] 4.2) taking the image data set of the target to be recognized obtained in step 1) as an input object, and providing the input image to the backbone feature extraction network;

[0024] 4.3) the backbone feature extraction network extracts features of the input image in step 4.2), and outputs the feature maps of the three different feature extraction stages to the enhanced feature extraction network DFPNet;

[0025] 4.4) the enhanced feature extraction network DFPNet further extracts features of the three different feature extraction stages of the feature maps input in step 4.3), and completes the regression of the target position and the classification of the target category in the ordinary convolution layer of the last layer until the training of the overall target recognition positioning network is completed;

[0026] 4.5) generating the final network prediction weight according to the training result.

[0027] As preferred, the specific implementation of step 4.4) adopted by the present application is:

[0028] 4.4) the enhanced feature extraction network DFPNet further extracts features of the three different feature extraction stages of the feature maps input in step 4.3), and completes the regression of the target position and the classification of the target category in the ordinary convolution layer of the last layer, then introduces the EIOU loss function in the training, accelerates the training convergence of the network and guides the update of the weight parameters of the recognition positioning network in the training process to the direction of higher positioning accuracy.

[0029] The present application has the advantages of:

[0030] The application provides a target recognition method, which realizes performance optimization of target detection precision by bidirectional fusion of multiple features, improvement of a loss function and setting of hyperparameters in a training process while maintaining a certain complexity. The target recognition and positioning method provided by the application based on a Dual Feature Pyramid Networks (DFPNet) uses a multi-feature bidirectional fusion feature map method for feature enhancement for a common feature enhancement feature extraction network PANet, greatly improves the reuse of multi-layer feature maps and enhances the utilization efficiency of feature maps; in terms of a loss function, an EIOU loss function IOU positioning loss function is used, positioning precision is obviously improved, and an IOU hyperparameter is increased in the training process, further improving the positioning precision problem. The method of applying a convolutional neural network has good learning ability, fault tolerance and strong classification ability, and the great improvement of current computing power also allows more data to be calculated, a suitable network model is selected according to the characteristics (color, area, center point, etc.) of the target for training, and the main purpose of the application is to improve the accuracy and positioning precision of target detection by using the multi-feature bidirectional fusion enhanced feature extraction network. BRIEF DESCRIPTION OF DRAWINGS

[0031] Figure 1 is the enhanced feature extraction network DFPNet structure diagram based on multi-feature bidirectional fusion adopted by the application;

[0032] Figure 2 is the loss value change in the training process of the application adding EIOU and CIOU loss functions;

[0033] Figure 3 is the feature extraction network Swin-Transformer used by the application;

[0034] Figure 4 is an improved EIOU calculation schematic diagram of the application;

[0035] Figure 5 is a detection example diagram of the application on a charging interface data set. DETAILED DESCRIPTION

[0036] The purpose, technical scheme in the embodiments of the application will be clearly and completely described below in combination with the drawings in the embodiments of the application.

[0037] The application provides a target recognition method, which includes the following steps:

[0038] 1) determine the target to be identified, and obtain the image data set of the target to be identified; wherein the image data set of the target to be identified is a set of images obtained by shooting the target to be identified at different positions, different angles, different quantities, different sizes and different light intensities and using a labeling tool to make a label file, and the image data set of the target to be identified is used to train the overall target recognition positioning network;

[0039] 2) construct an overall target recognition positioning network, as shown in Figure 1 , the overall target recognition positioning network comprises a backbone feature extraction network and a strengthened feature extraction network DFPNet (Dual Feature Pyramid Networks, DFPNet, multi-feature bidirectional fusion strengthened feature extraction network) placed after the backbone feature extraction network; the backbone feature extraction network is Swin-Transformer, and the network structure thereof is as shown in Figure 3 ; the strengthened feature extraction network DFPNet is obtained based on PANet, and the bottleneck structure of the backbone feature extraction network is added to the strengthened feature extraction network DFPNet to further extract the fused feature map. The backbone network is selected at will, and the main function is to extract features according to the input image, and output feature maps at three different feature extraction stages. For example, the present application takes Swin-Transformer as an example, Swin-Transformer is the top feature extraction network of Transformer in the visual field, and the performance is outstanding in the tasks of target detection and instance segmentation. Swin Transformer is a backbone extraction network published by Microsoft Research Institute in 2021, which uses a hierarchical construction method similar to the convolutional neural network, and uses stage1-4 output feature maps whose sizes are reduced by 4 times, 8 times and 16 times. Such a backbone extraction network helps to construct target detection, instance segmentation and other tasks on this basis. Of course, the backbone feature network can also select other network structures. As shown in Figure 3 , the present application uses stage2, stage3 and stage4 in the Swin-Transformer network to output three different size feature maps as the input feature maps of the DFPNet.

[0040] 3) use the standard data set MSCOCO to pre-train the overall target recognition positioning network to generate pre-training weights, specifically:

[0041] 3.1) the standard data set MSCOCO provides input images to the backbone feature extraction network;

[0042] 3.2) The backbone feature extraction network extracts features from the image input in step 3.1) and outputs feature maps of three different feature extraction stages to the enhanced feature extraction network DFPNet, specifically: the Swin-Transformer is used to extract features from the image input in step 3.1), and the enhanced feature extraction network DFPNet outputs feature maps of three different feature extraction stages based on stage2, stage3 and stage4 in the Swin-Transformer.

[0043] 3.3) The enhanced feature extraction network DFPNet further extracts features from the three different feature extraction stage feature maps input in step 3.2) until pre-training is completed, specifically: the bottleneck structure of the backbone feature extraction network is added to the enhanced feature extraction network DFPNet, the enhanced feature extraction network DFPNet further extracts features from the three different feature extraction stage feature maps by fusing the bottom-up and top-down two paths, and the variable parameter convolution layer in the last layer completes the target position regression and target class classification until pre-training is completed. The enhanced feature extraction network DFPNet enhances the feature extraction capability of the backbone network by fusing the bottom-up and top-down two paths.

[0044] As shown in Figure 1 , the stage4 output feature map feat4 of the backbone feature extraction network is adjusted by 1x1 convolution to obtain P4, P4 is upsampled and combined with the stage3 output feature map feat3, then the bottleneck structure of the backbone feature extraction network is used for feature extraction, and after adjusting the channel by 1x1 convolution, P3 is obtained, P3 is upsampled and combined with the stage2 output feat2 feature map of the backbone feature extraction network to obtain P2, then the bottleneck structure of the backbone feature extraction network is used for feature extraction P2_out, the feature map of P2_out is convolved by 3x3 to downsample, and after downsample, it is combined with P3, then the bottleneck structure of the backbone feature extraction network is used for feature extraction P3_out, the feature map of P3_out is convolved by 3x3 to downsample, and after downsample, it is combined with P4, then the bottleneck structure of the backbone feature extraction network is used for feature extraction P4_out; the generated P2_out, P3_out, P4_out are used for one-time variable parameter convolution to perform target confidence classification and position regression analysis.

[0045] 3.4) Generate pre-training weights according to the pre-training results.

[0046] 4) training the enhanced feature extraction network DFPNet in step 2) based on the image data set of the target to be identified obtained in step 1) and the pre-training weight generated in step 3), to generate the final network prediction weight, specifically:

[0047] 4.1) adjusting the training parameters of the overall target identification and positioning network with the pre-training weight obtained in step 3);

[0048] 4.2) taking the image data set of the target to be identified obtained in step 1) as the input object, and providing the input image to the backbone feature extraction network;

[0049] 4.3) the backbone feature extraction network extracts features from the image input in step 4.2), and outputs feature maps of three different feature extraction stages to the enhanced feature extraction network DFPNet;

[0050] 4.4) further feature extraction of the three different feature extraction stages of the feature maps input in step 4.3) is performed by the enhanced feature extraction network DFPNet until the training of the overall target identification and positioning network is completed, specifically:

[0051] Further feature extraction of the three different feature extraction stages of the feature maps input in step 4.3) is performed by the enhanced feature extraction network DFPNet, and the target position regression and target category classification are completed in the last variable parameter convolution layer, and then the training loss function is improved.

[0052] As shown in Figure 2 under the same conditions, only the positioning loss function of the training process of the EIOU and CIOU loss functions is changed. It can be seen that after adding the EIOU loss function, the positioning loss function value of the network training decreases faster, and the convergence value is smaller, and after convergence, it is more stable. Therefore, the loss function introduced by the present application is the EIOU loss function, and the EIOU loss function can accelerate the training convergence of the network and guide the update of the identification and positioning network weight parameters in the training process to a higher positioning accuracy,

[0053] In the network training process, EIOU is introduced as the positioning loss function (this loss function is a prior art, and the present application only briefly describes it), and the specific expression of EIOU as the positioning loss function is:

[0054]

[0055] EIOU as a positioning loss function can guide the update of the weight parameters in the recognition positioning network during the training process, improve the positioning accuracy of the prediction box, and EIOU loss function contains three parts: the overlap loss of IOU, the center distance loss of the middle part, and the width and height loss of the last two polynomials, wherein the width and height loss is the aspect ratio (the aspect ratio is the ratio of the length and width of the network prediction box and the real labeled box, which belongs to the width and height loss in the EIOU loss function);

[0056] The last two polynomials The influence factor of the aspect ratio, the numerator is the Euclidean distance of the two points, and the denominator is the width or height of the minimum circumscribed rectangle) is calculated separately to calculate the relationship between the length and width of the prediction box and the real box (the prediction box is the box predicted by the network during the training process, which belongs to the prediction box generated by the network during training, verification and testing, and the real box is the box labeled by hand during the training of the data set, which belongs to the target box labeled by hand during the manual production of the data set), and the difference of the aspect ratio is clear, and the loss in two dimensions directly makes the convergence speed of the training faster. The calculation method of EIOU is as follows Figure 4 ( Figure 4 In which, the dark box represents the real labeled box, ground truth, and the code is gt; the light box represents the prediction box; the dashed box represents the minimum circumscribed rectangle of the real labeled box and the prediction box, and the code is c, b represents the center point of the box, h represents the height of the box, and w represents the width of the box), and the formula is as follows (wherein p is a coefficient):

[0057] In which, the calculation expression of IOU is:

[0058]

[0059] Increase the threshold value of the network prediction IOU during the training process, and increase the punishment intensity, from 0.5 to 0.75, that is, when the real box and the prediction candidate box overlap to 0.7, it is considered that the target is detected, so that the network iteration training pays more attention to the overlap degree of the prediction box and the real box, rather than simply detecting the target.

[0060] 4.5) According to the training result, generate the final network prediction weight.

[0061] 5) Use the final network prediction weight generated in step 4) to identify and locate the target to be identified.

[0062] Exemplarily, the data set of the application is a data set specially made for electric vehicle charging interface. The data set plays a crucial role in target detection. An excellent data set is beneficial to the training of the network and improves the accuracy of the algorithm result. The electric vehicle charging interface vehicle data set has a total of 5000 pictures. Electric vehicle direct current and alternating current charging interface images under different positions, different quantities, different sizes and different light intensities are collected, and corresponding label files are made by using an image labeling tool to obtain the charging interface data set. In network training, the proportion of the training set, the validation set and the test set is 5:3:2. After the target recognition method provided by the application, the experimental effect is as shown in Figure 5 . Figure 5 In the figure, a is a detection example image using the ciou positioning loss function with an iou threshold of 0.5, b is a detection example image using the ciou loss function and increasing the training iou threshold to 0.75, and c is a detection example image using the eiou loss function and increasing the training iou threshold to 0.75. Among them, the blue box is the real annotation box manually annotated, and the green box is the network prediction box. From the figure, the gap between the blue box and the green box is getting smaller and smaller, proving that the positioning accuracy of the network prediction is getting higher and higher. The red box represents a low probability box predicted by the network, which is directly discarded.

Claims

1. A target recognition method, characterized in that: The target recognition method includes the following steps: 1) Identify the target to be identified and simultaneously acquire the image dataset of the target; 2) Construct an overall target recognition and localization network, which includes a backbone feature extraction network and an enhanced feature extraction network DFPNet placed after the backbone feature extraction network; 3) The overall target recognition and localization network is pre-trained using the standard dataset MSCOCO to generate pre-trained weights; 4) Based on the image dataset of the target to be identified obtained in step 1) and the pre-trained weights generated in step 3), train the enhanced feature extraction network DFPNet in step 2) to generate the final network prediction weights; 5) Use the final network prediction weights generated in step 4) to identify and locate the target to be identified; In step 2), the backbone feature extraction network is the Swin-Transformer; the enhanced feature extraction network DFPNet is constructed based on PANet. The specific implementation method of step 3) is as follows: 3.1) The standard dataset MSCOCO provides input images to the backbone feature extraction network; 3.2) The backbone feature extraction network extracts features from the input image in step 3.1) and outputs feature maps of three different feature extraction stages to the enhanced feature extraction network DFPNet; 3.3) Further feature extraction is performed on the feature maps of the three different feature extraction stages input in step 3.2) by strengthening the feature extraction network DFPNet, and the target location regression and target category classification are completed in the last variable parameter convolutional layer until the pre-training is completed; 3.4) Generate pre-training weights based on the pre-training results; The specific implementation method of step 4) is as follows: 4.1) Adjust the weight parameters of the overall target recognition and localization network based on the pre-trained weights obtained in step 3); 4.2) Using the image dataset of the target to be identified obtained in step 1) as the input object, provide the input image to the backbone feature extraction network; 4.3) The backbone feature extraction network extracts features from the input image in step 4.2) and outputs feature maps of three different feature extraction stages to the enhanced feature extraction network DFPNet; 4.4) Further feature extraction is performed on the feature maps of the three different feature extraction stages input in step 4.3) by strengthening the feature extraction network DFPNet, and the target location regression and target category classification are completed in the last variable parameter convolutional layer until the training of the overall target recognition and localization network is completed. 4.5) Generate the final network prediction weights based on the training results; The specific implementation method of step 4.4) is as follows: 4.4) Further feature extraction is performed on the feature maps from the three different feature extraction stages input in step 4.3) by strengthening the feature extraction network DFPNet. Subsequently, the EIOU loss function is introduced during training to accelerate the network's training convergence and guide the updating of the target recognition and localization network weight parameters towards higher localization accuracy, until the training of the overall target recognition and localization network is completed. The calculation form of IOU in the EIOU loss function is as follows: ， The threshold for IOU is 0.

75.

2. The target recognition method according to claim 1, characterized in that: The image dataset of the target to be identified in step 1) is a collection of images of the target to be identified taken at different locations, angles, quantities, sizes, and light intensities, and labeled using annotation tools.

3. The target recognition method according to claim 1, characterized in that: The specific implementation of step 3.2) is as follows: the Swin-Transformer is used to extract features from the input image in step 3.1), and at the same time, based on stage2, stage3 and stage4 in the Swin-Transformer, feature maps of three different feature extraction stages are output to the enhanced feature extraction network DFPNet.

4. The target recognition method according to claim 1, characterized in that: The specific implementation of step 3.3) is as follows: a bottleneck structure of the backbone feature extraction network is added to the enhanced feature extraction network DFPNet. The enhanced feature extraction network DFPNet further extracts features from the feature maps of the three different feature extraction stages by fusing two paths, bottom-up and top-down. The target location regression and target category classification are completed in the last variable parameter convolutional layer until the pre-training is completed.

Images