Target recognition model optimization method and device and computer readable storage medium

Abstract

The invention discloses a target recognition model optimization method and device and a computer readable storage medium, and the method comprises the steps: obtaining a first training sample set which comprises a plurality of sample images and first label information, the first label information reflects whether any target object exists in the corresponding sample image or not and reflects whether various preset types of target objects exist in the corresponding sample image or not; a first training sample set is adopted to train a target recognition model, the target recognition model comprises a feature extraction backbone network and a detection head, the feature extraction backbone network comprises a first convolution layer, a residual layer and a second convolution layer which are connected in sequence, the first convolution layer is used for receiving an image of a first preset size, and the residual layer is used for receiving an image of a second preset size; and the second convolutional layer is used for outputting the feature map of the second preset size to the detection head. According to the invention, while the accuracy of low-computing-power equipment target identification in a traffic target identification scene is ensured, the identification efficiency of low-computing-power equipment target identification is improved.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a method and device for optimizing a target recognition model and a computer-readable storage medium. Background Art

[0002] Due to its efficient performance, the target detection model based on deep learning is being used more and more widely in all aspects of life. Traffic target recognition scenario is an application scenario of target detection in road traffic. For example, target detection in autonomous driving can collect real-time environmental information for vehicles to ensure safety and provide correct planning decisions. At present, the target detection model based on deep learning has been applied in traffic target recognition scenarios. However, there is a lack of target recognition model network structures specially designed for low-computing power devices, which leads to low recognition efficiency of low-computing power devices when performing target recognition, which in turn affects the application effect of traffic target recognition, such as affecting the vehicle's timely autonomous driving planning decisions. Summary of the invention

[0003] The main purpose of this application is to provide a target recognition model optimization method, device and computer-readable storage medium, aiming to provide an optimization method for target recognition models in the field of autonomous driving, so as to ensure the accuracy of target recognition of low-computing power devices in traffic target recognition scenarios while improving the recognition efficiency of low-computing power devices.

[0004] To achieve the above object, the present application provides a target recognition model optimization method, which comprises the following steps:

[0005] Acquire a first training sample set, wherein the first training sample set includes a plurality of sample images collected in a traffic target recognition scenario and first label information corresponding to the plurality of sample images, respectively, the first label information reflecting whether there are any target objects in the corresponding sample images and reflecting whether there are various preset types of target objects in the corresponding sample images;

[0006] The target recognition model is trained using the first training sample set to optimize the target recognition model, wherein the target recognition model includes a feature extraction backbone network and a detection head, the feature extraction backbone network includes a first convolutional layer, a residual layer, and a second convolutional layer connected in sequence, the first convolutional layer is used to receive an image of a first preset size, the second convolutional layer is used to output a feature map of a second preset size to the detection head, the detection head is used to perform classification regression according to the feature map of the second preset size to obtain a target recognition result of the image of the first preset size, the target recognition result reflects whether there is any target object in the image of the first preset size, and reflects whether there are targets of the various preset types in the image of the first preset size.

[0007] To achieve the above-mentioned objectives, the present application also provides a target recognition model optimization device, which includes: a memory, a processor, and a target recognition model optimization program stored in the memory and executable on the processor, and the target recognition model optimization program, when executed by the processor, implements the steps of the target recognition model optimization method described above.

[0008] To achieve the above objectives, the present application also proposes a computer-readable storage medium, on which a target recognition model optimization program is stored. When the target recognition model optimization program is executed by a processor, the steps of the target recognition model optimization method described above are implemented.

[0009] In the embodiment of the present application, in view of the fact that the targets to be identified in traffic scenes are mainly vehicles and the size of vehicles is relatively fixed, a network structure of a target recognition model is specifically proposed, including a feature extraction backbone network and a detection head. The feature extraction backbone network includes a first convolutional layer, a residual layer, and a second convolutional layer connected in sequence. The first convolutional layer is used to receive an image of a first preset size, and the second convolutional layer is used to output a feature map of a second preset size to the detection head. The detection head is used to perform classification and regression based on the feature map to obtain a target recognition result of the input image. Compared with a complex network that extracts feature maps of multiple sizes, the network structure proposed in the embodiment of the present application extracts a feature map of one size, which is aimed at targets of relatively fixed size to be identified in traffic scenes, thereby ensuring the accuracy of target recognition and reducing the scale of the network model, thereby improving the efficiency of target recognition for low-computing-power devices. In view of the fact that low-computing-power devices in traffic scenes only need to identify whether the result contains a target, the target recognition result output by the network structure is set to reflect whether there is any target object in the input image and whether there are various preset types of targets in the input image. This makes the output result meet the target recognition needs of low-computing-power devices in traffic scenes, and can reduce the complexity of the network model, thereby improving the efficiency of target recognition of low-computing-power devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 This is a flow chart of an embodiment of a method for optimizing a target recognition model of the present application;

[0011] Figure 2 A schematic diagram of the structure comparison between a backbone network and a feature extraction backbone network in YOLOv3 involved in an embodiment of the present application;

[0012] Figure 3 A schematic diagram of the structure of a cyclic generative adversarial network involved in an embodiment of the present application;

[0013] Figure 4 A schematic diagram of the structure of a target recognition model optimization device involved in an embodiment of the present application;

[0014] Figure 5 A schematic diagram of a computer-readable storage medium involved in an embodiment of the present application.

[0015] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0016] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0017] Reference Figure 1, Figure 1 This is a flow chart of the first embodiment of the target recognition model optimization method of the present application. The embodiment of the present application provides an embodiment of the target recognition model optimization method. It should be noted that although the logical order is shown in the flow chart, in some cases, the steps shown or described may be performed in an order different from that here. In this embodiment, there is no restriction on the execution subject of the target recognition model optimization method. For example, it can be a personal computer, a server, a smart phone, a vehicle-mounted device, a monitoring device and other devices. For the sake of ease of description, the following description of each embodiment by the execution subject is omitted. In this embodiment, the target recognition model optimization method includes:

[0018] Step S10, obtaining a first training sample set, wherein the first training sample set includes multiple sample images collected in a traffic target recognition scenario and first label information corresponding to the multiple sample images respectively, and the first label information reflects whether there are any target objects in the corresponding sample images and whether there are various preset types of target objects in the corresponding sample images.

[0019] The training sample set used to train the target recognition model is referred to as the first training sample set for distinction. The method for obtaining the first training sample set is not limited in this embodiment. For example, the first training sample set uploaded by the user can be obtained. The traffic target recognition scenario refers to a scenario in which target recognition is required in road traffic. The target object can be a target object that needs to be identified in road traffic; for example, in an autonomous driving scenario, an autonomous driving vehicle needs to capture images and identify targets in the images to assist the autonomous driving vehicle in making planning decisions; for another example, in a traffic monitoring scenario, monitoring equipment installed on the road needs to capture images and identify targets in the images to control vehicles on the road based on the recognition results.

[0020] The first training sample set may include multiple sample images collected in a traffic target recognition scenario and label information corresponding to the multiple sample images (hereinafter referred to as first label information for distinction). For example, the multiple sample images collected in an autonomous driving scenario may be images collected by a vehicle-mounted image acquisition system, and the multiple sample images collected in a traffic monitoring scenario may be images collected by a monitoring device.

[0021] According to the needs of the application scenario, the type of target to be identified can be set in advance (called preset type for distinction), and there is at least one preset type. The first label information corresponding to the sample image reflects whether there is any target in the sample image, and reflects whether there are various preset types of targets in the corresponding sample image. In a feasible implementation, the first label information includes two kinds of information, one of which reflects whether there is any target in the sample image, for example, 0 indicates that there is no target, and 1 indicates that there is a target. This information can also be called the confidence of the sample image; the other information reflects whether there are various preset types of targets in the sample image. For example, various preset types correspond to a value of 0 or 1, respectively, 0 indicates that there is no target of the preset type, and 1 indicates that there is a target of the preset type. For example, assuming that a preset type is set, the first label information may include XY, X and Y may take the values ​​of 0 and 1 respectively, and XY has four combinations: 00, 01, 10, and 11; wherein X=1 indicates that any target object exists in the sample image, and X=0 indicates that no target object exists in the sample image; Y=1 indicates that a target object of the preset type exists in the sample image, and Y=0 indicates that no target object of the preset type exists in the sample image.

[0022] Step S20, using the first training sample set to train the target recognition model to optimize the target recognition model, wherein the target recognition model includes a feature extraction backbone network and a detection head, the feature extraction backbone network includes a first convolutional layer, a residual layer, and a second convolutional layer connected in sequence, the first convolutional layer is used to receive an image of a first preset size, the second convolutional layer is used to output a feature map of a second preset size to the detection head, the detection head is used to perform classification regression according to the feature map of the second preset size to obtain a target recognition result of the image of the first preset size, the target recognition result reflects whether there is any target object in the image of the first preset size, and reflects whether there are targets of the various preset types in the image of the first preset size.

[0023] The model structure of the target recognition model is proposed in this embodiment for low-computing power devices in traffic target recognition scenarios, in order to improve the target recognition efficiency of low-computing power devices. Optimizing the target recognition model refers to optimizing the model parameters in the target recognition model. In this embodiment, the model structure of the target recognition model includes a feature extraction backbone network and a detection head, and the feature extraction backbone network includes a first convolutional layer, a residual layer, and a second convolutional layer connected in sequence. The specific structure of the first convolutional layer, the residual layer, and the second convolutional layer is not limited in this embodiment and can be set as needed. In one feasible implementation, the branch network used to extract the maximum size feature map in the backbone network of YOLOv3 can be selected, and the network remaining after the splicing layer is removed from the branch network is used as the feature extraction backbone network. Figure 2 As shown, the corresponding relationship between the backbone network of YOLOv3 and each network layer in the feature extraction backbone network of this embodiment is shown, and y1, y2 and y3 are feature maps of different sizes. It should be noted that Figure 2 The internal structure of each network module is not explained in this embodiment. For details, please refer to the introduction of YOLOv3 in the relevant technology. In one feasible implementation, in order to speed up the training process, the backbone network of YOLOv3 is improved, and the calculation of three feature maps is omitted in the optimization stage to enhance the small target recognition performance. Only one layer of features is used for classification and prediction. Compared with the original Darknet, there are 28 fewer convolutional layers. The improved backbone network has 25 convolutional layers. Finally, the backbone network is simplified from outputting three 75-channel feature layers to a 21-channel feature layer, 21 is composed of 20+1, 20 represents 20 preset types, and 1 represents confidence. In addition, for the image format in the training sample set, the input vector can be set to 416×416.

[0024] The first convolution layer is used to receive an image of a first preset size, and the second convolution layer is used to output a feature map of a second preset size to the detection head. The first preset size and the second preset size can be set as needed, and the second preset size is smaller than the first preset size. The residual layer is between the first convolution layer and the second convolution layer, and is used to receive the result output by the first convolution layer, perform residual processing on the result output by the first convolution layer, and output the processed result to the second convolution layer. The second convolution layer is used to perform convolution processing on the result output by the residual layer, and output a feature map of the second preset size to the detection head.

[0025] The detection head is used to perform classification regression based on the feature map to obtain a target recognition result of an image of a first preset size. The target recognition result reflects whether there is any target in the image of the first preset size, and reflects whether there are various preset types of targets in the image of the first preset size. In a feasible implementation, the target recognition result of the image of the first preset size includes two results, one of which reflects whether there is any target in the image, for example, 0 indicates that there is no target, and 1 indicates that there is a target, and this result can also be called the confidence prediction result of the image; the other result reflects whether there are various preset types of targets in the image, for example, various preset types correspond to a value of 0 or 1, 0 indicates that there is no target of the preset type, and 1 indicates that there is a target of the preset type. For example, assuming that two preset types are set, the target recognition result may include XY1Y2, X, Y1 and Y2 may take the values ​​of 0 and 1 respectively, and XY1Y2 has 8 combinations: 000, 001, 010, 011, 100, 101, 110, 111; wherein, X=1 indicates that any target exists in the image of the first preset size, and X=0 indicates that no target exists in the image of the first preset size; Y1=1 indicates that a target of the first preset type exists in the image of the first preset size, and Y1=0 indicates that a target of the first preset type does not exist in the image of the first preset size; Y2=1 indicates that a target of the second preset type exists in the image of the first preset size, and Y2=0 indicates that a target of the second preset type does not exist in the image of the first preset size.

[0026] It is understandable that when a sample image is input into a target recognition model for target recognition and a target recognition result corresponding to the sample image is obtained, the target recognition result represents the result predicted by the target recognition model, while the first label information corresponding to the sample image represents the actual situation of the target object in the sample image. There will be an error between the predicted result and the actual situation, and the size of the error reflects the accuracy of the target recognition model in target recognition to a certain extent. In this embodiment, the target recognition model is optimized by using the first training sample set, with the aim of improving the accuracy of the target recognition model in target recognition.

[0027] In this embodiment, the specific implementation method of optimizing the target recognition model using the first training sample set is not limited, and a suitable optimization method, such as a gradient descent algorithm, can be selected as needed.

[0028] In one feasible implementation, in the process of optimizing the target recognition model, a learning rate annealing algorithm can be used, that is, the learning rate is adjusted after each batch of training. Specifically, it increases linearly when it increases, and decreases according to the cosine function when it decreases. This algorithm can avoid falling into a local minimum.

[0029] In one feasible implementation, in order to avoid overfitting and encourage the target recognition model to learn more generalizable features, DropBlock can be used to process the target recognition model during the optimization of the target recognition model. DropBlock is a regularization technique used in the forward propagation during the training process. In each forward propagation, DropBlock randomly selects some blocks in the feature map and sets them to zero. This prevents neurons from being activated together in a block, thereby further reducing redundant information between neurons. It should be noted that DropBlock does not perform any operations in the back propagation, it is only applied to the feature map in the forward propagation.

[0030] In the embodiment of the present application, in view of the fact that the targets to be identified in traffic scenes are mainly vehicles and the size of vehicles is relatively fixed, a network structure of a target recognition model is specifically proposed, including a feature extraction backbone network and a detection head. The feature extraction backbone network includes a first convolutional layer, a residual layer, and a second convolutional layer connected in sequence. The first convolutional layer is used to receive an image of a first preset size, and the second convolutional layer is used to output a feature map of a second preset size to the detection head. The detection head is used to perform classification and regression based on the feature map to obtain a target recognition result of the input image. Compared with a complex network that extracts feature maps of multiple sizes, the network structure proposed in the embodiment of the present application extracts a feature map of one size, which is aimed at targets of relatively fixed size to be identified in traffic scenes, thereby ensuring the accuracy of target recognition and reducing the scale of the network model, thereby improving the efficiency of target recognition for low-computing-power devices. In view of the fact that low-computing-power devices in traffic scenes only need to identify whether the result contains a target, the target recognition result output by the network structure is set to reflect whether there is any target object in the input image and whether there are various preset types of targets in the input image. This makes the output result meet the target recognition needs of low-computing-power devices in traffic scenes, and can reduce the complexity of the network model, thereby improving the efficiency of target recognition of low-computing-power devices.

[0031] Based on the above first embodiment, a second embodiment of the target recognition model optimization method of the present application is proposed. In this embodiment, step S20 includes:

[0032] Step S201: Input the plurality of sample images into the target recognition model for recognition, and obtain the target recognition results corresponding to the plurality of sample images.

[0033] In this embodiment, a feasible implementation method of training a target recognition model using a first training sample set is proposed. Multiple sample images in the first training sample set can be respectively input into the target recognition model for recognition, and target recognition results corresponding to the multiple sample images can be obtained. It can be understood that the detection head of the target recognition model is configured to perform classification regression based on the feature map to obtain a target recognition result of an image of a first preset size, and the target recognition result reflects whether there are any target objects in the image of the first preset size, and reflects whether there are various preset types of target objects in the image of the first preset size. Therefore, the target recognition result obtained by inputting the sample image into the target recognition model for recognition can reflect whether there are any target objects in the sample image, and reflect whether there are various preset types of target objects in the sample image.

[0034] It should be noted that the size of the sample images in the first training sample set may be the first preset size or may not be the first preset size. When the size of the sample images is not the first preset size, the sample images may be converted into images of the first preset size by size conversion, image stitching, etc., and then input into the target recognition model for recognition.

[0035] Step S202: Calculate target confidence loss and target category loss according to the first label information and the target recognition results respectively corresponding to the plurality of sample images.

[0036] Each sample image corresponds to the first label information and the target recognition result. According to the first label information and the target recognition result corresponding to each sample image, the target confidence loss and the target category loss can be calculated. The target confidence loss reflects the error between the result of the target recognition result reflecting whether there is any target in the sample image and the information in the first label information reflecting whether there is any target in the sample image. The target category loss reflects the error between the result of the target recognition result reflecting whether there are various preset types of targets in the sample image and the information in the first label information reflecting whether there are various preset types of targets in the sample image.

[0037] In this embodiment, there is no restriction on the method of calculating the target confidence loss and the target category loss, and the loss calculation method can be selected as needed. For example, the target confidence loss can be calculated using the following formula:

[0038]

[0039] Among them, L conf represents the target confidence loss, o i ∈{0,1} indicates whether the first label information of the i-th sample image contains any target object. The target recognition result of the i-th sample image indicates whether the sample image contains any target object.

[0040] The target category loss can be calculated using the following formula:

[0041]

[0042] Among them, L cla represents the target category loss, O ij ∈{0,1} indicates that the first label information of the i-th sample image reflects whether the sample image contains the j-th preset type of target object. The target recognition result of the i-th sample image indicates whether the sample image contains a target of the j-th preset type.

[0043] Step S203: Calculate a first total loss of the target recognition model according to the target confidence loss and the target category loss, and optimize the target recognition model according to the first total loss.

[0044] The total loss of the target recognition model is calculated based on the target confidence loss and the target category loss (hereinafter referred to as the first total loss for distinction). In a specific implementation, the target confidence loss and the target category loss can be added to obtain the first total loss; or, considering the different effects of the target confidence loss and the target category loss on the target recognition model, the weights corresponding to the target confidence loss and the target category loss can be set respectively, and the first total loss can be obtained by weighted summation. For example, the first total loss can be calculated according to the following formula:

[0045] L(O,o,C,c)=λ1L conf (o,c)+λ2L cla (O,C)

[0046] Among them, λ1 and λ2 represent the weights corresponding to the target confidence loss and target category loss, respectively, which can be set as needed.

[0047] In one feasible implementation, step S202 includes:

[0048] Step S2021, smoothing the first label information respectively corresponding to the plurality of sample images, and calculating the target confidence loss and the target category loss according to the first label information after the smoothing of the plurality of sample images and the target recognition result.

[0049] In this embodiment, the first label information can be smoothed, and then the target confidence loss and target category loss are calculated based on the smoothed first label information and target recognition results corresponding to each sample image. The purpose of smoothing the first label information is to avoid the problem that the target recognition model is prone to overfitting when the number of sample images in the first training sample set is small. By smoothing the label information, the generalization performance of the target recognition model can be increased, and the anti-noise ability of the target recognition model can be improved. In this embodiment, the specific implementation method of smoothing the first label information is not limited, and it can be set as needed. For example, in a feasible implementation, the smoothing process can be performed in the following manner:

[0050]

[0051] ε can be set as needed, for example, to 0.005, that is, the original label information 0 and 1 are adjusted to 0.005 and 0.995, which penalizes the classification accuracy and prevents overfitting.

[0052] Based on the above first and / or second embodiments, a third embodiment of the target recognition model optimization method of the present application is proposed. In this embodiment, step S10 includes:

[0053] Step S101, selecting at least one target sample image from a preset test sample set, wherein the preset test sample set includes multiple sample images collected in a traffic target recognition scenario and the first label information corresponding to the multiple sample images respectively, and the target recognition result obtained by inputting the target sample image into the target recognition model for recognition is inconsistent with the first label information of the target sample image.

[0054] In this embodiment, a feasible implementation method for obtaining the first training sample set is proposed to further improve the optimization effect of the target recognition model, improve the recognition accuracy of the target recognition model, and thus improve the application effect of traffic target recognition.

[0055] A test sample set (hereinafter referred to as a preset test sample set) may be set in advance. The preset test sample set may include multiple sample images and label information corresponding to the multiple sample images. Since the label information reflects whether there is any target object in the sample image and whether there are various preset types of targets in the sample image, the label information is also referred to as the first label information, as it is the same as the first label information.

[0056] The preset test sample set is used to test the target recognition model. By inputting each sample image in the preset test sample set into the target recognition model for recognition, the target recognition results corresponding to each sample image can be obtained, and the target recognition results of the sample images are compared with the first label information of the sample images. If the target recognition results are inconsistent with the first label information, it means that the target recognition results of the target recognition model for the sample image are incorrect. If the target recognition results are consistent with the first label information, it means that the target recognition results of the target recognition model for the sample image are correct.

[0057] In this embodiment, for each sample image in the preset test sample set, if any sample image is input into the target recognition model for target recognition and the target recognition result obtained is inconsistent with the first label information corresponding to the sample image, then the sample image can be considered as a sample image that is incorrectly recognized by the target recognition model, and the sample image is used as the target sample image. According to this method, at least one target sample image can be selected from the preset test sample set.

[0058] It should be noted that, in a specific implementation, if no target sample image can be selected from the preset test sample set or the selected target sample image is smaller than a certain threshold, it means that the current target recognition model can always recognize each sample image in the preset test sample set and obtain correct results. In this case, there is no need to optimize the target recognition model; and if the target sample image selected from the preset test sample set exceeds a certain threshold, it means that the recognition accuracy of the current target recognition model is low. In this case, the selected target sample images can be used to construct the first training sample set, and then the sample images that are incorrectly recognized by the target recognition model can be used to specifically train the target recognition model to improve the recognition accuracy of the target recognition model.

[0059] Step S102: construct the first training sample set according to each of the target sample images.

[0060] After selecting the target sample images from the preset test sample set, a first training sample set can be constructed based on each target sample image. In one feasible implementation, each target sample image can be combined into a first training sample set. In another feasible implementation, each target sample image can be sample expanded to obtain an expanded sample image, and the expanded sample image and the target sample image can be combined into a first training sample set to expand the number of the first training sample set, thereby improving the training effect of the target recognition model. In this embodiment, the specific implementation method of sample expansion is not limited.

[0061] In a feasible implementation, the target recognition model can be trained multiple times. After each training, the trained target recognition model is used to recognize each sample image in the preset test sample set, and then the target sample image is selected from the preset test sample set, and the target sample image is used to construct the first training sample set, and the target recognition model is trained again using the first training sample set, and the target recognition model is iterated in this way, and the target recognition model is trained multiple times until the preset stop condition is met and the training is stopped. The stop condition can be set as needed and is not limited here.

[0062] Due to the limitations of the deep learning network itself, if there is not enough data set for training, and if it is used in a complex road environment where various conditions are difficult to predict, its detection accuracy cannot meet the application requirements in the traffic target recognition scenario, or it will always encounter detection failures. In this embodiment, by using sample images with target recognition model recognition errors to specifically train the target recognition model, the recognition accuracy of the target recognition model is improved, so as to improve the target recognition model's ability to cope with complex road environments.

[0063] Based on the third embodiment above, a fourth embodiment of the target recognition model optimization method of the present application is proposed. In this embodiment, step S102 includes:

[0064] Step S1021, input each of the target sample images into a pre-trained recurrent generative adversarial network, and generate different-perspective images corresponding to each of the target sample images, wherein the different-perspective images and the corresponding target sample images contain the same target object under different perspectives, and the recurrent generative adversarial network is trained using a second training data set, and the second training data set includes multiple groups of images, each group of images includes two images, and the two images contain the same target object under different perspectives.

[0065] In this embodiment, a feasible implementation method for sample expansion of a target sample image is proposed. The target sample image is expanded by generating a different-viewing angle image corresponding to the target sample image. The different-viewing angle image is an image of the same target object under different viewing angles than the target sample image. For example, if the target sample image contains a vehicle from a frontal perspective, the generated different-viewing angle image may be the vehicle from a side perspective.

[0066] In a specific implementation, there are many ways to generate different-perspective images corresponding to the target sample image. In this embodiment, it is proposed to use a pre-trained cyclic generative adversarial network to generate different-perspective images. The cyclic generative adversarial network is trained using a second training set, which includes multiple groups of images, each group of images includes two images, and the two images contain the same target object under different perspectives, so that the trained cyclic generative adversarial network can generate a different-perspective image corresponding to a sample image. In this embodiment, there is no restriction on the training method of using multiple groups of images to train the cyclic generative adversarial network, which can be set as needed.

[0067] Step S1022: construct the first training sample set according to each of the target sample images and each of the different-viewing angle images.

[0068] After generating different-viewing angle images corresponding to each target sample image, a first training sample set can be constructed based on each target sample image and each different-viewing angle image. In a specific implementation, each target sample image and each different-viewing angle image can be directly combined into the first training sample set, or each target sample image and each different-viewing angle image can be expanded or processed in other ways to form the first training sample set.

[0069] In this embodiment, in view of the situation in which the collected images mostly contain the same target object from different perspectives in the traffic target recognition scenario, it is proposed to expand the number of samples by generating different-perspective images of the target sample images, so that the trained target recognition model has better recognition accuracy for the targets in the traffic target recognition scenario.

[0070] In a feasible implementation manner, before step S1021, the method further includes:

[0071] Step S30, inputting the multiple groups of images into the recurrent generative adversarial network to be trained, and calculating the second total loss of the recurrent generative adversarial network to be trained based on the output results and the preset second label information corresponding to the multiple groups of images.

[0072] In this embodiment, a feasible training method for a recurrent generative adversarial network is proposed. The recurrent generative adversarial network in the training phase is referred to as a recurrent generative adversarial network to be trained for distinction.

[0073] The recurrent generative adversarial network to be trained needs to undergo multiple rounds of training. After the set stop condition is met, the training is stopped, and the finally optimized recurrent generative adversarial network to be trained is used as the recurrent generative adversarial network that has completed the training. The stop condition can be preset as needed and is not limited in this embodiment.

[0074] Take one round of training as an example for explanation. In one round of training, multiple groups of images in the second training data set are input into the recurrent generative adversarial network to be trained, and the recurrent generative adversarial network processes the multiple groups of images and outputs the results. Based on the output results and the preset label information corresponding to the multiple groups of images (referred to as the second label information for distinction), the total loss of the recurrent generative adversarial network to be trained (hereinafter referred to as the second total loss for distinction) can be calculated. The second total loss is used to measure the accuracy of the different-perspective images generated by the recurrent generative adversarial network to be trained. There are many ways to calculate the second total loss, which is not limited in this embodiment.

[0075] Step S40, optimizing the recurrent generative adversarial network to be trained according to the second total loss to complete a round of training for the recurrent generative adversarial network to be trained, and obtaining the trained recurrent generative adversarial network after multiple rounds of training for the recurrent generative adversarial network to be trained.

[0076] Optimizing the recurrent generative adversarial network to be trained according to the second total loss can specifically include calculating the gradient values ​​corresponding to each network parameter in the recurrent generative adversarial network to be trained according to the second total loss, updating each network parameter according to the gradient value, and the recurrent generative adversarial network after updating the network parameters is the optimized recurrent generative adversarial network. At this point, one round of training is completed, and the next round of training is based on the recurrent generative adversarial network after this round of optimization, and the operations of steps S30 to S40 are performed again.

[0077] In a feasible implementation, the cyclic generative adversarial network may include two generative adversarial networks (GANs), referred to as a first generative adversarial network and a second generative adversarial network, the first generative adversarial network may include a generative network (referred to as a first generative network for distinction) and a discriminative network (referred to as a first discriminative network for distinction), and the second generative adversarial network may include a generative network (referred to as a second generative network for distinction) and a discriminative network (referred to as a second discriminative network for distinction). Figure 3 As shown, a structural schematic diagram of a cyclic generative adversarial network is given, where G represents the first generative network, D represents the first discriminative network, F represents the second generative network, D represents the second discriminative network, X represents the first perspective, and Y represents the second perspective.

[0078] In one feasible implementation, the step S30 includes:

[0079] Step S301, input the multiple groups of images into the cyclic generative adversarial network to be trained, and based on the output results and the preset second label information corresponding to the multiple groups of images, calculate the first adversarial loss of the first generative adversarial network, the second adversarial loss of the second generative adversarial network, and the cycle consistency loss between the first generative network and the second generative network.

[0080] In this embodiment, a feasible implementation method for calculating the second total loss is proposed. Multiple groups of images in the second training data set can be input into the cyclic generative adversarial network to be trained, and the training generative adversarial network is used to process and output the results. Based on the output results and the multiple groups of images corresponding to the preset second label information, the adversarial loss of the first generative adversarial network (hereinafter referred to as the first adversarial loss for distinction), the adversarial loss of the second generative adversarial network (hereinafter referred to as the second adversarial loss for distinction), and the cyclic consistency loss between the first generative network and the second generative network can be calculated.

[0081] Step S302: Calculate a second total loss of the cyclic generative adversarial network to be trained according to the first adversarial loss, the second adversarial loss, and the cycle consistency loss.

[0082] In a specific implementation, the first adversarial loss, the second adversarial loss, and the cycle consistency loss are added or weighted together to obtain a second total loss of the cycle generative adversarial network to be trained.

[0083] In a feasible implementation, a feasible improvement method for the first generative adversarial network is proposed. The first generative adversarial network may also include a network mapping layer (hereinafter referred to as the first network mapping layer) composed of several fully connected layers (FC) in series. The first network mapping layer is located before the first generative network and is used to perform network mapping on the image before the image is input into the first generative network. The function of the first network mapping layer is to control the style of the generated image. The image is input into the first network mapping layer, and an intermediate variable is obtained after passing through several fully connected layers. This process can better classify and generate data.

[0084] Accordingly, in a feasible implementation, a feasible improvement method for the second generative adversarial network is proposed. The second generative adversarial network may also include a network mapping layer (hereinafter referred to as the second network mapping layer) composed of several fully connected layers in series. The second network mapping layer is located before the second generative network and is used to perform network mapping on the image before the image is input into the second generative network. The function of the second network mapping layer is to control the style of the generated image. The image is input into the second network mapping layer, and an intermediate variable is obtained after passing through several fully connected layers. This process can better classify and generate data.

[0085] In one feasible implementation, the step S1022 includes:

[0086] Step S10221: Perform CutMix data enhancement processing on any two of the target sample images and the different-viewing angle images to obtain a data enhanced image.

[0087] In this embodiment, a feasible implementation method for constructing a first training sample set based on each target sample image and each different-viewing angle image is proposed. Any two images among each target sample image and each different-viewing angle image can be subjected to CutMix data enhancement processing, and the obtained image is called a data-enhanced image for distinction. CutMix is ​​to combine two images into one image through a series of processing. The specific operation is to randomly cut out a part of image A, and then fill the corresponding position of image B into the cut area to form a new image. When calculating the loss, the loss of the labels of image A and image B is assigned corresponding weights according to the size of the cropping box. It can be understood that the data-enhanced image contains information from both images.

[0088] Step S10222: randomly crop the multiple data-enhanced images that have undergone the CutMix data enhancement process and then splice them into one image to obtain a spliced ​​image.

[0089] According to the above method, two images are obtained from each target sample image and each different-viewing angle image for data enhancement processing each time. This operation is repeated multiple times to obtain multiple data-enhanced images. Multiple data-enhanced images can be randomly cropped and then spliced ​​into one image to obtain a spliced ​​image. It can be understood that since each data-enhanced image contains information of two images, after randomly cropping and splicing n data-enhanced images, the spliced ​​image obtained can contain information of up to 2n images.

[0090] Step S10223: construct the first training sample set according to the plurality of stitched images obtained based on the target sample images and the different-viewing angle images.

[0091] According to the above method, multiple data augmented images can be obtained each time for random cropping and splicing, and multiple spliced ​​images can be obtained by performing this operation multiple times. The first training sample set can be constructed based on the multiple spliced ​​images. Since the background of the images are all real traffic conditions, the dividing line of the image splicing will not affect the target detection, but since a spliced ​​image contains information of multiple images, it can achieve the effect of processing multiple images at one time during BN (Batch Normalization) processing, thereby speeding up the running speed of single-core processing and reducing the computing power required for the model.

[0092] Based on the fourth embodiment, a fifth embodiment of the target recognition model optimization method of the present application is proposed. In this embodiment, in step S301, the multiple groups of images are input into the recurrent generative adversarial network to be trained, and based on the output results and the preset second label information corresponding to the multiple groups of images, the step of calculating the first adversarial loss of the first generative adversarial network includes:

[0093] Step S3011: For any target image group among the multiple groups of images, a third image obtained by splicing the first image and the second image is input into the first discriminant network for discrimination to obtain a first discrimination result corresponding to the target image group, wherein the first image is an image with a first perspective in the target image group, and the second image is an image with a second perspective in the target image group.

[0094] In this embodiment, a feasible implementation method for calculating the first adversarial loss is proposed. An image group among multiple groups of images is used as an example for description, and the image group is referred to as a target image group for distinction. The perspectives of the target object in the two images in the target image group are referred to as a first perspective and a second perspective for distinction, and the image with the first perspective is referred to as a first image, and the image with the second perspective is referred to as a second image for distinction.

[0095] The first image and the second image can be spliced, and the resulting image is referred to as a third image. The splicing method is not limited in this embodiment. It can be understood that the third image contains the first image and the second image. The third image is input into the first discriminant network for discrimination, and the resulting discrimination result is referred to as the first discrimination result corresponding to the target image group. The first discrimination result can be a result that characterizes whether the first image is a real image and whether the first image and the second image are from the same group of images.

[0096] Step S3012: Calculate the first loss corresponding to the first generative adversarial network according to the first label values ​​and the first discrimination results respectively corresponding to the multiple groups of images, wherein the first label value corresponding to the target image group represents that the first image is a real image and represents that the first image and the second image are from the same group of images.

[0097] The second label information may include a first label value, and the first label value corresponding to the target image group indicates that the first image is a real image and indicates that the first image and the second image are from the same group of images.

[0098] After performing the above step S3011 on multiple groups of images respectively, the first discrimination results corresponding to the multiple groups of images can be obtained. According to the first label values ​​and the first discrimination results corresponding to the multiple groups of images respectively, a loss can be calculated, which is referred to as the first loss corresponding to the first generative adversarial network for distinction. The first loss represents the error between the first discrimination result and the first label value, and is used to constrain the discrimination accuracy of the first discriminant network in the first generative adversarial network, that is, the purpose is to enable the first discriminant network to accurately discriminate whether an image is a real image and to accurately discriminate whether two images are a group of images.

[0099] Step S3013: obtain a first adversarial loss of the first generative adversarial network according to the first loss calculation.

[0100] In a specific implementation, the first loss can be directly used as the first adversarial loss of the first generative adversarial network, or the first loss can be combined with other losses to calculate the first adversarial loss.

[0101] In one feasible implementation, the step S3013 includes:

[0102] Step S30131: input the first image into the first generation network for image generation to obtain a fourth image whose perspective is the second perspective, splice the fourth image with the second image and input them into the first discrimination network for discrimination to obtain a second discrimination result corresponding to the target image group.

[0103] In this embodiment, a feasible implementation method is proposed to combine the first loss with other losses to calculate the first adversarial loss, so as to further improve the training effect of the first generative adversarial network.

[0104] The first image can be input into the first generative network for image generation to obtain an image with a second perspective (referred to as the fourth image for distinction). It should be noted that it is not necessary for the perspective of the fourth image to be the second perspective. During the training process, the purpose is to enable the first generative network to accurately generate an image with a second perspective, that is, to make the generated fourth image closer to the second image.

[0105] The fourth image is concatenated with the second image and then input into the first discriminant network for discrimination. The result obtained is called the second discrimination result corresponding to the target image group. The second discrimination result is a result indicating whether the fourth image is a real image and whether the second image and the fourth image are from the same group of images.

[0106] Step S30132: Calculate the second loss corresponding to the first generative adversarial network according to the second label values ​​and the second discrimination results respectively corresponding to the multiple groups of images, wherein the second label value corresponding to the target image group represents that the fourth image is a real image and represents that the second image and the fourth image are from the same group of images.

[0107] The second label information may further include a second label value, and the second label value corresponding to the target image group indicates that the fourth image is a real image and indicates that the second image and the fourth image are from the same group of images.

[0108] After performing the above-mentioned step S30131 on multiple groups of images respectively, the second discrimination results corresponding to the multiple groups of images can be obtained. According to the second label values ​​and the second discrimination results corresponding to the multiple groups of images respectively, a loss can be calculated, which is referred to as the second loss corresponding to the first generative adversarial network for distinction. The second loss represents the error between the second discrimination result and the second label value, and is used to constrain the generation accuracy of the first generative network in the first generative adversarial network. That is, the purpose is to enable the first generative network to accurately generate different-perspective images of the first-perspective image in order to "fool" the first discriminant network.

[0109] Step S30133: Calculate a first adversarial loss of the first generative adversarial network according to the first loss and the second loss.

[0110] In a specific implementation, the first loss and the second loss can be directly added together, or weighted summed up as the first adversarial loss of the first generative adversarial network, or the first loss and the second loss can be combined with other losses to calculate the first adversarial loss.

[0111] In one feasible implementation, the step S30133 includes:

[0112] Step a, calculating the third loss corresponding to the first generative adversarial network according to the third label values ​​and the second discrimination results respectively corresponding to the multiple groups of images, wherein the third label value corresponding to the target image group represents that the fourth image is a generated image and represents that the second image and the fourth image are from different image groups.

[0113] In this embodiment, a feasible implementation method is proposed to calculate the first adversarial loss by combining the first loss and the second loss with other losses, so as to further improve the training effect of the first generative adversarial network.

[0114] The second label information may further include a third label value, and the third label value corresponding to the target image group indicates that the fourth image is a generated image and indicates that the second image and the fourth image are from different groups of images.

[0115] After performing the above step S30131 on multiple groups of images respectively, the second discrimination results corresponding to the multiple groups of images can be obtained. According to the third label values ​​and the second discrimination results corresponding to the multiple groups of images respectively, a loss can be calculated, which is referred to as the third loss corresponding to the first generative adversarial network for distinction. The third loss represents the error between the second discrimination result and the third label value, and is used to constrain the discrimination accuracy of the first discriminant network in the first generative adversarial network, that is, the purpose is to enable the first discriminant network to accurately discriminate whether an image is a generated image and to accurately discriminate whether two images are from different image groups.

[0116] Step b: calculating a first adversarial loss of the first generative adversarial network according to the first loss, the second loss and the third loss corresponding to the first generative network.

[0117] In a specific implementation, the first loss, the second loss, and the third loss can be directly added together, or weighted summed up as the first adversarial loss of the first generative adversarial network, or the first loss, the second loss, and the third loss can be combined with other losses to calculate the first adversarial loss. Since the first loss, the second loss, and the third loss constrain the discrimination accuracy of the first discriminant network and the generation accuracy of the first generative network, the first generative network can accurately generate an image of the second perspective based on the image of the first perspective under the confrontation with the first discriminant network with high discrimination accuracy, and the generated image and the original image contain the same target object, so that the trained cyclic generative adversarial network can generate a different-perspective image of the target sample image of the first perspective.

[0118] In one feasible implementation, the first adversarial loss may be calculated as follows:

[0119]

[0120] Among them, L GAN (G,D Y , X, Y) represents the first adversarial loss, G represents the first generative network, D Y represents the first discriminant network, X represents the first perspective, Y represents the second perspective, x represents the image of the first perspective, y represents the image of the second perspective, a, b and c represent the first label value, the second label value and the third label value respectively.

[0121] In a feasible implementation, a feasible implementation for calculating the first adversarial loss is proposed. Take one image group among multiple groups of images as an example for explanation, and the image group is referred to as a target image group for distinction. The perspectives of the target object in the two images in the target image group are referred to as a first perspective and a second perspective for distinction, and the image with the first perspective is referred to as a first image, and the image with the second perspective is referred to as a second image for distinction.

[0122] The first image and the second image can be spliced, and the resulting image is referred to as a third image. The splicing method is not limited in this embodiment. It can be understood that the third image contains the first image and the second image. The third image is input into the second discriminant network for discrimination, and the resulting discrimination result is referred to as a third discrimination result corresponding to the target image group. The third discrimination result can be a result that characterizes whether the first image is a real image and whether the first image and the second image are from the same group of images.

[0123] The second label information may include a first label value, and the first label value corresponding to the target image group indicates that the first image is a real image and indicates that the first image and the second image are from the same group of images.

[0124] After performing the above operations on multiple groups of images respectively, the third discrimination results corresponding to the multiple groups of images can be obtained. According to the first label values ​​and the third discrimination results corresponding to the multiple groups of images respectively, a loss can be calculated, which is referred to as the fourth loss corresponding to the second generative adversarial network for distinction. The fourth loss represents the error between the third discrimination result and the first label value, and is used to constrain the discrimination accuracy of the second discriminant network in the second generative adversarial network. That is, the purpose is to enable the second discriminant network to accurately discriminate whether an image is a real image and to accurately discriminate whether two images are a group of images.

[0125] In a specific implementation, the fourth loss can be directly used as the second adversarial loss of the second generative adversarial network, or the fourth loss can be combined with other losses to calculate the second adversarial loss.

[0126] In one feasible implementation, a feasible implementation is proposed for calculating the second adversarial loss by combining the fourth loss with other losses to further improve the training effect of the second generative adversarial network.

[0127] The second image can be input into the second generative network for image generation to obtain an image whose perspective is the first perspective (referred to as the fifth image for distinction). It should be noted that it is not necessary for the perspective of the fifth image to be the first perspective. During the training process, the purpose is to enable the second generative network to accurately generate an image of the first perspective, that is, to make the generated fifth image closer to the first image.

[0128] The fifth image is concatenated with the first image and then input into the second discriminant network for discrimination. The result obtained is called the fourth discrimination result corresponding to the target image group. The fourth discrimination result is a result indicating whether the fifth image is a real image and whether the fifth image and the first image are from the same group of images.

[0129] The second label information may further include a fourth label value, and the fourth label value corresponding to the target image group indicates that the fifth image is a real image and indicates that the fifth image and the first image are from the same group of images.

[0130] After performing the above operations on multiple groups of images respectively, the fourth discrimination results corresponding to the multiple groups of images can be obtained. According to the fourth label values ​​and the fourth discrimination results corresponding to the multiple groups of images respectively, a loss can be calculated, which is referred to as the fifth loss corresponding to the second generative adversarial network for distinction. The fifth loss represents the error between the fourth discrimination result and the fourth label value, and is used to constrain the generation accuracy of the second generative network in the second generative adversarial network. That is, the purpose is to enable the second generative network to accurately generate different-perspective images of the second-perspective image in order to "fool" the second discriminant network.

[0131] In a specific implementation, the fourth loss and the fifth loss can be directly added or weightedly summed as the second adversarial loss of the second generative adversarial network. Alternatively, the fourth loss and the fifth loss can be combined with other losses to calculate the second adversarial loss.

[0132] In one feasible implementation, a feasible implementation is proposed for calculating the second adversarial loss by combining the fourth loss and the fifth loss with other losses to further improve the training effect of the second generative adversarial network.

[0133] The second label information may further include a fifth label value, where the fifth label value corresponding to the target image group indicates that the fifth image is a generated image and indicates that the fifth image and the first image are from different groups of images.

[0134] After performing the above operations on multiple groups of images respectively, the fourth discrimination results corresponding to the multiple groups of images can be obtained. According to the fifth label values ​​and the fourth discrimination results corresponding to the multiple groups of images respectively, a loss can be calculated, which is referred to as the sixth loss corresponding to the second generative adversarial network for distinction. The sixth loss represents the error between the fourth discrimination result and the fifth label value, and is used to constrain the discrimination accuracy of the second discriminant network in the second generative adversarial network. That is, the purpose is to enable the second discriminant network to accurately discriminate whether an image is a generated image and to accurately discriminate whether two images are from different image groups.

[0135] In a specific implementation, the fourth loss, the fifth loss, and the sixth loss can be directly added together, or weighted summed up as the second adversarial loss of the second generative adversarial network, or the fourth loss, the fifth loss, and the sixth loss can be combined with other losses to calculate the first adversarial loss. Since the fourth loss, the fifth loss, and the sixth loss constrain the discrimination accuracy of the second discriminant network and the generation accuracy of the second generative network, the second generative network can accurately generate an image of the first perspective based on the image of the second perspective under the confrontation with the second discriminant network with high discrimination accuracy, and the generated image and the original image contain the same target object, so that the trained cyclic generative adversarial network can generate a different-perspective image of the target sample image of the second perspective.

[0136] In one feasible implementation, the second adversarial loss may be calculated as follows:

[0137]

[0138] Among them, L GAN (D,D X , Y, X) represents the second adversarial loss, F represents the second generation network, D X represents the second discriminant network, X represents the first perspective, Y represents the second perspective, x represents the image of the first perspective, y represents the image of the second perspective, a, d and e represent the first label value, the fourth label value and the fifth label value respectively.

[0139] In one feasible implementation, a feasible implementation of calculating the cycle consistency loss between the first generation network and the second generation network is proposed. The cycle consistency loss can be calculated in the following manner:

[0140]

[0141] Among them, L cyc (G, F) represents the cycle consistency loss, G represents the first generation network, F represents the first discriminative network, x represents the image of the first perspective, and y represents the image of the second perspective.

[0142] In addition, the embodiment of the present application also proposes a target recognition model optimization device, the target recognition model optimization device comprising:

[0143] An acquisition module, used to acquire a first training sample set, wherein the first training sample set includes a plurality of sample images collected in a traffic target recognition scenario and first label information corresponding to the plurality of sample images, respectively, wherein the first label information reflects whether there is any target object in the corresponding sample image and reflects whether there are various preset types of targets in the corresponding sample image;

[0144] A training module, used to train a target recognition model using the first training sample set to optimize the target recognition model, wherein the target recognition model includes a feature extraction backbone network and a detection head, the feature extraction backbone network includes a first convolutional layer, a residual layer, and a second convolutional layer connected in sequence, the first convolutional layer is used to receive an image of a first preset size, the second convolutional layer is used to output a feature map of a second preset size to the detection head, the detection head is used to perform classification regression based on the feature map of the second preset size to obtain a target recognition result of the image of the first preset size, the target recognition result reflects whether there is any target object in the image of the first preset size, and reflects whether there are targets of the various preset types in the image of the first preset size.

[0145] In one feasible implementation, a branch network for extracting a feature map of maximum size is selected from the backbone network of YOLOv3, and the network remaining after removing the splicing layer from the branch network is used as the feature extraction backbone network.

[0146] In one feasible implementation, the training module is further used for:

[0147] Inputting the plurality of sample images into the target recognition model for recognition respectively, and obtaining the target recognition results respectively corresponding to the plurality of sample images;

[0148] Calculating target confidence loss and target category loss according to the first label information and the target recognition results respectively corresponding to the plurality of sample images;

[0149] A first total loss of the target recognition model is calculated according to the target confidence loss and the target category loss, and the target recognition model is optimized according to the first total loss.

[0150] In one feasible implementation, the training module is further used for:

[0151] The first label information corresponding to the plurality of sample images is smoothed, and the target confidence loss and the target category loss are calculated according to the first label information after the smoothing of the plurality of sample images and the target recognition result.

[0152] In a feasible implementation manner, the acquisition module is further used for:

[0153] At least one target sample image is selected from a preset test sample set, wherein the preset test sample set includes a plurality of sample images collected in a traffic target recognition scenario and the first label information respectively corresponding to the plurality of sample images, and a target recognition result obtained by inputting the target sample image into the target recognition model for recognition is inconsistent with the first label information of the target sample image;

[0154] The first training sample set is constructed according to each of the target sample images.

[0155] In a feasible implementation manner, the acquisition module is further used for:

[0156] Inputting each of the target sample images into a pre-trained recurrent generative adversarial network, respectively, to generate different-perspective images corresponding to each of the target sample images, wherein the different-perspective images and the corresponding target sample images contain the same target object at different perspectives, and the recurrent generative adversarial network is trained using a second training data set, wherein the second training data set includes multiple groups of images, each group of images includes two images, and the two images contain the same target object at different perspectives;

[0157] The first training sample set is constructed according to each of the target sample images and each of the different-viewing angle images.

[0158] In one feasible implementation, the training module is further used for:

[0159] Inputting the multiple groups of images into the recurrent generative adversarial network to be trained, and calculating a second total loss of the recurrent generative adversarial network to be trained based on the output results and the preset second label information corresponding to the multiple groups of images respectively;

[0160] The recurrent generative adversarial network to be trained is optimized according to the second total loss to complete one round of training of the recurrent generative adversarial network to be trained. After multiple rounds of training of the recurrent generative adversarial network to be trained, the trained recurrent generative adversarial network is obtained.

[0161] In one feasible implementation, the cyclic generative adversarial network includes a first generative adversarial network and a second generative adversarial network, the first generative adversarial network includes a first generative network and a first discriminative network, and the second generative adversarial network includes a second generative network and a second discriminative network;

[0162] The training module is also used to:

[0163] Input the multiple groups of images into the cyclic generative adversarial network to be trained, and calculate the first adversarial loss of the first generative adversarial network, the second adversarial loss of the second generative adversarial network, and the cycle consistency loss between the first generative network and the second generative network based on the output results and the preset second label information corresponding to the multiple groups of images respectively;

[0164] A second total loss of the cycle generative adversarial network to be trained is calculated according to the first adversarial loss, the second adversarial loss, and the cycle consistency loss.

[0165] In one feasible implementation, the first generative adversarial network further includes a first network mapping layer formed by connecting a plurality of fully connected layers in series, the first network mapping layer being located before the first generative network and configured to perform network mapping on the image before inputting the image into the first generative network;

[0166] The second generative adversarial network also includes a second network mapping layer formed by connecting a plurality of fully connected layers in series. The second network mapping layer is located before the second generative network and is used to perform network mapping on the image before the image is input into the second generative network.

[0167] In one feasible implementation, the second label information includes a first label value; and the training module is further configured to:

[0168] For any target image group among the multiple groups of images, a third image obtained by splicing the first image and the second image is input into the first discriminant network for discrimination, so as to obtain a first discrimination result corresponding to the target image group, wherein the first image is an image of the target image group whose viewing angle is the first viewing angle, and the second image is an image of the target image group whose viewing angle is the second viewing angle;

[0169] Calculating a first loss corresponding to the first generative adversarial network according to the first label values ​​and the first discrimination results respectively corresponding to the multiple groups of images, wherein the first label value corresponding to the target image group indicates that the first image is a real image and that the first image and the second image are from the same group of images;

[0170] A first adversarial loss of the first generative adversarial network is obtained according to the first loss calculation.

[0171] In one feasible implementation, the second label information further includes a second label value, and the training module is further configured to:

[0172] Input the first image into the first generation network for image generation to obtain a fourth image with the second perspective, splice the fourth image with the second image and input the spliced ​​image into the first discrimination network for discrimination to obtain a second discrimination result corresponding to the target image group;

[0173] Calculating a second loss corresponding to the first generative adversarial network according to the second label values ​​and the second discrimination results respectively corresponding to the multiple groups of images, wherein the second label value corresponding to the target image group indicates that the fourth image is a real image and indicates that the second image and the fourth image are from the same group of images;

[0174] A first adversarial loss of the first generative adversarial network is calculated according to the first loss and the second loss.

[0175] In one feasible implementation, the training module is further used for:

[0176] Calculating a third loss corresponding to the first generative adversarial network according to the third label values ​​and the second discrimination results respectively corresponding to the multiple groups of images, wherein the third label value corresponding to the target image group indicates that the fourth image is a generated image and indicates that the second image and the fourth image are from different image groups;

[0177] The first adversarial loss of the first generative adversarial network is calculated according to the first loss, the second loss, and the third loss corresponding to the first generative network.

[0178] In a feasible implementation manner, the acquisition module is further used for:

[0179] Performing CutMix data enhancement processing on any two of the target sample images and the different-viewing angle images to obtain a data enhanced image;

[0180] The plurality of data-enhanced images processed by CutMix data enhancement are randomly cropped and then stitched into one image to obtain a stitched image;

[0181] The first training sample set is constructed according to a plurality of the stitched images obtained based on each of the target sample images and each of the different-viewing angle images.

[0182] The present application also provides a target recognition model optimization device, which includes: a memory, a processor, and a target recognition model optimization program stored in the memory and executable on the processor, wherein the target recognition model optimization program implements the steps of the target recognition model optimization method described in the above embodiments when executed by the processor. Figure 4As shown, the target recognition model optimization device may also include a network interface, a user interface and a communication bus. Among them, the communication bus is used to realize the connection and communication between these components. The user interface may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface may also include a standard wired interface and a wireless interface. The network interface may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory may be a high-speed RAM memory or a stable memory (non-volatile memory), such as a disk memory. The memory may optionally be a storage device independent of the aforementioned processor. As a computer storage medium, the memory may include an operating system, a network communication module, a user interface module, and a target recognition model optimization program. The operating system is a program that manages and controls the hardware and software resources of the device, and supports the operation of the target recognition model optimization program and other software or programs. Those skilled in the art will understand that Figure 4 The device structure shown in the figure does not constitute a limitation on the target recognition model optimization device, and may include more or fewer components than shown in the figure, or a combination of certain components, or a different arrangement of components.

[0183] Reference Figure 5 The present application also provides a computer-readable storage medium on which a target recognition model optimization program is stored. When the target recognition model optimization program is executed by a processor, the steps of the target recognition model optimization method described in the above embodiments are implemented. Figure 5 In a specific implementation, a memory as a computer-readable storage medium can be connected to communicate with the processor via a communication bus.

[0184] The various embodiments of the target recognition model optimization device, equipment and computer-readable storage medium of the present application can all refer to the various embodiments of the target recognition model optimization method of the present application, and will not be repeated here.

[0185] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0186] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0187] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0188] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A target recognition model optimization method, characterized in that: The target recognition model optimization method comprises: Acquire a first training sample set, wherein the first training sample set includes a plurality of sample images collected in a traffic target recognition scenario and first label information corresponding to the plurality of sample images, respectively, the first label information reflecting whether there are any target objects in the corresponding sample images and reflecting whether there are various preset types of target objects in the corresponding sample images; The target recognition model is trained using the first training sample set to optimize the target recognition model, wherein the target recognition model includes a feature extraction backbone network and a detection head, the feature extraction backbone network includes a first convolutional layer, a residual layer, and a second convolutional layer connected in sequence, the first convolutional layer is used to receive an image of a first preset size, the second convolutional layer is used to output a feature map of a second preset size to the detection head, the detection head is used to perform classification regression according to the feature map of the second preset size to obtain a target recognition result of the image of the first preset size, the target recognition result reflects whether there is any target object in the image of the first preset size, and reflects whether there are targets of the various preset types in the image of the first preset size.

2. The target recognition model optimization method according to claim 1, characterized in that: A branch network for extracting the maximum-size feature map in the backbone network of YOLOv3 is selected, and the network remaining after removing the splicing layer from the branch network is used as the feature extraction backbone network.

3. The target recognition model optimization method according to claim 1, characterized in that: The step of using the first training sample set to train the target recognition model to optimize the target recognition model comprises: Inputting the plurality of sample images into the target recognition model for recognition respectively, and obtaining the target recognition results respectively corresponding to the plurality of sample images; Calculating target confidence loss and target category loss according to the first label information and the target recognition results respectively corresponding to the plurality of sample images; A first total loss of the target recognition model is calculated according to the target confidence loss and the target category loss, and the target recognition model is optimized according to the first total loss.

4. The target recognition model optimization method according to claim 3, characterized in that: The step of calculating the target confidence loss and the target category loss according to the first label information and the target recognition results respectively corresponding to the plurality of sample images comprises: The first label information corresponding to the plurality of sample images is smoothed, and the target confidence loss and the target category loss are calculated according to the first label information after the smoothing of the plurality of sample images and the target recognition result.

5. The target recognition model optimization method according to any one of claims 1 to 4, characterized in that: The step of obtaining a first training sample set comprises: At least one target sample image is selected from a preset test sample set, wherein the preset test sample set includes a plurality of sample images collected in a traffic target recognition scenario and the first label information respectively corresponding to the plurality of sample images, and a target recognition result obtained by inputting the target sample image into the target recognition model for recognition is inconsistent with the first label information of the target sample image; The first training sample set is constructed according to each of the target sample images.

6. The target recognition model optimization method according to claim 5, characterized in that: The step of constructing the first training sample set according to each of the target sample images comprises: Inputting each of the target sample images into a pre-trained recurrent generative adversarial network, respectively, to generate different-perspective images corresponding to each of the target sample images, wherein the different-perspective images and the corresponding target sample images contain the same target object at different perspectives, and the recurrent generative adversarial network is trained using a second training data set, wherein the second training data set includes multiple groups of images, each group of images includes two images, and the two images contain the same target object at different perspectives; The first training sample set is constructed according to each of the target sample images and each of the different-viewing angle images.

7. The target recognition model optimization method according to claim 6, characterized in that: Before the step of inputting each of the target sample images into a pre-trained recurrent generative adversarial network to generate different-viewing angle images corresponding to each of the target sample images, the method further includes: Inputting the multiple groups of images into the recurrent generative adversarial network to be trained, and calculating a second total loss of the recurrent generative adversarial network to be trained based on the output results and the preset second label information corresponding to the multiple groups of images respectively; The recurrent generative adversarial network to be trained is optimized according to the second total loss to complete one round of training of the recurrent generative adversarial network to be trained. After multiple rounds of training of the recurrent generative adversarial network to be trained, the trained recurrent generative adversarial network is obtained.

8. The target recognition model optimization method according to claim 7, characterized in that: The cyclic generative adversarial network includes a first generative adversarial network and a second generative adversarial network, the first generative adversarial network includes a first generative network and a first discriminative network, and the second generative adversarial network includes a second generative network and a second discriminative network; The step of inputting the multiple groups of images into the recurrent generative adversarial network to be trained, and calculating the second total loss of the recurrent generative adversarial network to be trained based on the output results and the preset second label information corresponding to the multiple groups of images respectively comprises: Input the multiple groups of images into the cyclic generative adversarial network to be trained, and calculate the first adversarial loss of the first generative adversarial network, the second adversarial loss of the second generative adversarial network, and the cycle consistency loss between the first generative network and the second generative network based on the output results and the preset second label information corresponding to the multiple groups of images respectively; A second total loss of the cycle generative adversarial network to be trained is calculated according to the first adversarial loss, the second adversarial loss, and the cycle consistency loss.

9. The target recognition model optimization method according to claim 8, characterized in that: The first generative adversarial network further includes a first network mapping layer formed by connecting a plurality of fully connected layers in series, the first network mapping layer being located before the first generative network and being used to perform network mapping on the image before inputting the image into the first generative network; The second generative adversarial network also includes a second network mapping layer formed by connecting a plurality of fully connected layers in series. The second network mapping layer is located before the second generative network and is used to perform network mapping on the image before the image is input into the second generative network.

10. The target recognition model optimization method according to claim 8, characterized in that: The second label information includes a first label value; the steps of inputting the multiple groups of images into the recurrent generative adversarial network to be trained, and calculating the first adversarial loss of the first generative adversarial network based on the output results and the preset second label information corresponding to the multiple groups of images respectively include: For any target image group among the multiple groups of images, a third image obtained by splicing the first image and the second image is input into the first discriminant network for discrimination, so as to obtain a first discrimination result corresponding to the target image group, wherein the first image is an image of the target image group whose viewing angle is the first viewing angle, and the second image is an image of the target image group whose viewing angle is the second viewing angle; Calculating a first loss corresponding to the first generative adversarial network according to the first label values ​​and the first discrimination results respectively corresponding to the multiple groups of images, wherein the first label value corresponding to the target image group indicates that the first image is a real image and that the first image and the second image are from the same group of images; A first adversarial loss of the first generative adversarial network is obtained according to the first loss calculation.

11. The target recognition model optimization method according to claim 10, characterized in that: The second label information also includes a second label value, and the step of calculating the first adversarial loss of the first generative adversarial network according to the first loss includes: Input the first image into the first generation network for image generation to obtain a fourth image with the second perspective, splice the fourth image with the second image and input the spliced ​​image into the first discrimination network for discrimination to obtain a second discrimination result corresponding to the target image group; Calculating a second loss corresponding to the first generative adversarial network according to the second label values ​​and the second discrimination results respectively corresponding to the multiple groups of images, wherein the second label value corresponding to the target image group indicates that the fourth image is a real image and indicates that the second image and the fourth image are from the same group of images; A first adversarial loss of the first generative adversarial network is calculated according to the first loss and the second loss.

12. The target recognition model optimization method according to claim 11, characterized in that: The second label information also includes a third label value, and the step of calculating the first adversarial loss of the first generative adversarial network according to the first loss and the second loss includes: Calculating a third loss corresponding to the first generative adversarial network according to the third label values ​​and the second discrimination results respectively corresponding to the multiple groups of images, wherein the third label value corresponding to the target image group indicates that the fourth image is a generated image and indicates that the second image and the fourth image are from different image groups; The first adversarial loss of the first generative adversarial network is calculated according to the first loss, the second loss, and the third loss corresponding to the first generative network.

13. The target recognition model optimization method according to claim 6, characterized in that: The step of constructing the first training sample set according to each of the target sample images and each of the different-viewing angle images comprises: Performing CutMix data enhancement processing on any two of the target sample images and the different-viewing angle images to obtain a data enhanced image; The plurality of data-enhanced images processed by CutMix data enhancement are randomly cropped and then stitched into one image to obtain a stitched image; The first training sample set is constructed according to a plurality of the stitched images obtained based on each of the target sample images and each of the different-viewing angle images.

14. A target recognition model optimization device, characterized in that: The target recognition model optimization device includes: a memory, a processor, and a target recognition model optimization program stored in the memory and executable on the processor. When the target recognition model optimization program is executed by the processor, the target recognition model optimization method according to any one of claims 1 to 13 is implemented.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a target recognition model optimization program, and when the target recognition model optimization program is executed by a processor, the target recognition model optimization method according to any one of claims 1 to 13 is implemented.

Images