Model training method, vehicle identification method, device, electronic equipment and medium

Abstract

The present application discloses a model training method, a vehicle identification method, a device, an electronic device and a medium, and relates to the field of neural network technology. The model training method inputs a vehicle training image into an initial vehicle identification model to obtain a first training image feature, and inputs a vehicle enhancement training image into a training auxiliary model to obtain a second training image feature. Then, local contrast loss, global contrast loss, inter-class contrast loss and class-to-class contrast loss are obtained respectively, thereby obtaining a total loss, and the first network parameter of the initial vehicle identification model is iteratively updated based on the total loss. In the process of iterative updating, the accuracy of the target vehicle identification model in distinguishing vehicles of different categories is improved through inter-class contrast loss and class-to-class contrast loss, and the position regression accuracy of vehicles of the same category is improved. The local and global feature distributions of the target vehicle identification model can be enhanced through local contrast loss and global contrast loss.

Description

Technical Field

[0001] The present application relates to the field of neural network technology, and in particular to a model training method, a vehicle identification method, a device, an electronic device and a medium. Background Art

[0002] With the increasing popularity and development of the concept of smart cities, intelligent transportation has received more and more attention as an important component of smart cities. Intelligent Transportation System (ITS) can update and allocate traffic resources in real time according to the traffic volume on different roads in different time periods to ensure the fast and safe passage of cars in the city. Therefore, the main purpose of the vehicle detection task is to accurately and efficiently classify and count the vehicles on the road.

[0003] In related technologies, pre-trained neural network models are usually used to identify vehicles. However, in actual application scenarios, it is difficult to use these pre-trained models in complex traffic scenarios, and the accuracy of model classification in actual applications is low. Summary of the invention

[0004] The present application aims to solve at least one of the technical problems existing in the prior art. To this end, the present application proposes a model training method, a vehicle identification method, a device, an electronic device and a medium, which can improve the accuracy of the target vehicle identification model.

[0005] To achieve the above objectives, the first embodiment of the present application provides a vehicle recognition model training method, comprising:

[0006] Acquire a vehicle training image and a real label and a real detection frame of the vehicle training image, and perform enhancement processing on the vehicle training image to obtain a vehicle enhanced training image;

[0007] Inputting the vehicle training image into a first feature extraction module of an initial vehicle recognition model to obtain first training image features, and inputting the vehicle enhanced training image into a second feature extraction module of a training auxiliary model to obtain second training image features;

[0008] Obtaining a local contrast loss based on the first training image feature and the second training image feature;

[0009] Inputting the first training image feature into the first classification detection module of the initial vehicle recognition model to obtain a first classification result; inputting the second training image feature into the second classification detection module of the training auxiliary model to obtain a second classification result;

[0010] Obtaining a global contrast loss based on the first classification result and the second classification result;

[0011] Obtaining an inter-class contrast loss based on the predicted label of the first classification result and the true label of the vehicle training image;

[0012] Obtaining a class-to-class comparison loss based on a predicted detection box in the first classification result and a real detection box of the vehicle training image;

[0013] Obtaining a total loss based on the local contrast loss, the global contrast loss, the inter-class contrast loss, and the class-class contrast loss;

[0014] The first network parameter of the initial vehicle recognition model is iteratively updated based on the total loss to obtain a target vehicle recognition model.

[0015] According to some embodiments of the first aspect of the present application, the step of inputting the vehicle training image into a first feature extraction module of an initial vehicle recognition model to obtain first training image features, and inputting the vehicle enhanced training image into a second feature extraction module of a training auxiliary model to obtain second training image features includes:

[0016] Performing feature extraction on the vehicle training image by using the first feature extraction module to obtain a first extracted feature; performing feature extraction on the vehicle enhanced training image by using the second feature extraction module to obtain a second extracted feature;

[0017] The first extracted features are subjected to multiple consecutive bilinear interpolation processes to obtain the first training image features; and the second extracted features are subjected to multiple consecutive bilinear interpolation processes to obtain the second training image features.

[0018] According to some embodiments of the first aspect of the present application, obtaining the local contrast loss based on the first training image feature and the second training image feature includes:

[0019] Performing a local projection transformation on the first training image feature to obtain a first projection feature spectrum; performing a local projection transformation on the second training image feature to obtain a second projection feature spectrum;

[0020] The local contrast loss is calculated based on the pixel points of the first projection feature spectrum, the pixel points of the second projection feature spectrum, and a local contrast loss function; the local contrast loss function is: ;

[0021] ;

[0022] in, Loss LCLrepresents the local contrast loss, and P represents a collection of pixel points of the first projection feature spectrum and the second projection feature spectrum; p c Characterizing pixel points of the second projection characteristic spectrum; p sc Characterizing pixel points of the first projection characteristic spectrum; h×w Represents the range of pixels; MMD represents the maximum mean difference; Characterize the temperature coefficient, Characterize the mapping function, Fx Characterize the i-th pixel point of the first projection feature spectrum, Fyi represents the i-th pixel point of the second projection characteristic spectrum, H represents the reproducing kernel Hilbert space; c represents the index of the pixel point of the second projection characteristic spectrum, sc represents the index of the pixel point of the first projection characteristic spectrum, A collection representing the index sc.

[0023] According to some embodiments of the first aspect of the present application, obtaining a global contrast loss based on the first classification result and the second classification result includes:

[0024] The global contrast loss is calculated based on the first classification result, the second classification result and the global contrast loss function; the global contrast loss function is:

[0025] ;

[0026] in, Loss GCL Characterizing the global contrast loss; characterize the first classification result, Characterizing the second classification result, Characterizes the second-order norm.

[0027] According to some embodiments of the first aspect of the present application, obtaining the inter-class contrast loss based on the predicted label of the first classification result and the real label of the vehicle training image includes:

[0028] The inter-class contrast loss is calculated based on the predicted label of the first classification result, the real label of the vehicle training image and the inter-class contrast loss function; the inter-class contrast loss function is:

[0029] ;

[0030] in, Loss InterCL represents the inter-class contrast loss; C represents the number of categories, yi Characterization i The real labels, yjCharacterization j The real labels, ai Characterization i predicted labels, aj Characterization j predicted labels, Characterizes the temperature coefficient.

[0031] According to some embodiments of the first aspect of the present application, obtaining the class-to-class contrast loss based on the predicted detection box in the first classification result and the real detection box of the vehicle training image includes:

[0032] The class-to-class contrast loss is calculated based on the predicted detection box in the classification result, the real detection box of the vehicle training image, and the class-to-class contrast loss function; the class-to-class contrast loss function is:

[0033] ;

[0034] in, Loss IntraCL represents the class-to-class contrast loss; C represents the number of categories, S is the product of the number of aspect ratio types and the number of scales, xi Characterization i A real detection box, bi Characterization i Predicted detection boxes, bj Characterization j predicted detection boxes, N represents the number of detection boxes of the same category; IoU Represents intersection and union ratio; Characterizes the temperature coefficient.

[0035] According to some embodiments of the first aspect of the present application, the iterative updating of the first network parameter of the initial vehicle recognition model based on the total loss to obtain the target vehicle recognition model includes:

[0036] Updating the first network parameter based on the total loss and the optimizer algorithm to obtain an updated network parameter;

[0037] The second network parameter of the training auxiliary model is updated based on the updated network parameter and the exponential moving average algorithm.

[0038] According to some embodiments of the first aspect of the present application, the step of inputting the first training image feature into a first classification detection module of the initial vehicle recognition model to obtain a first classification result; and inputting the second training image feature into a second classification detection module of the training auxiliary model to obtain a second classification result includes:

[0039] Performing a region of interest interpolation process on the first training image feature to obtain a first training foreground feature; performing a region of interest interpolation process on the second training image feature to obtain a second training foreground feature;

[0040] The first training foreground feature is detected by the detection head of the first classification detection module to obtain a first detection result; the second training foreground feature is detected by the detection head of the second classification detection module to obtain a second detection result;

[0041] The first detection result is mapped through the projection layer of the first classification detection module to obtain the first classification result; the second detection result is mapped through the projection layer of the second classification detection module to obtain the second classification result.

[0042] To achieve the above-mentioned purpose, a second aspect of the present application provides a vehicle identification method, including:

[0043] Acquire the image of the vehicle to be detected;

[0044] The image of the vehicle to be detected is input into the target vehicle recognition model described in any one of the embodiments of the first aspect of the present application to obtain a target classification result.

[0045] To achieve the above-mentioned purpose, the third aspect of the present application provides a vehicle recognition model training device, including:

[0046] An acquisition module is used to acquire a vehicle training image and a real label and a real detection frame of the vehicle training image, and perform enhancement processing on the vehicle training image to obtain a vehicle enhanced training image;

[0047] A first input module is used to input the vehicle training image into a first feature extraction module of an initial vehicle recognition model to obtain first training image features, and input the vehicle enhanced training image into a second feature extraction module of a training auxiliary model to obtain second training image features;

[0048] A first loss calculation module, used for obtaining a local contrast loss based on the first training image feature and the second training image feature;

[0049] A second input module, used to input the first training image feature into the first classification detection module of the initial vehicle recognition model to obtain a first classification result; and input the second training image feature into the second classification detection module of the training auxiliary model to obtain a second classification result;

[0050] A second loss calculation module, used for obtaining a global contrast loss based on the first classification result and the second classification result;

[0051] A third loss calculation module, configured to obtain an inter-class contrast loss based on a predicted label of the first classification result and a true label of the vehicle training image;

[0052] a fourth loss calculation module, configured to obtain a class-to-class comparison loss based on a predicted detection box in the first classification result and a real detection box in the vehicle training image;

[0053] a fifth loss calculation module, configured to obtain a total loss based on the local contrast loss, the global contrast loss, the inter-class contrast loss, and the class-class contrast loss;

[0054] The iterative training module is used to iteratively update the first network parameter of the initial vehicle recognition model based on the total loss to obtain a target vehicle recognition model.

[0055] To achieve the above-mentioned purpose, the fourth aspect embodiment of the present application provides an electronic device, which includes a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, it implements the vehicle identification model training method described in any one of the first aspect embodiments, or the vehicle identification method described in the second aspect embodiments.

[0056] To achieve the above-mentioned objectives, the fifth aspect embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it is the vehicle identification model training method described in any one of the first aspect embodiments of the present application, or the vehicle identification method described in the second aspect embodiments of the present application.

[0057] The model training method, vehicle recognition method, device, electronic device and medium of the embodiments of the present application input the vehicle training image into the first feature extraction module of the initial vehicle recognition model to obtain the first training image feature, input the vehicle enhancement training image into the second feature extraction module of the training auxiliary model to obtain the second training image feature; obtain the local contrast loss based on the first training image feature and the second training image feature; input the first training image feature into the first classification detection module of the initial vehicle recognition model to obtain the first classification result; input the second training image feature into the second classification detection module of the training auxiliary model to obtain the second classification result; obtain the global contrast loss based on the first classification result and the second classification result; obtain the inter-class contrast loss based on the predicted label of the first classification result and the real label of the vehicle training image; obtain the class-to-class contrast loss based on the predicted detection box in the first classification result and the real detection box of the vehicle training image; iteratively update the first network parameter of the initial vehicle recognition model based on the local contrast loss, the global contrast loss, the inter-class contrast loss and the class-to-class contrast loss. During the iterative updating process, the inter-class contrast loss and class-class contrast loss can improve the accuracy of the target vehicle recognition model in distinguishing vehicles of different categories, and improve the position regression accuracy of vehicles of the same category. The local contrast loss and global contrast loss can enhance the local and global feature distribution of the target vehicle recognition model, thereby improving the accuracy of the target vehicle recognition model.

[0058] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] The present application is further described below with reference to the accompanying drawings and embodiments, wherein:

[0060] Figure 1 A training framework for the training method of an embodiment of the present application;

[0061] Figure 2 A schematic diagram of the steps of the vehicle recognition model training method according to an embodiment of the present application;

[0062] Figure 3 for Figure 1 A specific flow chart of step S220;

[0063] Figure 4 for Figure 1 A specific flow chart of step S290;

[0064] Figure 5 for Figure 1 A specific flow chart of step S240;

[0065] Figure 6 A schematic diagram of a specific process of the vehicle identification method of the present application;

[0066] Figure 7 A schematic diagram of the structure of a vehicle recognition model training device according to an embodiment of the present application;

[0067] Figure 8 A schematic diagram of the structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0068] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be understood as limiting the present application.

[0069] In the description of the present application, it should be understood that descriptions involving orientation, such as up, down, front, back, left, right, etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present application.

[0070] In the description of this application, "several" means more than one, "more" means more than two, "greater than", "less than", "exceed", etc. are understood to exclude the number itself, and "above", "below", "within", etc. are understood to include the number itself. If there is a description of "first" or "second", it is only used for the purpose of distinguishing technical features, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0071] In the description of this application, unless otherwise clearly defined, terms such as setting, installing, connecting, etc. should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meanings of the above terms in this application based on the specific content of the technical solution.

[0072] In the description of the present application, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0073] First, some nouns involved in this application are analyzed:

[0074] Supervised Learning: Supervised learning is one of the most common learning methods in neural networks. In this learning method, the model receives a large amount of labeled input data during the training phase. The model predicts the label of the new input data by learning the mapping relationship between the input data and the label. During the training process, the supervised learning neural network model adjusts the network parameters according to the difference between the predicted result and the true label to minimize the prediction error. Common supervised learning neural network models include convolutional neural networks (CNN), recurrent neural networks (RNN), etc., which are widely used in image classification, speech recognition, natural language processing and other fields.

[0075] Self-Supervised Learning: Self-supervised learning is a type of unsupervised learning, also known as a pretext task. It mainly uses auxiliary tasks (pretext) to mine its own supervisory information from large-scale unsupervised data, and trains the network through this constructed supervisory information, so that it can learn representations that are valuable for downstream tasks. Its core principle is how to automatically generate labels for data and use these automatically generated labels to guide the model to learn. The generation of these labels does not rely on external manual annotations, but is achieved by designing some tasks that automatically generate labels.

[0076] Online Network: An online network is a neural network used to encode raw input data and generate representation vectors. During training, the network processes the input data in real time and continuously updates its parameters based on the gradient information fed back by the loss function. The online network is dynamic, and its parameters change as training progresses to better adapt to the characteristics of the input data. This enables the online network to learn the intrinsic representation of the data and use it for subsequent tasks such as classification, regression, or generation.

[0077] Target Network: The target network is another neural network with the same structure as the online network but different parameters. Its main function is to provide a stable learning target for calculating the loss between the representation vector generated by the online network and the target representation vector. The parameters of the target network are not updated in real time, but are copied from the online network through methods such as exponential moving average (EMA). This update method makes the parameter changes of the target network relatively stable and will not fluctuate violently due to the rapid update of the online network. During the training process, the target network provides a fixed and relatively stable learning target for the online network. This helps the online network learn the intrinsic representation of the data more stably and reduce the instability during the training process. During the training process, the loss between the representation vector generated by the online network and the representation vector generated by the target network is calculated. This loss value is used to update the parameters of the online network so that the representation vector generated by it is closer to the target representation vector. The collaborative work of the online network and the target network makes self-supervised learning possible. By continuously updating the parameters of the online network and maintaining a stable target network, the intrinsic representation of the data can be learned and used for subsequent tasks. This method has achieved remarkable results in computer vision tasks such as image classification, object detection, and semantic segmentation.

[0078] With the increasing popularity and development of the concept of smart cities, intelligent transportation has received more and more attention as an important component of smart cities. Intelligent Transportation System (ITS) can update and allocate traffic resources in real time according to the traffic volume on different roads in different time periods to ensure the fast and safe passage of cars in the city. Therefore, the main purpose of the vehicle detection task is to accurately and efficiently classify and count the vehicles on the road.

[0079] Traditional vehicle detection methods often use artificially designed prior information to process road conditions in specific situations, and can count the number of vehicles more accurately. However, the robustness of traditional methods is weak and cannot be extended to more complex and changeable actual scenarios. With the development and evolution of neural networks, vehicle detection methods based on deep learning can detect more complex vehicle categories and quantity changes, thereby greatly improving the accuracy of vehicle statistics. Considering the differences between existing training data and actual scenes in ambient lighting, size complexity of the vehicles to be detected, and observation angle, the detection performance of current methods will tend to weaken after migration to complex scenes. A variety of complex factors lead to a decrease in the accuracy of vehicle detection methods in distinguishing different vehicle types. At the same time, the predicted position of the same vehicle is inaccurate, which is prone to the problem of multiple detection frames.

[0080] Based on this, the embodiments of the present application propose a model training method, a vehicle identification method, a device, an electronic device and a medium, which can improve the accuracy of the target vehicle identification model. By identifying the vehicle through the target vehicle identification model, the accuracy of vehicle type discrimination can be improved, and the accuracy of vehicle position prediction can be improved.

[0081] First, the training framework of the vehicle recognition model training method of the embodiment of the present application is described. Figure 1 , Figure 1 The training framework of the training method of the embodiment of the present application. The training framework includes an initial vehicle recognition model and a training auxiliary model. The structures of the vehicle recognition model and the training auxiliary model are basically the same, but the weights are different. During the training process, the initial vehicle recognition model is used as an online network, and the training auxiliary model is used as a target network. The initial vehicle recognition model includes a first feature extraction module, a region of interest layer, a detection head, and two projection layers. The training auxiliary model includes a second feature extraction module, a region of interest layer, a detection head, and a projection layer. Figure 1 In the above, the candidate regions are obtained through the region of interest layer. I θ Characterize vehicle training images, Characterizing vehicle augmented training images.

[0082] based on Figure 1 A training framework is provided to propose a vehicle recognition model training method in an embodiment of the present application.

[0083] The vehicle identification model training method of the embodiments of the present application can be applied to a terminal or a server. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a router, a programmable switch, a network card, etc.; the server can be configured as an independent physical server, or a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the vehicle identification model training method, etc., but is not limited to the above forms.

[0084] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific services or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which services are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0085] Reference Figure 2 , Figure 2 The vehicle recognition model training method of the embodiment of the present application is a flowchart of the steps. The vehicle recognition model training method of the embodiment of the present application includes but is not limited to the following steps:

[0086] Step S210, obtaining a vehicle training image and a real label and a real detection frame of the vehicle training image, and performing enhancement processing on the vehicle training image to obtain a vehicle enhanced training image;

[0087] It is worth noting that the vehicle training image can be obtained by photographing the vehicle with a camera, and the real label and real detection frame of the vehicle training image can be obtained by manual annotation. In one embodiment, the vehicle training image and the real label and real detection frame of the vehicle training image can be obtained from a public material library. There are many ways to obtain from the material library. For example, the material library can be accessed through a custom application to obtain the vehicle training image and the real label and real detection frame of the vehicle training image.

[0088] It should be noted that the true detection box is a box used to mark the vehicle in the vehicle training image. Usually, in the vehicle training image, the vehicle is located inside the true detection box. The true label of the vehicle training image is used to characterize the category of the vehicle in the vehicle training image. For example, the true label can be "1" or "2" or "3". When the true label is "1", it means that the category of the vehicle in the vehicle training image is a bus; when the true label is "2", it means that the category of the vehicle in the vehicle training image is a taxi; when the true label is "3", it means that the category of the vehicle in the vehicle training image is a private car.

[0089] In one embodiment, enhancing the vehicle training image includes adjusting the brightness and contrast of the vehicle training image, horizontally flipping and vertically flipping the vehicle training image, and cropping and scaling the vehicle training image, so as to obtain a vehicle enhanced training image that is different from the vehicle training image.

[0090] In one embodiment, the vehicle training image is enhanced by: converting the vehicle training image from RGB space to YCrCb space, where the Y channel represents the brightness component; then, applying the MSRCR algorithm to the Y channel to enhance the image by estimating the ambient light illumination component and calculating the reflection component of the object. During the calculation process, the MSRCR algorithm uses Gaussian filters of multiple scales to estimate the ambient light illumination component, and adjusts the proportional relationship between the three color channels in the original image through the color restoration factor, thereby eliminating the defect of image color distortion. Then, the enhanced Y channel and the original Cr and Cb channels are merged back into the RGB space to obtain an enhanced color image, which is used as the vehicle enhanced training image. YCrCb space, also known as YUV space, is a color encoding method, which is mainly used to optimize the transmission of color video signals; YCrCb space includes Y component, Cr component and Cb component. The Y component represents brightness (Luminance or Luma), that is, the grayscale value. It reflects the brightness information of the image, which is related to the red, green and blue components in the RGB signal, and is obtained by superimposing specific parts of the RGB signal. The Cr component represents the red chromaticity (the red part of Chrominance or Chroma), which reflects the difference between the red part of the RGB input signal and the brightness value of the RGB signal. The Cb component represents the blue chromaticity (the blue part of Chrominance or Chroma), which reflects the difference between the blue part of the RGB input signal and the brightness value of the RGB signal. The MSRCR (Multi-Scale Retinex with Color Restoration) algorithm is an image enhancement method based on the Retinex theory.

[0091] Step S220, inputting the vehicle training image into the first feature extraction module of the initial vehicle recognition model to obtain first training image features, and inputting the vehicle enhanced training image into the second feature extraction module of the training auxiliary model to obtain second training image features;

[0092] Step S230, obtaining a local contrast loss based on the first training image feature and the second training image feature;

[0093] Step S240, inputting the first training image feature into the first classification detection module of the initial vehicle recognition model to obtain a first classification result; inputting the second training image feature into the second classification detection module of the training auxiliary model to obtain a second classification result;

[0094] It should be noted that the first classification result includes a predicted label and a predicted detection box; the second classification result includes a predicted label and a predicted detection box. The predicted label is used to indicate the category of the vehicle in the vehicle training image, and the predicted detection box is used to mark the vehicle in the vehicle training image.

[0095] Step S250, obtaining a global contrast loss based on the first classification result and the second classification result;

[0096] Step S260, obtaining an inter-class contrast loss based on the predicted label of the first classification result and the real label of the vehicle training image;

[0097] Step S270, obtaining a class-to-class comparison loss based on the predicted detection box in the first classification result and the real detection box of the vehicle training image;

[0098] Step S280, obtaining a total loss based on the local contrast loss, the global contrast loss, the inter-class contrast loss, and the class-class contrast loss;

[0099] Step S290, iteratively updating the first network parameter of the initial vehicle recognition model based on the total loss to obtain a target vehicle recognition model.

[0100] It is worth noting that steps S210 to S290 are repeatedly performed, and each time the total loss is calculated, the first network parameter of the initial vehicle recognition model is updated once, for example, after the total loss is obtained, the first network parameter is updated using a gradient descent algorithm. In this way, the first network parameter is iteratively updated until the number of iterations reaches a preset iteration threshold.

[0101] The embodiment of the present application, through the above steps S210 to S290, inputs the vehicle training image into the first feature extraction module of the initial vehicle recognition model to obtain the first training image feature, and inputs the vehicle enhanced training image into the second feature extraction module of the training auxiliary model to obtain the second training image feature; obtains the local contrast loss based on the first training image feature and the second training image feature; inputs the first training image feature into the first classification detection module of the initial vehicle recognition model to obtain the first classification result; inputs the second training image feature into the second classification detection module of the training auxiliary model to obtain the second classification result; obtains the global contrast loss based on the first classification result and the second classification result; obtains the inter-class contrast loss based on the predicted label of the first classification result and the real label of the vehicle training image; obtains the class-to-class contrast loss based on the predicted detection box in the first classification result and the real detection box of the vehicle training image; and iteratively updates the first network parameters of the initial vehicle recognition model based on the local contrast loss, the global contrast loss, the inter-class contrast loss, and the class-to-class contrast loss. During the iterative updating process, the inter-class contrast loss and class-class contrast loss can improve the accuracy of the target vehicle recognition model in distinguishing vehicles of different categories, and improve the position regression accuracy of vehicles of the same category. The local contrast loss and global contrast loss can enhance the local and global feature distribution of the target vehicle recognition model, thereby improving the accuracy of the target vehicle recognition model.

[0102] It is worth noting that the model training method of the embodiment of the present application combines supervised learning and self-supervised learning, the initial vehicle recognition model is used as the online network, and the training auxiliary model is used as the target network. The present application improves the classification and regression accuracy of the online network, as well as the feature distribution of the target network and the online network, by designing inter-class contrast loss, class-class contrast loss, and local contrast loss, thereby improving the detection performance of the online network.

[0103] In one embodiment, the initial vehicle recognition model is a pre-trained model. Usually, the pre-trained model needs to be fine-tuned before it can be actually applied. The embodiment of the present application implements the updating of the first network parameters through steps S210 to S290, that is, the initial vehicle recognition model is fine-tuned to obtain the target vehicle recognition model, so that the target vehicle detection model can be applied in practice and the accuracy of the target vehicle detection model is improved.

[0104] In one embodiment, referring to Figure 3 , Figure 3 for Figure 1 A specific flow chart of step S220 in FIG. 1 is shown in FIG. 1 . Step S220 may include but is not limited to the following steps:

[0105] Step S310, extracting features from the vehicle training image using a first feature extraction module to obtain a first extracted feature; extracting features from the vehicle enhanced training image using a second feature extraction module to obtain a second extracted feature;

[0106] It is worth noting that both the first feature extraction module and the second feature extraction module use ResNet-50 as the feature extraction layer for feature extraction. The dimensions of the first extracted feature and the second extracted feature are , H and W represent the height and width of the vehicle training image, respectively. ResNet-50 is a deep convolutional neural network based on the residual network (ResNet) architecture, which has a wide range of applications in the field of computer vision. ResNet-50 introduces residual connections (SkipConnection), allowing the network input to skip one or more layers and be added to the output directly, which can alleviate the gradient vanishing problem and allow the network to be trained deeper. In order to further reduce the number of parameters and the amount of computation, ResNet-50 uses a Bottleneck structure in the residual block, that is, a 1x1 convolution layer is used to reduce and increase the dimension, with a 3x3 convolution layer in the middle, which can reduce the number of parameters while maintaining performance.

[0107] Step S320, performing a plurality of consecutive bilinear interpolation processes on the first extracted features to obtain a first training image feature; performing a plurality of consecutive bilinear interpolation processes on the second extracted features to obtain a second training image feature.

[0108] It is worth noting that referring to Figure 1 ,exist Figure 1 middle, Characterize the bilinear interpolation processing. After obtaining the first extracted feature, the first extracted feature is subjected to a plurality of consecutive bilinear interpolation processings to obtain a training image feature. Specifically, the number of bilinear interpolation processings is n. The first extracted feature is subjected to the first bilinear interpolation processing to obtain the first interpolation processing feature, and then the i-1th bilinear interpolation processing is performed to obtain the i-th interpolation processing feature, where i is a positive integer greater than 1 and less than or equal to n. The nth interpolation processing feature is used as the first training image feature. The process of performing a plurality of consecutive bilinear interpolation processings on the second extracted feature is the same as the process of the first extracted feature, and will not be repeated here. Reference Figure 1 , the first extracted feature is ; The second extracted feature is .

[0109] It should be noted that bilinear interpolation is a commonly used interpolation method in image processing and computer graphics, especially in geometric transformations such as image scaling and rotation. It works by performing linear interpolation in two directions (usually x and y directions) to obtain smoother image quality.

[0110] The embodiment of the present application obtains the first training image feature and the second training image feature through step S310 to step S320. In this process, by performing feature extraction through ResNet-50, features at different levels can be extracted. Shallow features contain more spatial information (such as edges, corners, etc.), while deep features contain more abstract semantic information (such as partial or overall representation of objects). This combination of multi-scale features is very helpful for understanding image content. In addition, multiple bilinear interpolation processes are performed on the first extracted features and the second extracted features to obtain multiple first extracted features of different scales and multiple scale dimensions of second extracted features, so as to capture information from local to global. The combination of multi-scale features helps the model better understand the data and improve the accuracy of the model.

[0111] In one embodiment, referring to Figure 4 , Figure 4 for Figure 1 A specific flow chart of step S290 in FIG. 1 is shown in FIG. 1 . Step S290 includes:

[0112] Step S410, updating the first network parameter based on the total loss and the optimizer algorithm to obtain an updated network parameter;

[0113] Specifically, step S410 can be expressed as:

[0114] ,

[0115] in, θ Characterize the first network parameter, Loss Total represents the total loss, optimizer represents the optimizer, Characterizes the gradient operator. In neural networks, optimizers are mainly used to update the parameters of the neural network to reduce or minimize the value of the loss function, thereby improving the accuracy and performance of the model.

[0116] Step S420, updating the second network parameters of the training auxiliary model based on updating the network parameters and the exponential moving average algorithm.

[0117] It is worth noting that the Exponential Moving Average (EMA) is an algorithm for smoothing model parameters, which can help improve the stability and generalization ability of the model. In step S420, the algorithm of the exponential moving average can be expressed as:

[0118] ,

[0119] in, Characterize the second network parameter, .

[0120] The embodiment of the present application implements the updating of the first network parameter and the second network parameter through step S410 and step S420, thereby optimizing the initial vehicle recognition model to obtain the target vehicle recognition model and improve the accuracy of the target vehicle recognition model.

[0121] In one embodiment, referring to Figure 5 , Figure 5 for Figure 1 A specific flow chart of step S240 in FIG. 1 is shown in FIG. 1 . Step S240 may include but is not limited to the following steps:

[0122] Step S510, performing a region of interest interpolation process on the first training image feature to obtain a first training foreground feature; performing a region of interest interpolation process on the second training image feature to obtain a second training foreground feature;

[0123] It is worth noting that referring to Figure 1 , in the region of interest layer, the region of interest interpolation processing is performed. The region of interest interpolation processing (RoIAlign) is a technology used to accurately extract target area features in fields such as target detection and image recognition. It reduces the quantization error and improves the accuracy of feature extraction by avoiding quantization operations and using a bilinear interpolation algorithm to process floating-point coordinates.

[0124] Step S520, detecting the first training foreground feature by the detection head of the first classification detection module to obtain a first detection result; detecting the second training foreground feature by the detection head of the second classification detection module to obtain a second detection result;

[0125] It is worth noting that the detection head is responsible for extracting information from the feature map of the neural network and generating the final detection results.

[0126] Step S530, mapping the first detection result through the projection layer of the first classification detection module to obtain a first classification result; mapping the second detection result through the projection layer of the second classification detection module to obtain a second classification result.

[0127] It is worth noting that the first detection result is mapped twice through two consecutive projection layers to obtain the first classification result. The second detection result is mapped through a projection layer to obtain the second classification result. The projection layer (Projection) is composed of a multi-layer perceptron (MLP). The multi-layer perceptron can obtain the potential features in the first detection result, thereby obtaining the first classification result. The multi-layer perceptron can obtain the potential features in the second detection result, thereby obtaining the second classification result.

[0128] It should be noted that Multi-Layer Perceptron (MLP) is a feed-forward artificial neural network model. It is an algorithm commonly used in deep learning and machine learning, and consists of multiple layers, including an input layer, one or more hidden layers, and an output layer. MLP is trained using a supervised learning technique called backpropagation, which can learn the mapping relationship between input data and output data.

[0129] In one embodiment, step S230, obtaining the local contrast loss based on the first training image feature and the second training image feature, may include the following steps:

[0130] Performing a local projection transformation on the first training image feature to obtain a first projection feature spectrum; performing a local projection transformation on the second training image feature to obtain a second projection feature spectrum;

[0131] Based on the pixel points of the first projection feature spectrum, the pixel points of the second projection feature spectrum and the local contrast loss function, the local contrast loss is calculated; the local contrast loss function is:

[0132] ;

[0133] ;

[0134] in, Loss LCL represents the local contrast loss, P represents the set of pixels of the first projection feature spectrum and the second projection feature spectrum; p c Characterizing pixel points of the second projection characteristic spectrum; p sc Characterizing pixel points of the first projection characteristic spectrum; h×w Represents the range of pixels; MMD represents the maximum mean difference; Characterize the temperature coefficient, Characterize the mapping function, Fx Characterize the i-th pixel point of the first projection feature spectrum, Fyi represents the i-th pixel point of the second projection feature spectrum, H represents the reproducing kernel Hilbert space; c represents the index of the pixel point of the second projection feature spectrum, sc represents the index of the pixel point of the first projection feature spectrum, A collection representing the index sc.

[0135] Specifically, a local projection transformation (Local Projection) is performed on the first training image feature, and the local projection transformation is performed through two consecutive 1×1 convolution layers and a normalization layer (BatchNorm layer). The first training image feature is convolved through two consecutive 1×1 convolution layers, and then the first training image feature is normalized through the normalization layer to obtain the first projection feature spectrum. The process of obtaining the second projection feature spectrum is the same as that of the first projection feature spectrum, which will not be repeated here.

[0136] It is worth noting that in the local contrast loss function, the mapping function Responsible for mapping the original variables to a high-dimensional space, namely the Reproducing Kernel Hilbert Space (RKHS); Maximum Mean Discrepancy (MMD), mainly measures the distance between two different but related distributions x and y. Calculating the local contrast loss can update the first network parameters of the initial vehicle recognition model based on the local contrast loss, which can enhance the distribution of local features of the vehicle recognition model.

[0137] In one embodiment, step S250, obtaining a global contrast loss based on the first classification result and the second classification result, includes:

[0138] Based on the first classification result, the second classification result and the global contrast loss function, the global contrast loss is calculated; the global contrast loss function is:

[0139] ;

[0140] in, Loss GCL Characterize the global contrast loss; Characterize the first classification results, Characterize the second classification results, Characterizes the second-order norm.

[0141] It is worth noting that the global contrast loss is calculated by the global contrast loss function, which facilitates the subsequent use of the global contrast loss to update the first network parameters, and can enhance the distribution of global features of the vehicle recognition model.

[0142] In one embodiment, step S260, obtaining the inter-class contrast loss based on the predicted label of the first classification result and the real label of the vehicle training image, includes:

[0143] Based on the predicted label of the first classification result, the real label of the vehicle training image and the inter-class contrast loss function, the inter-class contrast loss is calculated; the inter-class contrast loss function is:

[0144] ;

[0145] in, Loss InterCL represents the inter-class contrast loss; C represents the number of categories, yi Characterization i The real labels, yj Characterization j The real labels, ai Characterization i predicted labels, aj Characterization j predicted labels, Characterize the temperature coefficient, i and j Respectively index.

[0146] It is worth noting that referring to Figure 1 , for the classification network of the detection head Head (composed of two layers of 3×3 convolution to generate prediction labels, non-prediction categories (such as trucks, buses) are used as negative samples, prediction categories (such as cars) are used as prediction samples, and detection categories (cars) are used as positive samples to design inter-class contrast loss, so that the distance between the prediction sample category and the detection category is shortened, while the distance with other non-prediction categories is increased, thereby improving the discrimination of the target vehicle recognition model for different types of vehicles. The inter-class contrast loss is calculated by the inter-class contrast loss function, which facilitates the subsequent use of the inter-class contrast loss to update the first network parameters, and can improve the accuracy of the target vehicle recognition model in discriminating different types of vehicles.

[0147] In one embodiment, step S270, obtaining a class-to-class comparison loss based on the predicted detection box in the first classification result and the real detection box of the vehicle training image, includes:

[0148] Based on the predicted detection box in the classification result, the real detection box of the vehicle training image and the class-to-class comparison loss function, the class-to-class comparison loss is calculated; the class-to-class comparison loss function is:

[0149] ;

[0150] in, Loss IntraCL represents the class-to-class contrast loss; C represents the number of categories, S is the product of the number of aspect ratio types and the number of scales, xi Characterization i A real detection box, bi Characterization i Predicted detection boxes, bj Characterization j predicted detection boxes, N represents the number of detection boxes of the same category; IoU Represents intersection and union ratio; Characterizes the temperature coefficient.

[0151] It is worth noting that referring to Figure 1 , the prediction label is generated through the regression network of the detection head (composed of two layers of 3×3 convolutions). Under the same prediction category, positive and negative samples are designed according to the detection frames of different aspect ratios and scales in the setting. Specifically, the detection frame with the largest intersection over union (IoU) with the real detection frame (positive sample) is the predicted detection frame, and the detection frames of other different aspect ratios and scales are negative samples, so as to improve the relative coordinate prediction accuracy of the regressor. The class-to-class contrast loss is calculated by the class-to-class contrast loss function, which is convenient for the subsequent use of the class-to-class contrast loss to update the first network parameters, which can improve the position regression accuracy of the target vehicle recognition model for vehicles of the same category.

[0152] In one embodiment, step S280, based on the local contrast loss, the global contrast loss, the inter-class contrast loss and the class-class contrast loss, obtains the total loss, including:

[0153] The total loss is calculated by the total loss function, which is:

[0154] ,

[0155] in, is the weight of the local contrast loss, is the weight of the global contrast loss, is the weight of the inter-class contrast loss, is the weight of class contrast loss, Loss LCL Characterizes the local contrast loss, Loss GCL Characterize the global contrast loss, Loss InterCL Represents the inter-class contrast loss, Loss IntraCL Representation class contrast loss, LossTotal Characterizes the total loss. Those skilled in the art can set , , , The value of .

[0156] The second aspect of the present application provides a vehicle identification method. Figure 6 , Figure 6 The specific flow chart of the vehicle identification method of the present application is as follows. The vehicle identification method comprises the following steps:

[0157] Step S610, obtaining an image of a vehicle to be detected;

[0158] It is worth noting that the road can be photographed by a camera to obtain images of the vehicle to be detected.

[0159] Step S620, inputting the image of the vehicle to be detected into the target vehicle recognition model to obtain a target classification result.

[0160] It is worth noting that the target vehicle recognition model is the target vehicle recognition model obtained in the first aspect embodiment. Since the target vehicle recognition model is trained by steps S210 to S290 in the first aspect embodiment, the vehicle recognition method of the embodiment of the present application has a high recognition accuracy.

[0161] The third aspect of the present application provides a vehicle recognition model training device. Figure 7 , Figure 7 The following is a schematic diagram of the structure of a vehicle recognition model training device according to an embodiment of the present application. The vehicle recognition model training device comprises:

[0162] An acquisition module 710 is used to acquire a vehicle training image and a real label and a real detection frame of the vehicle training image, and perform enhancement processing on the vehicle training image to obtain a vehicle enhanced training image;

[0163] A first input module 720 is used to input the vehicle training image into the first feature extraction module of the initial vehicle recognition model to obtain the first training image feature, and input the vehicle enhanced training image into the second feature extraction module of the training auxiliary model to obtain the second training image feature;

[0164] A first loss calculation module 730, configured to obtain a local contrast loss based on the first training image feature and the second training image feature;

[0165] The second input module 740 is used to input the first training image feature into the first classification detection module of the initial vehicle recognition model to obtain a first classification result; and input the second training image feature into the second classification detection module of the training auxiliary model to obtain a second classification result;

[0166] A second loss calculation module 750, configured to obtain a global contrast loss based on the first classification result and the second classification result;

[0167] A third loss calculation module 760 is used to obtain an inter-class contrast loss based on the predicted label of the first classification result and the real label of the vehicle training image;

[0168] A fourth loss calculation module 770, configured to obtain a class-to-class comparison loss based on the predicted detection box in the first classification result and the real detection box of the vehicle training image;

[0169] a fifth loss calculation module 780, for obtaining a total loss based on the local contrast loss, the global contrast loss, the inter-class contrast loss, and the class-class contrast loss;

[0170] The iterative training module 790 is used to iteratively update the first network parameter of the initial vehicle recognition model based on the total loss to obtain a target vehicle recognition model.

[0171] It is worth noting that the vehicle recognition model training device is used to execute the vehicle recognition model training method of the first aspect of the embodiment of the present application. When executing the method, the vehicle training image is input into the first feature extraction module of the initial vehicle recognition model to obtain the first training image feature, and the vehicle enhanced training image is input into the second feature extraction module of the training auxiliary model to obtain the second training image feature; based on the first training image feature and the second training image feature, a local contrast loss is obtained; the first training image feature is input into the first classification detection module of the initial vehicle recognition model to obtain the first classification result; the second training image feature is input into the second classification detection module of the training auxiliary model to obtain the second classification result; based on the first classification result and the second classification result, a global contrast loss is obtained; based on the predicted label of the first classification result and the true label of the vehicle training image, an inter-class contrast loss is obtained; based on the predicted detection box in the first classification result and the true detection box of the vehicle training image, a class-to-class contrast loss is obtained; based on the local contrast loss, the global contrast loss, the inter-class contrast loss and the class-to-class contrast loss, the first network parameter of the initial vehicle recognition model is iteratively updated. During the iterative updating process, the inter-class contrast loss and class-class contrast loss can improve the target vehicle recognition model's ability to discriminate vehicles of different categories and the accuracy of position regression of vehicles of the same category. The local contrast loss and global contrast loss can enhance the local and global feature distribution of the target vehicle recognition model, thereby improving the accuracy of the target vehicle recognition model.

[0172] In one embodiment, the first input module 720 includes:

[0173] A first extraction unit is used to extract features from the vehicle training image through a first feature extraction module to obtain a first extracted feature; and to extract features from the vehicle enhanced training image through a second feature extraction module to obtain a second extracted feature;

[0174] The second extraction unit is used to perform a plurality of consecutive bilinear interpolation processes on the first extracted features to obtain a first training image feature; and to perform a plurality of consecutive bilinear interpolation processes on the second extracted features to obtain a second training image feature.

[0175] In one embodiment, the first loss calculation module 730 includes:

[0176] A local projection transformation unit is used to perform a local projection transformation on the first training image feature to obtain a first projection feature spectrum; and perform a local projection transformation on the second training image feature to obtain a second projection feature spectrum;

[0177] The first calculation unit is used to calculate the local contrast loss based on the pixel points of the first projection feature spectrum, the pixel points of the second projection feature spectrum and the local contrast loss function.

[0178] In one embodiment, the second loss calculation module 750 includes:

[0179] The second calculation unit is used to calculate the global contrast loss based on the first classification result, the second classification result and the global contrast loss function.

[0180] In one embodiment, the third loss calculation module 760 includes:

[0181] The third calculation unit is used to calculate the inter-class contrast loss based on the predicted label of the first classification result, the actual label of the vehicle training image and the inter-class contrast loss function.

[0182] In one embodiment, the fourth loss calculation module 770 includes:

[0183] The fourth calculation unit is used to calculate the class-to-class contrast loss based on the predicted detection box in the classification result, the real detection box of the vehicle training image and the class-to-class contrast loss function.

[0184] In one embodiment, the iterative training module 790 includes:

[0185] A first updating unit, configured to update the first network parameter based on the total loss and the optimizer algorithm to obtain an updated network parameter;

[0186] The second updating unit is used to update the second network parameters of the training auxiliary model based on the updated network parameters and the exponential moving average algorithm.

[0187] In one embodiment, the second input module 740 includes:

[0188] An interpolation processing unit is used to perform interpolation processing of a region of interest on the first training image feature to obtain a first training foreground feature; and perform interpolation processing of a region of interest on the second training image feature to obtain a second training foreground feature;

[0189] A detection unit, configured to detect the first training foreground feature through the detection head of the first classification detection module to obtain a first detection result; and detect the second training foreground feature through the detection head of the second classification detection module to obtain a second detection result;

[0190] The mapping unit is used to map the first detection result through the projection layer of the first classification detection module to obtain the first classification result; and map the second detection result through the projection layer of the second classification detection module to obtain the second classification result.

[0191] The fifth aspect of the present application provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements any one of the model training methods of the first aspect of the embodiment or the vehicle identification method of the second aspect of the embodiment when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a vehicle-mounted computer, etc.

[0192] Reference Figure 8 , Figure 8 FIG. 1 is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. The electronic device includes:

[0193] The processor 801 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0194] The memory 802 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 802 can store an operating system and other application programs. When the technical solution provided in the embodiments of this specification is implemented by software or firmware, the relevant program code is stored in the memory 802, and the processor 801 calls and executes the model training method or vehicle identification method of the embodiments of this application;

[0195] Input / output interface 803, used to implement information input and output;

[0196] The communication interface 804 is used to realize the communication interaction between the device and other devices. The communication can be realized through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.);

[0197] A bus 805 that transmits information between various components of the device (e.g., the processor 801, the memory 802, the input / output interface 803, and the communication interface 804);

[0198] The processor 801 , the memory 802 , the input / output interface 803 and the communication interface 804 are connected to each other in communication within the device via a bus 805 .

[0199] The fifth aspect of the present application is a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the model training method of any one of the first aspect embodiments or the vehicle identification method of the second aspect embodiments.

[0200] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0201] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0202] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0203] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0204] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0205] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0206] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the mapping relationship of the mapping objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next mapping objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0207] In the several embodiments provided in the present application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0208] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0209] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0210] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0211] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A vehicle recognition model training method, characterized in that: include: Acquire a vehicle training image and a real label and a real detection frame of the vehicle training image, and perform enhancement processing on the vehicle training image to obtain a vehicle enhanced training image; Inputting the vehicle training image into a first feature extraction module of an initial vehicle recognition model to obtain first training image features, and inputting the vehicle enhanced training image into a second feature extraction module of a training auxiliary model to obtain second training image features; Obtaining a local contrast loss based on the first training image feature and the second training image feature; Inputting the first training image feature into the first classification detection module of the initial vehicle recognition model to obtain a first classification result; inputting the second training image feature into the second classification detection module of the training auxiliary model to obtain a second classification result; Obtaining a global contrast loss based on the first classification result and the second classification result; Obtaining an inter-class contrast loss based on the predicted label of the first classification result and the true label of the vehicle training image; Obtaining a class-to-class comparison loss based on a predicted detection box in the first classification result and a real detection box of the vehicle training image; Obtaining a total loss based on the local contrast loss, the global contrast loss, the inter-class contrast loss, and the class-class contrast loss; The first network parameter of the initial vehicle recognition model is iteratively updated based on the total loss to obtain a target vehicle recognition model.

2. The vehicle recognition model training method according to claim 1, characterized in that: The step of inputting the vehicle training image into a first feature extraction module of an initial vehicle recognition model to obtain first training image features, and inputting the vehicle enhanced training image into a second feature extraction module of a training auxiliary model to obtain second training image features comprises: Performing feature extraction on the vehicle training image by the first feature extraction module to obtain a first extracted feature; performing feature extraction on the vehicle enhanced training image by the second feature extraction module to obtain a second extracted feature; The first extracted features are subjected to multiple consecutive bilinear interpolation processes to obtain the first training image features; and the second extracted features are subjected to multiple consecutive bilinear interpolation processes to obtain the second training image features.

3. The vehicle recognition model training method according to claim 1, characterized in that: The obtaining of the local contrast loss based on the first training image feature and the second training image feature includes: Performing a local projection transformation on the first training image feature to obtain a first projection feature spectrum; performing a local projection transformation on the second training image feature to obtain a second projection feature spectrum; The local contrast loss is calculated based on the pixel points of the first projection feature spectrum, the pixel points of the second projection feature spectrum, and a local contrast loss function; the local contrast loss function is: ； ； in, Loss LCL represents the local contrast loss, and P represents a collection of pixel points of the first projection feature spectrum and the second projection feature spectrum; p c Characterizing pixel points of the second projection characteristic spectrum; p sc Characterizing pixel points of the first projection characteristic spectrum; h×w Represents the range of pixels; MMD represents the maximum mean difference; Characterize the temperature coefficient, Characterize the mapping function, Fx Characterize the i-th pixel point of the first projection feature spectrum, Fyi represents the i-th pixel point of the second projection characteristic spectrum, H represents the reproducing kernel Hilbert space; c represents the index of the pixel point of the second projection characteristic spectrum, sc represents the index of the pixel point of the first projection characteristic spectrum, A collection representing the index sc.

4. The vehicle recognition model training method according to claim 1, characterized in that: The obtaining of a global contrast loss based on the first classification result and the second classification result includes: The global contrast loss is calculated based on the first classification result, the second classification result and the global contrast loss function; the global contrast loss function is: ； in, Loss GCL Characterizing the global contrast loss; characterize the first classification result, Characterizing the second classification result, Characterizes the second-order norm.

5. The vehicle recognition model training method according to claim 1, characterized in that: The inter-class contrast loss is obtained based on the predicted label of the first classification result and the real label of the vehicle training image, including: The inter-class contrast loss is calculated based on the predicted label of the first classification result, the real label of the vehicle training image and the inter-class contrast loss function; the inter-class contrast loss function is: ； in, Loss InterCL represents the inter-class contrast loss; C represents the number of categories, yi Characterization i The true labels, yj Characterization j The true labels, ai Characterization i predicted labels, aj Characterization j predicted labels, Characterizes the temperature coefficient.

6. The vehicle recognition model training method according to claim 1, characterized in that: The obtaining of the class-to-class comparison loss based on the predicted detection box in the first classification result and the real detection box of the vehicle training image includes: The class-to-class contrast loss is calculated based on the predicted detection box in the classification result, the real detection box of the vehicle training image, and the class-to-class contrast loss function; the class-to-class contrast loss function is: ； in, Loss IntraCL represents the class-to-class contrast loss; C represents the number of categories, S is the product of the number of aspect ratio types and the number of scales, xi Characterization i A real detection box, bi Characterization i Predicted detection boxes, bj Characterization j predicted detection boxes, N represents the number of detection boxes of the same category; IoU Represents intersection and union ratio; Characterizes the temperature coefficient.

7. The vehicle recognition model training method according to claim 1, characterized in that: The iterative updating of the first network parameter of the initial vehicle recognition model based on the total loss to obtain the target vehicle recognition model includes: Updating the first network parameter based on the total loss and the optimizer algorithm to obtain an updated network parameter; The second network parameter of the training auxiliary model is updated based on the updated network parameter and the exponential moving average algorithm.

8. The vehicle recognition model training method according to claim 1, characterized in that: inputting the first training image feature into a first classification detection module of the initial vehicle recognition model to obtain a first classification result; Inputting the second training image feature into the second classification detection module of the training auxiliary model to obtain a second classification result includes: Performing a region of interest interpolation process on the first training image feature to obtain a first training foreground feature; performing a region of interest interpolation process on the second training image feature to obtain a second training foreground feature; The first training foreground feature is detected by the detection head of the first classification detection module to obtain a first detection result; the second training foreground feature is detected by the detection head of the second classification detection module to obtain a second detection result; The first detection result is mapped through the projection layer of the first classification detection module to obtain the first classification result; the second detection result is mapped through the projection layer of the second classification detection module to obtain the second classification result.

9. A vehicle identification method, characterized in that: include: Acquire the image of the vehicle to be detected; The image of the vehicle to be detected is input into the target vehicle recognition model described in any one of claims 1 to 8 to obtain a target classification result.

10. A vehicle recognition model training device, characterized in that: include: An acquisition module is used to acquire a vehicle training image and a real label and a real detection frame of the vehicle training image, and perform enhancement processing on the vehicle training image to obtain a vehicle enhanced training image; A first input module is used to input the vehicle training image into a first feature extraction module of an initial vehicle recognition model to obtain first training image features, and input the vehicle enhanced training image into a second feature extraction module of a training auxiliary model to obtain second training image features; A first loss calculation module, used for obtaining a local contrast loss based on the first training image feature and the second training image feature; A second input module, used for inputting the first training image feature into the first classification detection module of the initial vehicle recognition model to obtain a first classification result; Inputting the second training image feature into the second classification detection module of the training auxiliary model to obtain a second classification result; A second loss calculation module, used for obtaining a global contrast loss based on the first classification result and the second classification result; A third loss calculation module, configured to obtain an inter-class contrast loss based on a predicted label of the first classification result and a true label of the vehicle training image; a fourth loss calculation module, configured to obtain a class-to-class comparison loss based on a predicted detection box in the first classification result and a real detection box in the vehicle training image; a fifth loss calculation module, configured to obtain a total loss based on the local contrast loss, the global contrast loss, the inter-class contrast loss, and the class-class contrast loss; The iterative training module is used to iteratively update the first network parameter of the initial vehicle recognition model based on the total loss to obtain a target vehicle recognition model.

11. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the vehicle identification model training method according to any one of claims 1 to 8, or the vehicle identification method according to claim 9 when executing the computer program.

12. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the vehicle identification model training method according to any one of claims 1 to 8 or the vehicle identification method according to claim 9 is implemented.

Images