A vehicle identification model creation method, vehicle identification method and related components
By enhancing the vehicle sample images and end-to-end training of backbone network classification head network, the problem of large differences within the class and small differences between classes is solved, and the recognition efficiency and accuracy of the vehicle recognition model are improved. , enhances the vehicle's detailed feature extraction capability.
Patent Information
- Application Number
- CN202110467847.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-04-28
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2041-04-28
AI Technical Summary
Under the influence of factors such as lighting, occlusion, shooting angle, etc., traditional vehicle identification systems lead to large differences within categories and small differences between categories, making it difficult to achieve fine-grained vehicle classification, especially the vehicle logo recognition effect is not ideal, which affects the recognition rate.
By augmenting the vehicle sample images, building a training set, and using end-to-end training methods of backbone networks and classification head networks, the global and fine-grained feature information of the vehicle are extracted, and the recognition efficiency and accuracy of the vehicle recognition model are improved.
The recognition efficiency and accuracy of the vehicle recognition model are improved, the dependence on the extraction of small target positions of the vehicle is reduced, the ability to extract vehicle details is enhanced, and the accuracy of the identification of vehicle attribute types is improved.
Smart Images

Figure CN113191426B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a vehicle identification model creation method, a vehicle identification method, a device, a equipment and a storage medium. Background Art
[0002] With the rapid development of intelligent transportation systems, higher requirements are placed on vehicle management. At the entrance and exit of parking lots, vehicles can enter, exit and pay by identifying their license plates. However, for some vehicle violations, such as using duplicate license plates, vehicles can deceive smart surveillance cameras. For some similar license plates, when the license plate is blurred, the license plate recognition system may make mistakes. At this time, if the characteristic information of different attribute types of the vehicle can be added, the system's license plate recognition rate can be improved accordingly.
[0003] The same vehicle has a high intra-class difference due to a series of factors such as lighting, occlusion, and shooting angle in the monitoring area, while vehicles of different categories have a low inter-class difference due to their similar appearance. Traditional image classification algorithms are not ideal for fine-grained vehicle classification. For example, for the recognition of car logos, the position of the car logo is usually detected using a detection model first, and then the car logo is classified. However, the size, position, and angle of car logos of different brands are different, and the final recognition performance depends on the performance of the car logo detection, which greatly affects the final recognition rate. Summary of the invention
[0004] In view of this, the purpose of the present invention is to provide a vehicle identification model creation method, a vehicle identification method, an apparatus, a device and a storage medium, which can improve the efficiency and accuracy of vehicle identification. The specific scheme is as follows:
[0005] A first aspect of the present application provides a vehicle identification model creation method, comprising:
[0006] Acquire a first vehicle sample image, and perform a data enhancement operation on the first vehicle sample image to obtain a second vehicle sample image capable of representing fine-grained feature information of the first vehicle sample image;
[0007] Using information of different attribute types of vehicles in the first vehicle sample image to respectively annotate the first vehicle sample image and the second vehicle sample image to obtain corresponding sample labels, and construct a training set using the first vehicle sample image, the second vehicle sample image and the corresponding sample labels;
[0008] The training set is used to train a vehicle recognition model including a backbone network and a classification head network to obtain the trained vehicle recognition model; wherein the backbone network is used to extract features from the first vehicle sample image and the second vehicle sample image, and the classification head network is used to identify the extracted features to obtain information of different attribute types of the vehicle.
[0009] Optionally, performing a data enhancement operation on the first vehicle sample image to obtain a second vehicle sample image capable of representing fine-grained feature information of the first vehicle sample image includes:
[0010] The first vehicle sample image is divided into blocks based on a preset rule to obtain sub-sample image blocks, and the sub-sample image blocks are randomly spliced to obtain a second vehicle sample image with the same size as the first vehicle sample image.
[0011] Optionally, the dividing the first vehicle sample image into blocks based on a preset rule to obtain sub-sample image blocks further includes:
[0012] Performing random mirror processing and / or random rotation processing on the first vehicle sample image;
[0013] The processed first vehicle sample image is divided into blocks based on a preset rule to obtain sub-sample image blocks.
[0014] Optionally, the backbone network is specifically used to perform feature extraction on the first vehicle sample image and the second vehicle sample image respectively to obtain corresponding first feature maps and second feature maps; wherein the first feature map reflects the global feature information of the vehicle, and the second feature map reflects the fine-grained feature information of the vehicle;
[0015] Accordingly, the classification head network includes a first classification head network and a second classification head network, wherein:
[0016] The first classification head network is used to identify the mixed features of the vehicle, and the second classification head network is used to identify the fine-grained features of the vehicle in the second feature map to obtain information of different attribute types of the vehicle; wherein the mixed features are features obtained by mixing the global features of the vehicle in the first feature map and the fine-grained features of the vehicle in the second feature map.
[0017] Optionally, mixing the global features of the vehicle in the first feature map and the fine-grained features of the vehicle in the second feature map includes:
[0018] The concat function is used to calculate the global features of the vehicle in the first feature map and the fine-grained features of the vehicle in the second feature map.
[0019] Optionally, the first classification head network includes a vehicle logo classification head sub-network, a vehicle model classification head sub-network and a vehicle color classification head sub-network, and the second classification head network includes a vehicle logo classification head sub-network.
[0020] A second aspect of the present application provides a vehicle identification method, comprising:
[0021] Acquire an original image to be identified that contains a vehicle, and detect the vehicle in the original image to be identified using a target detection model to obtain a vehicle image to be identified that corresponds to the original image to be identified;
[0022] The vehicle image to be identified is input into the trained vehicle identification model so that the vehicle identification model can identify the vehicle and output information of different attribute types of the vehicle.
[0023] A third aspect of the present application provides a vehicle feature recognition model creation device, comprising:
[0024] A sample acquisition module, used to acquire a first vehicle sample image, and perform a data enhancement operation on the first vehicle sample image to obtain a second vehicle sample image capable of representing fine-grained feature information of the first vehicle sample image;
[0025] a training set construction module, configured to respectively label the first vehicle sample image and the second vehicle sample image using information of different attribute types of the vehicle in the first vehicle sample image to obtain corresponding sample labels, and construct a training set using the first vehicle sample image, the second vehicle sample image and the corresponding sample labels;
[0026] A training module is used to train a vehicle recognition model including a backbone network and a classification head network using the training set to obtain the trained vehicle recognition model; wherein the backbone network is used to extract features from the first vehicle sample image and the second vehicle sample image, and the classification head network is used to identify the extracted features to obtain information of different attribute types of the vehicle.
[0027] A fourth aspect of the present application provides an electronic device, comprising a processor and a memory; wherein the memory is used to store a computer program, and the computer program is loaded and executed by the processor to implement the aforementioned vehicle feature recognition model creation method.
[0028] The fifth aspect of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are loaded and executed by a processor, the aforementioned vehicle feature recognition model creation method is implemented.
[0029] In the present application, a first vehicle sample image is first obtained, and a data enhancement operation is performed on the first vehicle sample image to obtain a second vehicle sample image that can characterize the fine-grained feature information of the first vehicle sample image. Then, the first vehicle sample image and the second vehicle sample image are respectively annotated using the information of different attribute types of the vehicle in the first vehicle sample image to obtain the corresponding sample labels, and a training set is constructed using the first vehicle sample image, the second vehicle sample image and the corresponding sample labels. Finally, the vehicle recognition model including the backbone network and the classification head network is trained using the training set to obtain the trained vehicle recognition model; wherein the backbone network is used to extract features from the first vehicle sample image and the second vehicle sample image, and the classification head network is used to identify the extracted features to obtain information of different attribute types of the vehicle. The present application constructs a training set based on the data enhancement operation on the first vehicle sample image, and trains the backbone network and the classification head network of the vehicle recognition model, adopts an end-to-end training method, does not need to extract the specific location of small targets in the vehicle, and improves the recognition efficiency and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS
[0030] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.
[0031] Figure 1 A flow chart of a vehicle identification model creation method provided in this application;
[0032] Figure 2 A schematic diagram of a first vehicle sample image provided in this application;
[0033] Figure 3 A schematic diagram of a second vehicle sample image provided by the present application;
[0034] Figure 4 A flow chart of a vehicle identification method provided for this application;
[0035] Figure 5 A schematic diagram of the structure of a vehicle identification device provided in this application;
[0036] Figure 6 A structural diagram of a vehicle identification electronic device provided for this application. DETAILED DESCRIPTION
[0037] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0038] In the prior art, due to a series of influencing factors such as lighting, occlusion, and shooting angles in the monitoring area, the same vehicle has high intra-class differences, and vehicles of different categories have low inter-class differences due to similar appearances. Therefore, traditional image classification algorithms are not ideal for fine-grained vehicle classification. For example, for the recognition of car logos, the position of the car logo is usually detected by a detection model first, and then the car logo is classified. However, the size, position, and angle of car logos of different brands are different, and the final recognition performance depends on the performance of the car logo detection, which greatly affects the final recognition rate. In view of the above technical defects, a vehicle recognition model creation method is proposed. A training set is constructed on the basis of data enhancement operations on the first vehicle sample image, and the backbone network and classification head network of the vehicle recognition model are trained. An end-to-end training method is used, and there is no need to extract the specific positions of small targets in the vehicle, thereby improving the recognition efficiency and accuracy.
[0039] Figure 1 A flow chart of a vehicle identification model creation method provided in an embodiment of the present application. Figure 1 As shown, the vehicle identification model creation method includes:
[0040] S11: Acquire a first vehicle sample image, and perform a data enhancement operation on the first vehicle sample image to obtain a second vehicle sample image that can represent fine-grained feature information of the first vehicle sample image.
[0041] In this embodiment, a first vehicle sample image is first obtained, and then a data enhancement operation is performed on the first vehicle sample image to obtain a second vehicle sample image that can represent fine-grained feature information of the first vehicle sample image. Figure 2 As shown in (b), Figure 2 The box above the image in (a) is Figure 2 (b) The annotation box of the vehicle position, Figure 2 (b) is a small image of the vehicle captured based on the vehicle position annotation box, which is used to train the fine-grained model. Figure 2(a) is the original training data in this embodiment, which can be obtained by the surveillance camera deployed at the entrance and exit of the parking lot, that is, multiple pictures of vehicles passing through the gate captured by the camera. The picture data should be diverse, and the vehicle needs to include data under different scenes, lighting, angles, distances and weather conditions. In addition, in order to ensure that the trained vehicle recognition model has a high recognition accuracy, it is also necessary to screen the acquired picture data, requiring that at least the complete front part of the vehicle can be seen in the picture data, that is, the first vehicle sample image at least includes vehicles with complete front parts.
[0042] In order to improve the extraction of fine-grained features of the vehicle by the vehicle recognition model, this embodiment needs to perform data enhancement operations on the first vehicle sample image during the network training phase, and the data enhancement operations include but are not limited to random mirroring, random rotation, random cropping, etc. In particular, for the recognition of small targets such as car logos, the network needs to learn the detailed texture features of the vehicle. In order to further improve the fine-grained feature extraction capability of the vehicle recognition model, this embodiment adopts a random block method to obtain a second vehicle sample image that can characterize the fine-grained feature information of the first vehicle sample image, that is, the first vehicle sample image is divided into blocks based on a preset rule to obtain each sub-sample image block, and each sub-sample image block is randomly spliced to obtain a second vehicle sample image with the same size as the first vehicle sample image. The random block operation is to first divide the image into n*n sub-image blocks, and number the sub-image blocks in order, then randomly shuffle the order of the sub-image blocks, and finally splice the n*n sub-image blocks with the shuffled order into a picture of the same size as the original image. Specifically, the first vehicle sample image input into the network is first randomly divided into blocks, and the original structure of the image is disrupted, so that the network can pay more attention to the detailed texture features of the image. Figure 3 The shown is Figure 2 (b) The result of random partitioning, where Figure 3 (a) is to Figure 2 (b) The result of dividing into 4 blocks (2*2), Figure 3 (b) is to Figure 2 (b) The result of dividing into 16 blocks (4*4), Figure 3 (c) is to Figure 2(b) The result of being divided into 64 blocks (8*8). This embodiment uses three blocking methods to block the first vehicle sample image and obtain the corresponding three second vehicle sample images. Generally speaking, the sub-image obtained after three blocking operations can be considered as the smallest unit containing small vehicle target features such as vehicle logo features, but the blocking strategy can also be specifically set according to the size, resolution, etc. of the first vehicle sample image, and this embodiment does not limit this. In addition, in order to further improve the recognition efficiency of the vehicle recognition model, the first vehicle sample image can also be randomly mirrored and / or randomly rotated, and then the processed first vehicle sample image can be blocked based on preset rules to obtain each sub-sample image block.
[0043] S12: Annotate the first vehicle sample image and the second vehicle sample image respectively using information on different attribute types of the vehicle in the first vehicle sample image to obtain corresponding sample labels, and construct a training set using the first vehicle sample image, the second vehicle sample image and the corresponding sample labels.
[0044] In this embodiment, the first vehicle sample image and the second vehicle sample image are respectively labeled using the information of different attribute types of the vehicle in the first vehicle sample image to obtain corresponding sample labels. The information of different attribute types of the vehicle may be vehicle logo information, vehicle model information, vehicle color information, etc. Figure 2 The different types of vehicle attribute information shown in (a) are vehicle logo (Hyundai), vehicle model (sedan), and vehicle color (white). Figure 2 (b) Figure 3 (a) Figure 3 (b) Figure 3 (c) Annotate to obtain the corresponding sample labels. Then, the first vehicle sample image, the second vehicle sample image and the corresponding sample labels are used to construct a training set, that is, to use Figure 2 (b) Figure 3 (a) Figure 3 (b) Figure 3 (c) and its sample labels to construct a training set for training the vehicle recognition model. Of course, the training set includes a large number of Figure 2 (b) Figure 3 (a) Figure 3 (b) Figure 3 (c) and their sample labels.
[0045] S13: Using the training set to train a vehicle recognition model including a backbone network and a classification head network to obtain the trained vehicle recognition model; wherein the backbone network is used to extract features from the first vehicle sample image and the second vehicle sample image, and the classification head network is used to identify the extracted features to obtain information of different attribute types of the vehicle.
[0046] In this embodiment, the backbone network of the vehicle recognition model is used to extract features from the first vehicle sample image and the second vehicle sample image, and the classification head network is used to identify the extracted features to obtain information of different attribute types of the vehicle. Further, the backbone network is specifically used to extract features from the first vehicle sample image and the second vehicle sample image respectively to obtain corresponding first feature maps and second feature maps; wherein the first feature map reflects the global feature information of the vehicle, and the second feature map reflects the fine-grained feature information of the vehicle. Correspondingly, the classification head network includes a first classification head network and a second classification head network, the first classification head network is used to identify the mixed features of the vehicle, and the second classification head network is used to identify the fine-grained features of the vehicle in the second feature map to obtain information of different attribute types of the vehicle, wherein the mixed features are the features obtained by mixing the global features of the vehicle in the first feature map and the fine-grained features of the vehicle in the second feature map. Specifically, the concat function can be used to calculate the global features of the vehicle in the first feature map and the fine-grained features of the vehicle in the second feature map to obtain the mixed features.
[0047] In addition, when identifying the vehicle logo, model and color of a vehicle, the first classification head network includes a vehicle logo classification head sub-network, a vehicle model classification head sub-network and a vehicle color classification head sub-network, and the second classification head network includes a vehicle logo classification head sub-network. That is, the vehicle logo information, vehicle model information and vehicle color information of the first vehicle sample image are identified and output only through the first classification head network, and for the second vehicle sample image, the vehicle logo information is identified and output only through the second classification head network. The above-mentioned setting of the first classification head network and the second classification head network speeds up the inference speed of the model.
[0048] In this embodiment, the Figure 2 (b) Figure 3 (a) Figure 3 (b) Figure 3(c) is used as an example to illustrate the training process of the network structure of the vehicle recognition model in this embodiment. The training network includes the backbone network and the classification head network. The model learns the fine-grained features of the vehicle in the shallow layer and learns the larger-grained vehicle features in the deeper layer. The model derives 4 branch structures from the backbone network for connecting the subsequent classification head network. The 4 branch structures of the backbone network are Stage 1, Stage 2, Stage 3, and Stage 4, which represent different output feature maps of the backbone network. That is, first, Figure 3 (a) Figure 3 (b) Figure 3 (c) Figure 2 (b) Input to the backbone network and output Figure 3 (a) Figure 3 (b) Figure 3 (c) Figure 2 (b) A feature map including the vehicle logo, vehicle model and vehicle color features. Figure 3 (a) Output feature map, Figure 3 (b) Output feature map, Figure 3 (c) Output feature map, Figure 2 The output feature map of (b) is input to the Stage1, Stage2, Stage3, and Stage4 branches respectively. Each branch is connected to a convolutional block, which contains two convolutional layers, and finally connected to the classification head network for classification output. Figure 2 The output feature map of (b) is the first feature map, Figure 3 (a) Figure 3 (b) Figure 3 The output feature map of (c) is the second feature map. Figure 2 (b), the first classification head network connected includes a car logo classification head sub-network, a car model classification head sub-network and a car color classification head sub-network, which can be understood as a classifier for identifying the car logo, a classifier for identifying the car model and a classifier for identifying the car color. Figure 3 (a) Figure 3 (b) Figure 3(c), the second classification head network connected is the car logo classification head sub-network, that is, the classifier for identifying the car logo. Therefore, the output of Stage1 is y1_carlogo, the output of Stage2 is y2_carlogo, the output of Stage3 is y3_carlogo, and the output of Stage4 is y_concat_carlogo, y_concat_cartype, y_concat_carcolor, where y_concat_carlogo, y_concat_cartype, y_concat_carcolor are Figure 2 (b) Figure 3 (a) Figure 3 (b) Figure 3 (c) The features in the output feature map are obtained by concat calculation.
[0049] During the training process of the model, cross entropy is used as the loss function of the vehicle recognition model, as shown in the following formula:
[0050]
[0051] Where M is the number of categories, y i is an indicator variable (0 or 1), if the category is the same as the category of sample i, it is 1, otherwise it is 0, p i is the model's prediction probability of the category of sample i. In the learning process of the network, the shallow network has a smaller receptive field and its ability to represent features is limited. The network will use the texture details of the vehicle to distinguish information. The network can learn the local details of the vehicle. As the number of network layers increases, the deep network has a larger receptive field and the network can learn the global abstract features of the vehicle.
[0052] During the model testing process, the network's forward reasoning input uses the original vehicle image data, and the input image does not need to be randomly segmented, that is, the first vehicle sample image. In order to reduce the amount of model calculation and speed up the model reasoning, the fully connected layers of the Stage1, Stage2, and Stage3 branches are removed in the test phase. The model outputs in this phase are y_concat_carlogo, y_concat_cartype, and y_concat_carcolor.
[0053] It can be seen that the embodiment of the present application first obtains a first vehicle sample image, and performs a data enhancement operation on the first vehicle sample image to obtain a second vehicle sample image that can characterize the fine-grained feature information of the first vehicle sample image. Then, the first vehicle sample image and the second vehicle sample image are respectively annotated using the information of different attribute types of the vehicle in the first vehicle sample image to obtain corresponding sample labels, and a training set is constructed using the first vehicle sample image, the second vehicle sample image and the corresponding sample labels. Finally, the vehicle recognition model including the backbone network and the classification head network is trained using the training set to obtain the trained vehicle recognition model; wherein the backbone network is used to extract features from the first vehicle sample image and the second vehicle sample image, and the classification head network is used to identify the extracted features to obtain information of different attribute types of the vehicle. The embodiment of the present application constructs a training set based on the data enhancement operation on the first vehicle sample image, and trains the backbone network and the classification head network of the vehicle recognition model, adopts an end-to-end training method, does not need to extract the specific location of small targets in the vehicle, and improves recognition efficiency and accuracy.
[0054] Figure 4 A flow chart of a vehicle identification method provided in an embodiment of the present application. Figure 4 As shown, the vehicle identification method includes:
[0055] S21: Acquire an original image to be identified that contains a vehicle, and use a target detection model to detect the vehicle in the original image to be identified to obtain a vehicle image to be identified that corresponds to the original image to be identified.
[0056] S22: Inputting the image of the vehicle to be identified into a trained vehicle identification model, so that the vehicle identification model identifies the vehicle and outputs information of different attribute types of the vehicle.
[0057] In this embodiment, an original image to be identified containing a vehicle is obtained, and the vehicle in the original image to be identified is detected using a target detection model to obtain a vehicle image to be identified corresponding to the original image to be identified. The original image to be identified may be an image of a target vehicle entering or exiting a checkpoint captured in real time by a camera set at a checkpoint such as a parking lot, and the original image to be identified contains the vehicle to be detected. The target detection model can obtain the location information of the vehicle, and then obtain the corresponding vehicle image to be identified.
[0058] In this embodiment, the image of the vehicle to be identified is input into the trained vehicle identification model so that the vehicle identification model identifies the vehicle and outputs information of different attribute types of the vehicle. The trained vehicle identification model is a model obtained by the vehicle identification model creation method in the aforementioned embodiment. The backbone network in the trained vehicle identification model extracts features from the image of the vehicle to be identified, and the classification head network in the trained vehicle identification model identifies the extracted features to obtain information of different attribute types of the vehicle.
[0059] It can be seen that the embodiment of the present application first uses the target detection model to detect the vehicle in the original image to be identified to obtain the vehicle image to be identified corresponding to the original image to be identified. On this basis, the vehicle image to be identified is directly input into the trained vehicle recognition model, and an end-to-end recognition method is adopted. There is no need to extract the specific position of small targets of the vehicle such as the vehicle logo, thereby improving the recognition efficiency and accuracy.
[0060] See also Figure 5 As shown, the embodiment of the present application also discloses a vehicle identification model creation device, including:
[0061] The sample acquisition module 11 is used to acquire a first vehicle sample image and perform a data enhancement operation on the first vehicle sample image to obtain a second vehicle sample image that can represent fine-grained feature information of the first vehicle sample image;
[0062] A training set construction module 12 is used to label the first vehicle sample image and the second vehicle sample image respectively using information of different attribute types of the vehicle in the first vehicle sample image to obtain corresponding sample labels, and to construct a training set using the first vehicle sample image, the second vehicle sample image and the corresponding sample labels;
[0063] The training module 13 is used to train the vehicle recognition model including the backbone network and the classification head network using the training set to obtain the trained vehicle recognition model; wherein the backbone network is used to extract features from the first vehicle sample image and the second vehicle sample image, and the classification head network is used to identify the extracted features to obtain information of different attribute types of the vehicle.
[0064] It can be seen that the embodiment of the present application first obtains the first vehicle sample image, and performs data enhancement operation on the first vehicle sample image to obtain the second vehicle sample image that can characterize the fine-grained feature information of the first vehicle sample image. Then, the first vehicle sample image and the second vehicle sample image are respectively annotated using the information of different attribute types of the vehicle in the first vehicle sample image to obtain the corresponding sample labels, and the first vehicle sample image, the second vehicle sample image and the corresponding sample labels are used to construct a training set. Finally, the vehicle recognition model including the backbone network and the classification head network is trained using the training set to obtain the trained vehicle recognition model; wherein the backbone network is used to extract features from the first vehicle sample image and the second vehicle sample image, and the classification head network is used to identify the extracted features to obtain information of different attribute types of the vehicle. The embodiment of the present application constructs a training set based on the data enhancement operation on the first vehicle sample image, and trains the backbone network and the classification head network of the vehicle recognition model. An end-to-end training method is adopted, and the specific location of small targets in the vehicle does not need to be extracted, thereby improving the recognition efficiency and accuracy.
[0065] In some specific embodiments, the sample acquisition module 11 is specifically used to divide the first vehicle sample image into blocks based on preset rules to obtain sub-sample image blocks, and randomly splice the sub-sample image blocks to obtain a second vehicle sample image with the same size as the first vehicle sample image.
[0066] In some specific embodiments, the training set construction module 12 specifically includes:
[0067] A backbone network unit, configured to perform feature extraction on the first vehicle sample image and the second vehicle sample image respectively, so as to obtain a corresponding first feature map and a second feature map; wherein the first feature map reflects the global feature information of the vehicle, and the second feature map reflects the fine-grained feature information of the vehicle;
[0068] Correspondingly, the classification head network unit includes a first classification head network and a second classification head network, wherein the first classification head network is used to identify the mixed features of the vehicle, and the second classification head network is used to identify the fine-grained features of the vehicle in the second feature map to obtain information of different attribute types of the vehicle; wherein the mixed features are features obtained by mixing the global features of the vehicle in the first feature map and the fine-grained features of the vehicle in the second feature map.
[0069] Furthermore, an embodiment of the present application also provides an electronic device. Figure 6This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content in the diagram cannot be regarded as any limitation on the scope of use of the present application.
[0070] Figure 6 The present invention provides a schematic diagram of the structure of an electronic device 20 provided in an embodiment of the present application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is used to store a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the vehicle identification model creation method disclosed in any of the aforementioned embodiments.
[0071] In this embodiment, the power supply 23 is used to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and the external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs and is not specifically limited here.
[0072] In addition, the memory 22, as a carrier for storing resources, can be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon can include an operating system 221, a computer program 222 and vehicle data 223, etc. The storage method can be temporary storage or permanent storage.
[0073] The operating system 221 is used to manage and control the hardware devices and computer programs 222 on the electronic device 20, so as to realize the operation and processing of the massive data 223 in the memory 22 by the processor 21, which can be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program that can be used to complete the vehicle identification model creation method performed by the electronic device 20 disclosed in any of the aforementioned embodiments, the computer program 222 can further include a computer program that can be used to complete other specific tasks. The data 223 can include the image data of the vehicle collected by the electronic device 20.
[0074] Furthermore, an embodiment of the present application also discloses a storage medium, in which a computer program is stored. When the computer program is loaded and executed by a processor, the steps of the vehicle identification model creation method disclosed in any of the aforementioned embodiments are implemented.
[0075] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.
[0076] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.
[0077] The vehicle identification model creation method, vehicle identification method, device, equipment and storage medium provided by the present invention are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for general technical personnel in this field, according to the idea of the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.
Claims
1. A method for creating a vehicle identification model, characterized in that: include: Acquire a first vehicle sample image, and perform a data enhancement operation on the first vehicle sample image to obtain a second vehicle sample image capable of representing fine-grained feature information of the first vehicle sample image; the first vehicle sample image at least includes a vehicle with a complete front end; Using information of different attribute types of vehicles in the first vehicle sample image to respectively annotate the first vehicle sample image and the second vehicle sample image to obtain corresponding sample labels, and construct a training set using the first vehicle sample image, the second vehicle sample image and the corresponding sample labels; The vehicle recognition model including a backbone network and a classification head network is trained using the training set to obtain the trained vehicle recognition model; wherein the backbone network is used to extract features from the first vehicle sample image and the second vehicle sample image, and the classification head network is used to identify the extracted features to obtain information of different attribute types of the vehicle; in the model training process, cross entropy is used as the loss function of the vehicle recognition model; The performing of a data enhancement operation on the first vehicle sample image to obtain a second vehicle sample image capable of representing fine-grained feature information of the first vehicle sample image includes: Divide the first vehicle sample image into blocks based on a preset rule to obtain sub-sample image blocks, and randomly splice the sub-sample image blocks to obtain a second vehicle sample image with the same size as the first vehicle sample image; The backbone network is specifically used to perform feature extraction on the first vehicle sample image and the second vehicle sample image respectively to obtain a corresponding first feature map and a second feature map; wherein the first feature map reflects the global feature information of the vehicle, and the second feature map reflects the fine-grained feature information of the vehicle; Accordingly, the classification head network includes a first classification head network and a second classification head network, wherein: The first classification head network is used to identify the mixed features of the vehicle, and the second classification head network is used to identify the fine-grained features of the vehicle in the second feature map to obtain information of different attribute types of the vehicle; wherein the mixed features are features obtained by mixing the global features of the vehicle in the first feature map and the fine-grained features of the vehicle in the second feature map; Mixing the global features of the vehicle in the first feature map and the fine-grained features of the vehicle in the second feature map includes: The concat function is used to calculate the global features of the vehicle in the first feature map and the fine-grained features of the vehicle in the second feature map.
2. The vehicle identification model creation method according to claim 1, characterized in that: The step of dividing the first vehicle sample image into blocks based on a preset rule to obtain each sub-sample image block also includes: Performing random mirror processing and / or random rotation processing on the first vehicle sample image; The processed first vehicle sample image is divided into blocks based on a preset rule to obtain sub-sample image blocks.
3. The vehicle identification model creation method according to claim 1, characterized in that: The first classification head network includes a vehicle logo classification head sub-network, a vehicle model classification head sub-network and a vehicle color classification head sub-network, and the second classification head network includes a vehicle logo classification head sub-network.
4. A vehicle identification method, characterized in that: include: Acquire an original image to be identified that contains a vehicle, and detect the vehicle in the original image to be identified using a target detection model to obtain a vehicle image to be identified that corresponds to the original image to be identified; The vehicle image to be identified is input into the trained vehicle identification model as described in any one of claims 1 to 3, so that the vehicle identification model identifies the vehicle and outputs information of different attribute types of the vehicle.
5. A vehicle identification model creation device, characterized in that: include: A sample acquisition module, used to acquire a first vehicle sample image and perform a data enhancement operation on the first vehicle sample image to obtain a second vehicle sample image capable of representing fine-grained feature information of the first vehicle sample image; the first vehicle sample image at least includes a vehicle with a complete front end; a training set construction module, configured to respectively label the first vehicle sample image and the second vehicle sample image using information of different attribute types of the vehicle in the first vehicle sample image to obtain corresponding sample labels, and construct a training set using the first vehicle sample image, the second vehicle sample image and the corresponding sample labels; A training module, used to train a vehicle recognition model including a backbone network and a classification head network using the training set to obtain the trained vehicle recognition model; wherein the backbone network is used to extract features from the first vehicle sample image and the second vehicle sample image, and the classification head network is used to identify the extracted features to obtain information of different attribute types of the vehicle; in the process of model training, cross entropy is used as a loss function of the vehicle recognition model; The sample acquisition module is used to: divide the first vehicle sample image into blocks based on a preset rule to obtain sub-sample image blocks, and randomly splice the sub-sample image blocks to obtain a second vehicle sample image with the same size as the first vehicle sample image; The training set construction module specifically includes: A backbone network unit, specifically used to perform feature extraction on the first vehicle sample image and the second vehicle sample image respectively to obtain a corresponding first feature map and a second feature map; wherein the first feature map reflects the global feature information of the vehicle, and the second feature map reflects the fine-grained feature information of the vehicle; Correspondingly, the classification head network unit includes a first classification head network and a second classification head network, wherein the first classification head network is used to identify the mixed features of the vehicle, and the second classification head network is used to identify the fine-grained features of the vehicle in the second feature map to obtain information of different attribute types of the vehicle; wherein the mixed features are features obtained by mixing the global features of the vehicle in the first feature map and the fine-grained features of the vehicle in the second feature map; The classification head network unit is used to calculate the global features of the vehicle in the first feature map and the fine-grained features of the vehicle in the second feature map using a concat function.
6. An electronic device, characterized in that: The electronic device comprises a processor and a memory; wherein the memory is used to store a computer program, and the computer program is loaded and executed by the processor to implement the vehicle identification model creation method according to any one of claims 1 to 3.
7. A computer-readable storage medium, characterized in that: Used to store computer executable instructions, which, when loaded and executed by a processor, implement the vehicle identification model creation method according to any one of claims 1 to 3.
Citation Information
Cited By
Network training, and associated networks, models, systems, devices, methods, and computer-readable media
WO2026101515A1
