An efficient fine-grained vehicle recognition method based on deep learning

Abstract

The present invention discloses an efficient vehicle fine-grained recognition method based on deep learning, which extracts vehicle image features by constructing an efficient residual structure, and then improves the fine-grained classification ability of the feature extraction network through a spatial channel loss function, so that the network is more focused on different component areas of the vehicle image, ensuring the distinguishability and discernibility of the feature channels, and does not increase the computational complexity of convolutional neural network reasoning. The present invention automatically extracts features through deep learning, avoids the limitations of manually designed feature areas, can better overcome the interference of complex environmental noise in vehicle images, and can improve the accuracy of fine-grained vehicle recognition by focusing on richer feature areas through a spatial feature loss function.

Description

Technical Field

[0001] The present invention relates to the field of intelligent transportation technology, and in particular to an efficient vehicle fine-grained recognition method based on deep learning, which is a method for realizing vehicle fine-grained classification through a convolutional neural network. Background Art

[0002] With the rapid development of my country's transportation industry, the vehicle video information collected by road cameras has formed a massive data storage. The fine-grained vehicle classification technology based on image processing is an important technology in the field of intelligent transportation, which provides significant help for video surveillance and law enforcement departments.

[0003] The existing fine-grained vehicle classification technology mainly focuses on identifying the type of vehicle, such as the Volkswagen Passat 2007. The vehicle recognition method needs to ensure the clarity of the overall outline of the vehicle. In addition, the existing vehicle recognition technology mostly uses traditional extraction methods, such as gradient directional histogram, local binary pattern texture, support vector machine, etc. The invention patent (application number: CN201310416016.6, name: vehicle brand recognition method and system based on image) detects the position of the license plate in the input image, calculates the position of multiple vehicle parts according to the license plate position, extracts feature vectors from each of the multiple component positions, and classifies the extracted feature vectors and outputs the brand information of the vehicle. This method extracts the features of different components separately for recognition, and the detection accuracy is low and the recognition speed is slow; the invention patent (application number: CN202110514090.6, name: a fine-grained vehicle classification method based on channel grouping attention model) uses channel grouping and attention module to build and train the network to improve the network's attention to the discrimination area. However, this method cannot ensure that the network pays attention to the global information of vehicle features, and it is easy to miss some component features. Summary of the invention

[0004] In order to overcome the above-mentioned shortcomings of the prior art, the purpose of the present invention is to provide an efficient vehicle fine-grained recognition method based on deep learning, which improves the fine-grained classification ability of the feature extraction network through the loss function, makes the network more focused on different component areas of the vehicle image, ensures the distinguishability and discernibility of the feature channels, and does not increase the computational complexity of the convolutional neural network reasoning.

[0005] The present invention provides an efficient vehicle fine-grained recognition method based on deep learning, which improves the fine-grained classification capability of the feature extraction network through a loss function, so that the network can focus more on different component areas of the vehicle image, and is characterized by comprising the following steps:

[0006] Step 1: Normalize the test samples and training samples of vehicle images to 224×224 pixels;

[0007] Step 2: Use the training samples to train the convolutional neural network model and extract deep feature information of the vehicle image;

[0008] Step 3: During inference, only the feature vector is normalized using formula (1) to obtain the probability distribution of different categories;

[0009]

[0010] Among them, x i represents the feature vectors of different vehicle categories obtained in the above steps, i represents the vehicle category index, C represents the total number of vehicle categories, and e represents a natural number;

[0011] Step 4: Obtain the probability list of different vehicle categories according to the above steps, and output the vehicle category with the maximum probability.

[0012] Furthermore, the present invention further defines the step 2 of using the training samples to train the convolutional neural network model to extract deep feature information of the vehicle image, which specifically includes the following steps:

[0013] Step 2.1: Use 3×3 convolution kernels to perform random and sparse connections on the feature dimension, fuse the image spatial position information into the low-scale feature map, and increase the feature channel depth from 3 channels to 32 channels;

[0014] Step 2.2: Activate and normalize the feature information of different depths through ReLu;

[0015] Step 2.3: The feature map obtained in the above steps is sent to the efficient residual structure, and after multiple feature extractions, the feature map F is obtained as the lower layer input;

[0016] Step 2.4: Calculate the loss of feature map F through the spatial channel loss function;

[0017] Furthermore, the present invention also defines that the efficient residual structure is established in step 2.3, which specifically includes the following steps:

[0018] Step 2.3.1: Expand the channel depth through a 1×1 convolution kernel to increase the spatial correlation connection between different channels, obtain a deep feature map with rich inter-channel information, and activate and normalize it through ReLu;

[0019] Step 2.3.2 performs a group convolution operation of a 3×3 convolution kernel on the above feature map, performs random sparse connections on the channel features within the group, reduces the connection density and parameter complexity of the convolution operation, and performs activation and normalization operations through ReLu;

[0020] Step 2.3.3 reduces the number of channels through 1×1 convolution to restore the channel depth of feature information and avoid redundancy in extracted features;

[0021] Step 2.3.4 simultaneously sends the original feature map input into step 2.3.1 to the filter of 1×1 convolution kernel to obtain the feature map with rich spatial related information, and normalizes it;

[0022] Step 2.3.5 performs a cascade operation on the feature maps obtained in step 2.3.3 to fuse them into deep feature information containing rich dimensions;

[0023] Step 2.3.6 shuffles the feature maps obtained in step 2.3.4 and step 2.3.5, adjusts the channel arrangement order of the deep feature map, increases the exchange of information between channels, enriches the feature information within the group, and increases the expression ability of spatial correlation;

[0024] Furthermore, the present invention also defines the establishment of a spatial channel loss function in step 2.4, which specifically includes the following steps:

[0025] Step 2.4.1 Group the feature channels. Different groups of feature channels focus on extracting features of different region types. Each group of feature channels should be representative for each region. The focus loss L dis Calculated by the following formula:

[0026] L dis (F)=L ce (y,f(g(F))) (2)

[0027]

[0028]

[0029] Among them, L ce represents the cross entropy loss function, i represents the index of the vehicle type, y represents the true label, C represents the number of categories, F represents the feature map, W represents the width of the feature map, H represents the height of the feature map, η represents the depth of the feature map, j represents the index of each layer of the feature map, and max represents the maximum value of the set;

[0030] Step 2.4.2: Take the maximum response point of each feature map as the response center point p of the current feature map. According to the proportion of the minimum distance of the highest response point of the feature map to the average of all distances, the average distance coefficient D(p) is calculated by formula (5). This represents the difference in feature grouping. The smaller the average distance coefficient of the response point, the greater the loss, and the resolution loss L div Calculated by formula (6):

[0031]

[0032] L div=Exp(-D(p)) (6)

[0033] Among them, p represents the coordinate vector p of the feature response center point i ,p j , dist represents the Euclidean distance, min represents the minimum value of the set, and k represents the number of feature response center points;

[0034] Step 2.4.3 Based on the calculated concentration loss L dis and resolution loss L div The sum is used as the total loss to train the network.

[0035] By adopting the above technology, compared with the prior art, the beneficial effects of the present invention are as follows: the present invention automatically extracts features through deep learning, avoids the limitations of manually designed feature areas, can better overcome the interference of complex environmental noise in vehicle images, and can use the spatial feature loss function to make the features focus on richer feature areas, thereby improving the accuracy of fine-grained vehicle recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 Vehicle images selected for the embodiment of the present invention. DETAILED DESCRIPTION

[0037] The present invention is further described below through examples. Usually, the image of the vehicle sample is acquired by an image acquisition device, and the technical solution of the embodiment of the present application is applicable to the application scenario of vehicle type recognition of the vehicle sample image. The technical solution of the embodiment of the present application is adopted to improve the fine-grained classification ability of the feature extraction network through the loss function, so that the network can focus more on the different component areas of the vehicle image, ensure the distinguishability and discernibility of the feature channel, and do not increase the computational complexity of the convolutional neural network reasoning.

[0038] The vehicle image selected by the embodiment of the present invention is obtained by an image acquisition device, such as Figure 1 As shown, the efficient vehicle fine-grained classification method based on deep learning of the present invention specifically includes the following steps:

[0039] Step 1: Normalize the test samples and training samples of vehicle images to 224×224 pixels;

[0040] Step 2: Use the training samples to train the convolutional neural network model and extract the deep feature information of the vehicle image, specifically:

[0041] Step 2.1: Use 3×3 convolution kernels to perform random and sparse connections on the feature dimension, fuse the image spatial position information into the low-scale feature map, and increase the feature channel depth from 3 channels to 32 channels;

[0042] Step 2.2: Activate and normalize the feature information of different depths through ReLu to obtain the feature map;

[0043] Step 2.3: The feature map obtained in step 2.2 is sent to the efficient residual structure. After multiple feature extractions, the feature map F is obtained as the lower layer input, specifically:

[0044] Step 2.3.1: Expand the channel depth through 1×1 convolution kernel, increase the spatial correlation connection between different channels, obtain a deep feature map with rich inter-channel information, and activate and normalize it through ReLu;

[0045] Step 2.3.2: Perform a group convolution operation of 3×3 convolution kernel on step 2.3.1, perform random sparse connections on the channel features within the group, reduce the connection density and parameter complexity of the convolution operation, and perform activation and normalization operations through ReLu;

[0046] Step 2.3.3: Reduce the number of channels through 1×1 convolution to restore the channel depth of feature information and avoid redundancy in extracted features;

[0047] Step 2.3.4: At the same time, the original feature map input to step 2.3.1 is sent to the filter of the 1×1 convolution kernel to obtain a feature map with rich spatial related information and normalize it;

[0048] Step 2.3.5: Perform a cascade operation on the feature maps obtained in step 2.3.3 to fuse them into deep feature information containing rich dimensions;

[0049] Step 2.3.6: Shuffle the feature maps obtained in step 2.3.4 and step 2.3.5, adjust the channel arrangement order of the deep feature map, increase the exchange of information between channels, enrich the feature information within the group, and increase the expression ability of spatial correlation;

[0050] Step 2.4: Calculate the loss of feature map F through the spatial channel loss function, specifically:

[0051] Step 2.4.1 Group the feature channels. Different groups of feature channels focus on extracting features of different region types. Each group of feature channels should be representative for each region. The focus loss L dis Calculated by the following formula:

[0052] L dis (F)=L ce (y,f(g(F))) (1)

[0053]

[0054]

[0055] Among them, L ce represents the cross entropy loss function, i represents the index of the vehicle type, y represents the true label, C represents the number of categories, F represents the feature map, W represents the width of the feature map, H represents the height of the feature map, η represents the depth of the feature map, j represents the index of each layer of the feature map, and max represents the maximum value of the set;

[0056] Step 2.4.2: Take the maximum response point of each feature map as the response center point p of the current feature map. Calculate the average distance coefficient D(p) according to the proportion of the minimum distance of the highest response point of the feature map to the average of all distances. The calculation is shown in formula (4). This represents the difference in feature grouping. The smaller the average distance coefficient of the response point, the greater the loss, and the resolution loss L div Calculated by formula (5), specifically:

[0057]

[0058] L div =Exp(-D(p)) (5)

[0059] Among them, p represents the coordinate vector p of the feature response center point i ,p j , dist represents the Euclidean distance, min represents the minimum value of the set, and k represents the number of feature response center points;

[0060] Step 2.4.3 Based on the calculated concentration loss L dis and resolution loss L div Add them together as the total loss to train the network;

[0061] Step 3: During inference, only the feature vector is normalized using formula (6) to obtain the probability distribution of different categories;

[0062]

[0063] Wherein, xi represents the feature vectors of different vehicle categories obtained in the above steps, i represents the vehicle category index, C represents the total number of vehicle categories, and e represents a natural number;

[0064] Step 4: Obtain the probability list of different vehicle categories according to the above steps, and output the vehicle category with the maximum probability.

[0065] The contents described in the embodiments of this specification are merely an enumeration of implementation forms of the inventive concept. The protection scope of the present invention should not be regarded as limited to the specific forms described in the embodiments. The protection scope of the present invention also extends to equivalent technical means that can be conceived by those skilled in the art based on the inventive concept.

Claims

1. An efficient vehicle fine-grained recognition method based on deep learning, which improves the fine-grained classification ability of the feature extraction network through the loss function, so that the network can focus more on different component areas of the vehicle image, characterized by The steps include: Step 1: Normalize the test samples and training samples of vehicle images to 224×224 pixels; Step 2: Use the training samples to train the convolutional neural network model to extract the deep feature information of the vehicle image, which specifically includes the following steps: Step 2.1: Use 3×3 convolution kernels to perform random and sparse connections on the feature dimension, fuse the image spatial position information into the low-scale feature map, and increase the feature channel depth from 3 channels to 32 channels; Step 2.2: Activate and normalize the feature information of different depths through ReLu; Step 2.3: The feature map obtained in the above steps is sent to the efficient residual structure, and after multiple feature extractions, the feature map F is obtained as the lower layer input; Step 2.4: Calculate the loss of the feature map F through the spatial channel loss function, which specifically includes the following steps: Step 2.4.1 Group the feature channels. Different groups of feature channels focus on extracting features of different region types. Each group of feature channels should be representative for each region. The focus loss L dis Calculated by the following formula: L dis (F)=L ce (y,f(g(F))) (2) Among them, L ce represents the cross entropy loss function, i represents the index of the vehicle type, y represents the true label, C represents the number of categories, F represents the feature map, W represents the width of the feature map, H represents the height of the feature map, η represents the depth of the feature map, j represents the index of each layer of the feature map, and max represents the maximum value of the set; Step 2.4.2: Take the maximum response point of each feature map as the response center point p of the current feature map. According to the proportion of the minimum distance of the highest response point of the feature map to the average of all distances, the average distance coefficient D(p) is calculated by formula (5). This represents the difference in feature grouping. The smaller the average distance coefficient of the response point, the greater the loss, and the resolution loss L div Calculated by formula (6): L div =Exp(-D(p)) (6) Among them, p represents the coordinate vector p of the feature response center point i ,p j , dist represents the Euclidean distance, min represents the minimum value of the set, and k represents the number of feature response center points; Step 2.4.3 Based on the calculated concentration loss L dis and resolution loss L div Add them together as the total loss to train the network; Step 3: During inference, only the feature vector is normalized using formula (1) to obtain the probability distribution of different categories; Among them, x i represents the feature vectors of different vehicle categories obtained in the above steps, i represents the vehicle category index, C represents the total number of vehicle categories, and e represents a natural number; Step 4: Obtain the probability list of different vehicle categories according to the above steps, and output the vehicle category with the maximum probability.

2. The efficient vehicle fine-grained recognition method based on deep learning according to claim 1 is characterized in that In step 2.3, an efficient residual structure is established, which specifically includes the following steps: Step 2.3.1: Expand the channel depth through a 1×1 convolution kernel to increase the spatial correlation connection between different channels, obtain a deep feature map with rich inter-channel information, and activate and normalize it through ReLu; Step 2.3.2 performs a group convolution operation of a 3×3 convolution kernel on the above feature map, performs random sparse connections on the channel features within the group, reduces the connection density and parameter complexity of the convolution operation, and performs activation and normalization operations through ReLu; Step 2.3.3 reduces the number of channels through 1×1 convolution to restore the channel depth of feature information and avoid redundancy in extracted features; Step 2.3.4 simultaneously sends the original feature map input into step 2.3.1 to the filter of 1×1 convolution kernel to obtain the feature map with rich spatial related information, and normalizes it; Step 2.3.5 performs a cascade operation on the feature maps obtained in step 2.3.3 to fuse them into deep feature information containing rich dimensions; Step 2.3.6 shuffles the feature maps obtained in steps 2.3.4 and 2.3.5, adjusts the channel arrangement order of the deep feature map, increases the exchange of information between channels, enriches the feature information within the group, and increases the expression ability of spatial correlation.

Images