A vehicle recognition method and apparatus based on HV-LBP convolutional neural network

Abstract

This invention relates to the field of vehicle recognition technology, and discloses a vehicle recognition method and apparatus based on HV-LBP convolutional neural network. The method involves inputting a vehicle image, converting the original RGB image dataset to HSV color space, and simultaneously extracting LBP texture features from the RGB image. The saturation space of the HSV space is removed, and the LBP texture features are integrated into the H and V spaces, preserving hue and brightness while adding texture features to form a multi-feature fusion image. The HV-LBP image is then loaded into a Faster R-CNN network for training. Finally, a test dataset is used to perform vehicle recognition through the trained system, and the recognition accuracy is output. Compared with existing technologies, this invention improves vehicle recognition accuracy without increasing detection time or training samples by using the improved HV-LBP image as input, solving the problem of low recognition accuracy in existing technologies.

Description

Technical Field

[0001] This invention relates to the field of vehicle recognition technology, and specifically to a vehicle recognition method and apparatus based on HV-LBP convolutional neural network. Background Technology

[0002] Vehicle recognition refers to the technology of collecting information through hardware devices, transmitting the collected information on monitored road vehicles to a processing center, and automatically extracting and processing vehicle information to achieve vehicle identification. Vehicle recognition is a crucial component of modern intelligent transportation systems with wide-ranging applications. By analyzing vehicles, it enables vehicle type identification, vehicle location identification, vehicle detection, traffic flow statistics, vehicle attribute identification, and vehicle exterior damage identification, and is also a fundamental component of autonomous driving technology.

[0003] Many existing vehicle recognition methods are trained using raw RGB images as datasets, which can only extract raw feature information through convolutional neural networks, resulting in low recognition accuracy. Low recognition accuracy can lead to collisions between vehicles, for example, during autonomous driving when the vehicle in front cannot be correctly identified. Summary of the Invention

[0004] Purpose of the invention: To address the problems existing in the prior art, this invention provides a vehicle recognition method and device based on HV-LBP convolutional neural network. It uses multi-feature HV-LBP to fuse images to improve the accuracy of the training model and make the model more robust without increasing the number of training samples or the network.

[0005] Technical Solution: This invention provides a vehicle recognition method based on HV-LBP convolutional neural network, comprising the following steps:

[0006] Step 1: Input a vehicle image and convert the original image to RGB-HSV;

[0007] Step 2: Extract LBP texture features from the input image;

[0008] Step 3: Remove the saturation space of the HSV space and integrate the LBP texture features from Step 2 into the H and V spaces, so as to add texture features while maintaining hue and brightness, forming a multi-feature fusion image;

[0009] Step 4: Input the multi-feature fused image into the Faster R-CNN network for training. Before entering the neural network structure, the dataset is first classified, with 70% classified as training images, 20% as validation images, and the rest as test images. All images are then cropped to the same size: 224×224×3. The training images are then augmented by introducing a pre-trained ResNet50 model. After that, the RPN network is used to obtain proposal boxes, followed by ROI pooling layers, and finally, fully connected layers are used to output detection scores and locations for the object detection type.

[0010] Step 5: Vehicle Recognition and Verification: Input the image to be recognized, output the vehicle location and score, and simultaneously output the evaluation index of the detection system.

[0011] Furthermore, in step 1, the color space is converted from RGB to HSV:

[0012]

[0013] U=max(R,G,B), P=min(R,G,B).

[0014] Further, step 2, which involves extracting the LBP texture features from the input image, specifically involves:

[0015]

[0016] Where, x a y a It is the center pixel, i a It is the gray value of the center point, i b q is the grayscale value of adjacent pixels, q is the number of adjacent pixels at the center, and u is the function symbol.

[0017] Furthermore, the neural network model in step 4 is specifically as follows:

[0018] The backbone network uses a ResNet50 architecture with an input size of 224×224×3. After the first convolutional layer, a feature map of size 64×112×112 is output. A max-pooling layer then outputs a feature map of size 64×56×56. Three more convolutional layers increase the dimension to 256×56×56, followed by four more convolutional layers for further dimension increase and downsampling to obtain a 512×28×28 feature map. Six more convolutional layers then increase the dimension and downsampling to obtain a 1024×14×14 feature map. Finally, three more convolutional layers increase the dimension and downsampling to output a 2048×7×7 feature map. The feature map is then processed by average pooling to obtain a 2048×1×1 fully connected feature map. This 2048×1×1 feature map is then output through a 50-layer convolutional network. The output of this 2048×1×1 feature map is then fed into the RPN network, where a 3×3 sliding window is used to generate a 2048-dimensional feature map on the convolutional feature map. This 2048-dimensional feature map is then passed through fully connected layers to obtain the target probability and bounding box regression parameters. Simultaneously, the feature map obtained from ResNet50 is used for ROI pooling. The ROI candidate boxes obtained from the RPN network are then projected onto the feature map to obtain the corresponding feature matrix. Finally, the RPN loss and Fast-RCNN total loss correction parameters are calculated through fully connected layers to obtain the final prediction result.

[0019] Furthermore, the loss function of the neural network model is defined using binary cross-entropy loss:

[0020]

[0021] Where, Lf=-[q′ j ln(q j )+(1-q′ j )ln(1-q j )],q j Let q′ represent the probability that the j-th anchor is predicted to be the target. j This means that when the probability of a positive sample is 1, the probability of a negative sample is 0.

[0022]

[0023] w j =[w a w b w c w d ], w′ j =[w a ′, w b ′, w c ′, w d ′],w a =(aa) m ) / M m w b =(bbm ) / H m w c =ln(c / M) m ), w d =ln(d / H) m ), w a ′=(a′-a m ) / M m w b ′=(b′-b m ) / H m w c ′=ln(c′ / M m ).

[0024] Where w j w represents the regression parameters for predicting the position of the j-th neural network bounding box. j ′ represents the regression parameter for the position of the manually labeled box, w a w b w represents the center coordinate point. c w d Indicates width and height, a and b represent the coordinates of the predicted bounding box, c and d represent the width and height of the predicted bounding box, a m b m M represents the coordinates of the manually labeled box. m H m The coordinates represent the corresponding anchor coordinates, a′ and b′ represent the coordinates of the manually labeled box, and c′ and d′ represent the width and height of the manually labeled box.

[0025] The present invention also discloses a vehicle recognition device based on HV-LBP convolutional neural network, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the above-mentioned vehicle recognition method based on HV-LBP convolutional neural network.

[0026] Beneficial effects:

[0027] 1. This invention removes the saturation space from the HSV color space and integrates LBP texture features into the H and V spaces, increasing texture features while maintaining hue and brightness, thus forming a multi-feature fused image. Using multi-feature HV-LBP fused images improves the accuracy of the training model and enhances its robustness without increasing the number of training samples or the network.

[0028] 2. This invention achieves good test results on the Faster R-CNN network structure using residual network ResNet50 (which increases the number of network layers to obtain deeper foreground target characteristics) as the backbone network structure, and uses residual convolution to reduce the number of parameters and computation during the calculation process, ensuring that the detection time does not increase. Attached Figure Description

[0029] Figure 1 This is the convolutional neural network structure of the present invention;

[0030] Figure 2 This is a diagram of the RPN network structure of the feature extraction network of the present invention;

[0031] Figure 3 This is a diagram of the ResNet50 network structure of the feature extraction network of this invention;

[0032] Figure 4 The test results are for vehicle recognition using the multi-feature fusion convolutional neural network of this invention;

[0033] Figure 5 The results are from a test using a general method for vehicle identification. Detailed Implementation

[0034] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention.

[0035] like Figure 1 As shown, this invention discloses a vehicle recognition method and apparatus based on HV-LBP convolutional neural networks. The recognition method includes the following steps:

[0036] (1) Input a vehicle image, perform HSV conversion on the original image, converting it to the H, S, V color space, which consists of three components: hue (H), saturation (S), and value (V). Original RGB image to HSV conversion:

[0037]

[0038] U=max(R,G,B), P=min(R,G,B)

[0039] (2) Extract LBP texture features from the image. These features have the advantages of rotation invariance and grayscale invariance. The feature extraction method is as follows:

[0040]

[0041] Where xa y a It is the center pixel, i a It is the gray value of the center point, i b q is the grayscale value of adjacent pixels, q is the number of adjacent pixels at the center, and u is the function symbol.

[0042] Because LBP texture features are highly robust to lighting conditions, meaning the extracted maps remain consistent under both strong and low light conditions, this reduces the problem of low recognition accuracy caused by unstable ambient lighting during vehicle information extraction.

[0043] (3) Fuse multi-feature images, remove the saturation space of HSV space, and integrate LBP texture features into H and V spaces, so that texture features are added while maintaining hue and brightness, forming a multi-feature fused image.

[0044] (4) The multi-feature fused image is input into the Faster R-CNN network for training. Data augmentation is performed on the training dataset through data preprocessing (the amount of training sample data is increased by randomly horizontally flipping images), and then the input image size is cropped to 224×224×3. Then, the image is used... Figure 3 The ResNet50 is used as the backbone network. After the first convolutional layer, a feature map of size 64×112×112 is output. After max pooling, a feature map of size 64×56×56 is output. Then, three convolutional layers increase the dimension to 256×56×56, followed by four more convolutional layers for further dimension increase and downsampling to obtain a 512×28×28 feature map. Six more convolutional layers then increase the dimension and downsampling to obtain a 1024×14×14 feature map. Three more convolutional layers then increase the dimension and downsampling to output a 2048×7×7 feature map. Finally, average pooling yields a 2048×1×1 fully connected feature map, which then enters the next phase. Figure 2 The RPN network shown uses a 3×3 sliding window to generate feature maps on the convolutional feature maps. These feature maps are then processed by 1×1 convolutional kernels to obtain classification scores and bounding box positions. Afterwards, ROI_Polling flattens the map to obtain ROI object probabilities and bounding box position regression parameters. Simultaneously, the feature map obtained from ResNet-50 is projected onto the feature map using the ROI candidate box positions obtained from the RPN network to obtain the corresponding feature matrix. Finally, a fully connected layer is used to calculate the RPN loss and the Fast-RCNN total loss correction parameters to obtain the final prediction result.

[0045] The loss function is defined using binary cross-entropy loss:

[0046] RPN loss:

[0047] Fast R-CNN loss: L(e, f, g) v h)=L b (e, f) + α[f≥1]L b (g v h)

[0048] That is, the total loss function:

[0049]

[0050] Among them, L f =-[q′ j ln(q j )+(1-q′ j )ln(1-q j )],q j Let q′ represent the probability that the j-th anchor is predicted to be the target. j This means that when the probability of a positive sample is 1, the probability of a negative sample is... w j =[w a w b w c w d ], w j ′=[w a ′, w b ′, w c ′, w d ′],w a =(aa) m ) / M m w b =(bb m ) / H m w c =ln(c / M) m ), w d =ln(d / H) m ), w a ′=(a′-a m ) / M m w b ′=(b′-b m ) / H m w c ′=ln(c′ / M m ).

[0051] Where w j w represents the regression parameters for predicting the position of the j-th neural network bounding box. j ′ represents the regression parameter for the position of the manually labeled box, w a w b w represents the center coordinate point. c w dIndicates width and height, a and b represent the coordinates of the predicted bounding box, c and d represent the width and height of the predicted bounding box, a m b m M represents the coordinates of the manually labeled box. m H m The coordinates represent the corresponding anchor coordinates, a′ and b′ represent the coordinates of the manually labeled box, and c′ and d′ represent the width and height of the manually labeled box.

[0052] (5) Vehicle Recognition and Verification: Input a photo to be recognized, and through the trained model, output the vehicle location and score. Precision (P), recall (R), and PR curves are used to evaluate network performance, such as... Figure 4 and Figure 5 As shown in the diagram. Precision is the ratio of correctly predicted samples to the total number of samples included in the prediction. Recall is the ratio of correctly predicted samples to the total number of samples. The PR curve is a graph with recall on the x-axis and precision on the y-axis. The closer the precision is to 1, the better the network performance. Figure 4 and Figure 5 The comparison shows that network performance was significantly optimized after using this invention.

[0053] The above embodiments are only for illustrating the technical concept and features of the present invention, and are intended to enable those skilled in the art to understand the content of the present invention and implement it accordingly. They should not be construed as limiting the scope of protection of the present invention. All equivalent transformations or modifications made in accordance with the spirit and essence of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A vehicle recognition method based on HV-LBP convolutional neural network, characterized in that, Includes the following steps: Step 1: Input a vehicle image and convert the original image to RGB-HSV; In step 1, the color space is converted from RGB to HSV: Step 2: Extract LBP texture features from the input image; The LBP texture features of the input image are as follows: Where, x a y a It is the center pixel, i a It is the gray value of the center point, i b is the grayscale value of adjacent pixels, q is the number of the center adjacent pixels, and u is the function symbol; Step 3: Remove the saturation space of the HSV space and integrate the LBP texture features from Step 2 into the H and V spaces, so as to add texture features while maintaining hue and brightness, forming a multi-feature fusion image; Step 4: Input the multi-feature fused image into the Faster R-CNN network for training. Before entering the neural network structure, the dataset is first classified, with 70% classified as training images, 20% as validation images, and the rest as test images. All images are then cropped to the same size: 224×224×3. The training images are then augmented by introducing a pre-trained ResNet50 model. After that, the RPN network is used to obtain proposal boxes, followed by ROI pooling layers, and finally, fully connected layers are used to output detection scores and locations for the object detection type. Step 5: Vehicle Recognition and Verification: Input the image to be recognized, output the vehicle location and score, and simultaneously output the evaluation index of the detection system.

2. The vehicle recognition method based on HV-LBP convolutional neural network according to claim 1, characterized in that, The neural network model in step 4 is specifically as follows: The backbone network uses a ResNet50 architecture with an input size of 224×224×3. After the first convolutional layer, a feature map of size 64×112×112 is output. A max-pooling layer then outputs a feature map of size 64×56×56. Three more convolutional layers increase the dimension to 256×56×56, followed by four more convolutional layers for further dimension increase and downsampling to obtain a 512×28×28 feature map. Six more convolutional layers then increase the dimension and downsampling to obtain a 1024×14×14 feature map. Finally, three more convolutional layers increase the dimension and downsampling to output a 2048×7×7 feature map. The feature map is then processed by average pooling to obtain a 2048×1×1 fully connected feature map. This 2048×1×1 feature map is then output through a 50-layer convolutional network. The output of this 2048×1×1 feature map is then fed into the RPN network, where a 3×3 sliding window is used to generate a 2048-dimensional feature map on the convolutional feature map. This 2048-dimensional feature map is then passed through fully connected layers to obtain the target probability and bounding box regression parameters. Simultaneously, the feature map obtained from ResNet50 is used for ROI pooling. The ROI candidate boxes obtained from the RPN network are then projected onto the feature map to obtain the corresponding feature matrix. Finally, the RPN loss and Fast-RCNN total loss correction parameters are calculated through fully connected layers to obtain the final prediction result.

3. The vehicle recognition method based on HV-LBP convolutional neural network according to claim 2, characterized in that, The loss function of the neural network model is defined using binary cross-entropy loss: Among them, L f =-[q j ln(q) j )+(1-q j ′)ln(1-q j )],q j Let q′ represent the probability that the j-th anchor is predicted to be the target. j This means that when the probability of a positive sample is 1, the probability of a negative sample is 0. w j =[w a ,w b ,w c ,w d ],w j ′=[w a ′,w b ′,w c ′,w d ′],w a =(a-a m ) / M m ,w b =(b-b m ) / H m , w c =ln(c / M m ),w d =ln(d / H m ),w a ′=(a′-a m ) / M m ,w b ′=(b′-b m ) / H m ,w c ′=ln(c′ / M m ), Where w j w represents the regression parameters for predicting the position of the j-th neural network bounding box. j ′ represents the regression parameter for the position of the manually labeled box, w a w b w represents the center coordinate point. c w d Indicates width and height, a and b represent the coordinates of the predicted bounding box, c and d represent the width and height of the predicted bounding box, a m b m M represents the coordinates of the manually labeled box. m H m The coordinates represent the corresponding anchor coordinates, a′ and b′ represent the coordinates of the manually labeled box, and c′ and d′ represent the width and height of the manually labeled box.

4. A vehicle recognition device based on HV-LBP convolutional neural network, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements a vehicle recognition method based on HV-LBP convolutional neural network as described in any one of claims 1-3.

Images