A vehicle boundary positioning method for night scene

Abstract

The application belongs to the technical field of computer vision, and discloses a vehicle boundary positioning method for night scene, which collects image data of a night road scene, uses a detection model to mark the boundaries of preprocessed image data, trains the detection model using a labeled data training set, obtains a final teacher model, constructs a student model, distills boundary positioning knowledge learned by the teacher model using a positioning distillation method, and finally obtains a lightweight network model, and finally obtains the position of a vehicle in a night scene. The positioning distillation method effectively solves the problem of fuzzy boundary positioning and improves the positioning accuracy of the detection model. When arranged in a vehicle-mounted 360-degree panoramic image system, the method can realize relatively accurate positioning through real-time detection of surrounding vehicles, so as to avoid traffic accidents or provide better implementation conditions for automatic driving.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of computer vision, and particularly relates to a vehicle boundary positioning method for a night scene. BACKGROUND

[0002] In today's development and popularization of artificial intelligence technology, computer vision technology has gradually become a hot research field in the field of artificial intelligence. Among them, image processing technology has been widely and deeply applied in the field of intelligent transportation, such as self-driving cars, unmanned delivery vehicles, road traffic monitoring, etc., which makes the vehicle detection system based on deep learning method more and more valued. In the field of computer vision, the determination of the vehicle boundary box position is an important problem, aiming to accurately position the vehicle on the image and determine the size; however, the current night image recognition system still has the problem of poor recognition effect, the main reason of which is that the light source such as the car light of the oncoming vehicle or the street lamp on the road affects the system. The light distribution is uneven at night, and vehicles without street lamps or far away will be darker, while vehicles close by may appear too bright due to the oncoming vehicle's car light or street light. These changes in brightness and darkness make the night vehicle recognition system prone to errors, with low recognition accuracy, blurred vehicle boundary box, and easy integration of the vehicle boundary with the night environment background, which makes it difficult to define the boundary, resulting in decreased positioning accuracy of the vehicle boundary box, etc. In addition, the mobile terminal hardware computing power of the vehicle is generally low, and the network model requires a relatively lightweight model. These all seriously affect the detection accuracy and real-time performance of the vehicle boundary detection algorithm. Therefore, how to achieve accurate positioning of the vehicle boundary box in the night scene and make the network model as small as possible is an urgent need for current computer vision research.

[0003] Patent application CN115171079A discloses a vehicle detection method based on a night scene, which uses a dynamic filtering network to generate a sample-specific convolution kernel, and uses different enhancement methods to constrain each enhancement sub-network for different night image samples; but the model implemented by it does not take lightweight measures, and the volume of the model itself will be larger, requiring relatively strong computing power. When actually deployed to the vehicle, due to the low computing power of the vehicle-mounted device, the detection speed may decrease, and even the detection may fail, so it cannot be deployed to similar low-computing-power terminals; it also does not optimize the fuzziness of the boundary in the night environment. In actual scenarios, when encountering large vehicles such as trucks, the vehicle boundary cannot be accurately positioned, which may lead to a series of traffic accidents due to inaccurate judgment of the vehicle boundary. SUMMARY

[0004] To solve the above technical problems, the present application provides a vehicle boundary positioning method for a night scene, which enhances the night vehicle boundary positioning effect while lightening the network model.

[0005] The application discloses a vehicle boundary positioning method for a night scene, and comprises the following steps:

[0006] S1, collecting image data of a night road scene;

[0007] S2, dividing the collected image data into a training set, a verification set and a test set, and preprocessing the image data;

[0008] S3, using a detection model to mark a boundary of the preprocessed image data to obtain a labeled image data set, and using the labeled image data set and the training set to train the detection model to obtain a final teacher model;

[0009] S4, constructing a student model, and using a positioning distillation method to distill boundary positioning knowledge learned by the teacher model to finally obtain a lightweight network model;

[0010] S5, inputting the test set data into the lightweight network model to output position coordinate information of a vehicle, setting a threshold to filter a positioning frame, using non-maximum suppression to delete repeated positioning frames, and finally obtaining a vehicle position in a night scene.

[0011] In the implementation process, the selected model can be replaced, and the teacher model can be any larger detection model with more parameters.

[0012] Further, S2 is specifically as follows:

[0013] Image data is collected by using a vehicle-mounted camera or a road monitoring camera, and an image captured by a camera is originally in a RAW format, which records original information of a camera sensor and contains shutter speed and aperture values of the camera; the original format is used as input by using a deep learning network, which is equivalent to introducing the information, and a conventional RGB format picture after light enhancement can be better output.

[0014] Further, S3 is specifically as follows:

[0015] A detection model is used to mark the image data to obtain a marking result, and the marking result is corrected; the label obtained is coordinate information of a positioning frame, that is, coordinate points of four corners of a rectangular frame, which can be visualized by using a tool; an erroneous positioning frame can be seen by naked eyes, and the boundary frame does not frame a target or has a large position deviation, and the erroneous positioning frame is deleted by manual operation.

[0016] The corrected marking data is used as training data to continue training the detection model until the detection model converges.

[0017] Further, the construction of the student model is specifically as follows:

[0018] The initial stage is normalized by Batch Normalization after convolution operation, followed by ReLU activation function, and then by a max-pooling layer to reduce the size of the feature map;

[0019] Subsequently, the model is divided into three stages: the first stage includes one down-sampling module and four basic modules connected in turn; the second stage includes one down-sampling module and eight basic modules connected in turn to further extract and process the features; the third stage includes one down-sampling module and four basic modules to deepen the expression ability of the features; in each stage, the basic module is composed of convolution, normalization, activation function and CBAM attention module; the connection between the basic modules is realized by channel rearrangement mechanism;

[0020] Then, the model is again subjected to convolution and Batch Normalization, followed by global average pooling to convert the feature map to a fixed size.

[0021] Finally, the feature is mapped to the final output category through a fully connected layer to obtain the prediction result.

[0022] Further, S4 is specifically: inputting the labeling result and the image data of the training set and the validation set into the teacher model and the student model at the same time to obtain vehicle positioning frame information respectively;

[0023] The difference between the vehicle positioning frame information output by the teacher model and the vehicle positioning frame information output by the student model is taken as the first part LOSS1 of the loss function; the difference between the result output by the student model and the real label is taken as the second part LOSS2 of the loss function; the difference between the confidence of the positioning frame output by the student model and the confidence of the positioning frame output by the teacher model is taken as the third part LOSS3 of the loss function; the LOSS1, LOSS2 and LOSS3 all use smooth L1 loss function; the weighted sum of LOSS1, LOSS2 and LOSS3 is taken as the final loss function of the student model to train the student model until convergence, and the final lightweight network model is obtained. Among them, LOSS1 and LOSS2 enable the student model to learn more accurate boundary definition ability, and LOSS3 enables the student model to also learn the boundary frame confidence judgment ability of the teacher model.

[0024] The beneficial effects of the present application are: the method of the present application aims at the problem of vehicle bounding box blur in night scene, adopts a teacher-student model to process the vehicle data collected at night; first, a detection model is used for labeling, after the labeling result is obtained, manual auditing and correction are performed, and the corrected labeled data is used as training data to train the above-mentioned detection model, and the trained detection model is used for labeling again, the above-mentioned process is repeated, and iterative labeling and training are realized, and finally the performance and labeling quality of the model will approach a stable state, that is, the model converges; this strategy can finally achieve the purposes of data labeling and model training, and the trained model can be used as a teacher model; the structure of the student model is compact and efficient, through channel rearrangement and grouped convolution technology, the calculation and memory overhead are greatly reduced, and it is suitable for devices with limited resources; at the same time, the CBAM attention module is adopted, which further enhances the feature representation ability of the model and improves the performance and accuracy. Based on the low-power platform, the algorithm deployment requirement is high, the positioning distillation technology is introduced in the present application, which effectively solves the problem of fuzzy boundary positioning, prevents the vehicle from colliding or even colliding due to the fusion of the vehicle and the background and the inaccurate boundary judgment; improves the positioning accuracy of the detection model, and can accurately judge the vehicle violation in the intelligent transportation scene; arranged in the vehicle-mounted 360-degree panoramic image system, the surrounding vehicles can be detected in real time to realize more accurate positioning, so as to avoid traffic accidents or provide better implementation conditions for automatic driving. BRIEF DESCRIPTION OF DRAWINGS

[0025] Figure 1 is a flowchart of the method of the present application;

[0026] Figure 2 is a student model training structure framework schematic diagram;

[0027] Figure 3 is a student model structure schematic diagram. DETAILED DESCRIPTION

[0028] In order to make the content of the present application more easily understood, the present application will be further described in detail below according to specific embodiments and in conjunction with the drawings.

[0029] As shown in Figure 1 and 2 , the vehicle boundary positioning method for night scene of the present application comprises the following steps:

[0030] S1: A dataset was collected at seven intersections on urban roads. Each intersection was equipped with two cameras. When the camera detected an approaching vehicle, it estimated the distance to the vehicle and photographed the vehicle when the vehicle was 3 meters, 6 meters, and 10 meters away from the camera. This recorded image information of vehicles passing through the area between 8 pm and 2 am every day. After a week of collection and screening, a total of 100,000 images of vehicles in dark environments were collected.

[0031] S2. Divide the collected image data into a training set, a validation set, and a test set, and preprocess the image data;

[0032] S3. Use the detection model to annotate the preprocessed image data to obtain a labeled image dataset; and use the labeled image dataset and the training set to train the detection model to obtain the final teacher model;

[0033] S4. Construct student model, e.g. Figure 3 As shown in the figure, this model is an efficient neural network model that accepts RGB images as input. In the initial stage, convolution is followed by batch normalization, followed by ReLU activation and max pooling, effectively reducing the size of the feature map. The model then proceeds through three stages. The first stage consists of a downsampling module (DownsampleUnit) and four basic modules (BasicUnit), which are sequentially interconnected. The second stage consists of a downsampling module and eight basic modules, which are sequentially connected to further extract and process features. The third stage, consisting of a downsampling module and four basic modules, is used to deepen the expressive power of features. Within each stage, the basic module (BasicUnit) is the core of the model and consists of convolution, normalization, activation functions, and a CBAM attention module. The connections between basic modules are achieved through a channel reordering mechanism, which effectively promotes feature exchange and information flow, improving feature representation capabilities.

[0034] After these three stages, the model undergoes convolution and batch normalization again, followed by global average pooling to convert the feature map to a fixed size in preparation for the fully connected layer. Finally, the fully connected layer maps the features to the final output category to obtain the prediction result.

[0035] like Figure 2 As shown in the figure, the boundary positioning knowledge learned by the teacher model is distilled using the positioning distillation method. The annotation results and the image data of the training set and the validation set are input into the teacher model and the student model at the same time to obtain the vehicle positioning frame information respectively, and finally a lightweight network model is obtained.

[0036] The difference between the vehicle positioning box information output by the teacher model and the vehicle positioning box information output by the student model is taken as the first part LOSS1 of the loss function; the difference between the result output by the student model and the real label is taken as the second part LOSS2 of the loss function; the difference between the confidence of the positioning box output by the student model and the confidence of the positioning box output by the teacher model is taken as the third part LOSS3 of the loss function; the LOSS1, LOSS2 and LOSS3 all use a smooth L1 loss function; the weighted sum of LOSS1, LOSS2 and LOSS3 is taken as the final loss function of the student model to train the student model until convergence, and finally obtain the lightweight network model. Wherein, LOSS1 and LOSS2 enable the student model to learn more accurate boundary defining ability, and LOSS3 enables the student model to also learn the boundary box confidence judgment ability of the teacher model.

[0037] S5, input the test set data into the lightweight network model, output the position coordinate information of the vehicle, set a threshold to filter the positioning box, use non-maximum suppression to delete the repeated positioning box, and finally obtain the vehicle position in the night scene.

[0038] Specifically, S2: for the collected image information, first divide the image data into a training set, a validation set and a test set according to the ratio of 8:1:1, then use a traditional image processing method, use Gaussian filtering to pre-process the collected image information, so that the human eye is more easy to distinguish the vehicle boundary, and facilitate subsequent labeling work, then use the vehicle recognition model weight pre-trained by the Yolov5 model to perform the first round of detection on all data, obtain the detection result label file corresponding to the image, then use the picture labeling tool, import the data set processed by Gaussian filtering in step S2 and the obtained label file into the tool, delete the positioning box with detection error, increase the positioning box not detected, and finely adjust the boundary of the vehicle positioning box, so as to accurately define the boundary of the vehicle. Manually complete the second round of labeling of the image.

[0039] Step S3: select the YoloV5 model as the teacher model, input the label generated by labeling and the training set and the validation set in the original picture without processing into the model for training, take the smooth L1 loss function of the model detection result of the vehicle position and the actual labeling result as the model loss function, use the pre-trained model weight file Yolov5m.pt as the initial weight to reduce the training time and data amount, train 300 rounds on the NVIDIA 3090 type GPU server, select the model weight file with the highest accuracy, and take it as the final teacher model.

[0040] Step S4: select the SAD (Shuffer Attention Det) network model as the student model, input the labels generated by the annotation and the training set and the validation set in the original picture without processing into the teacher model and the student model, and take the weighted sum of the smooth L1 loss function of the vehicle position information output by the teacher model and the vehicle position information output by the student model and the smooth L1 loss function of the confidence output by the two models respectively as the loss function of training, only update the student network during back propagation, train for 200 rounds, and select the model weight with the highest accuracy rate as the final student model.

[0041] Step S5: input the data of the test set into the obtained student model, output the position coordinate information of the vehicle, set the threshold value to 0.5, filter the positioning frame with a confidence lower than 0.5, and delete the redundant and repeated positioning frame output by the network by non-maximum suppression, and finally obtain the vehicle position in the night scene.

[0042] The above only describes the preferred scheme of the present application, and is not intended to further limit the present application. Any equivalent changes made according to the content of the specification and drawings of the present application are within the protection scope of the present application.

Claims

1. A vehicle boundary positioning method for night scenes, characterized in that: The following steps are involved: S1, collecting image data of nighttime road scenes; S2. Divide the collected image data into a training set, a validation set, and a test set, and preprocess the image data; S3. Use the detection model to annotate the preprocessed image data to obtain a labeled image dataset; and use the labeled image dataset and the training set to train the detection model to obtain the final teacher model; S4. Build a student model and use the positioning distillation method to distill the boundary positioning knowledge learned by the teacher model, and finally obtain a lightweight network model; S5. Input the test set data into the lightweight network model, output the vehicle's location coordinate information, set a threshold to filter the positioning frame, use non-maximum suppression to delete duplicate positioning frames, and finally obtain the vehicle position in the night scene; Among them, the construction of the student model is specifically as follows: In the initial stage, the convolution operation is followed by Batch Normalization, followed by ReLU activation function and then the maximum pooling layer to reduce the size of the feature map. The model is then divided into three stages: the first stage includes a downsampling module and four basic modules connected in sequence; the second stage includes a downsampling module and eight basic modules connected in sequence to further extract and process features; the third stage includes a downsampling module and four basic modules to deepen the expressive power of features; in each stage, the basic module consists of convolution, normalization, activation function and CBAM attention module; the connection between basic modules is achieved through the channel reordering mechanism; Then, the model is again normalized by convolution and Batch Normalization, followed by global average pooling to convert the feature map to a fixed size; Finally, the features are mapped to the final output category through the fully connected layer to obtain the prediction result; S4 is specifically: The annotation results and the image data of the training set and the validation set are input into the teacher model and the student model at the same time to obtain the vehicle positioning frame information respectively; The difference between the vehicle positioning frame information output by the teacher model and the vehicle positioning frame information output by the student model is used as the first part LOSS1 of the loss function; the difference between the result output by the student model and the true label is used as the second part LOSS2 of the loss function; the difference between the confidence of the positioning frame output by the student model and the confidence of the positioning frame output by the teacher model is used as the third part LOSS3 of the loss function; LOSS1, LOSS2, and LOSS3 all use the smooth L1 loss function; the weighted sum of LOSS1, LOSS2, and LOSS3 is used as the final loss function of the student model to train the student model until convergence to obtain the final lightweight network model.

2. The vehicle boundary positioning method for night scenes according to claim 1 is characterized in that: S2 is specifically: Image data is collected using vehicle-mounted cameras or road monitoring cameras, originally in RAW format. A deep learning network is used to take this raw format as input and output an RGB format image with enhanced illumination.

3. The vehicle boundary positioning method for night scenes according to claim 1 is characterized in that: S3 specifically: Use the detection model to annotate the image data, obtain the annotation results, i.e., the coordinate information of the vehicle positioning frame, and correct the annotation results to delete the incorrect positioning frame; The corrected labeled data is used as training data to continue training the detection model until the detection model converges to obtain the final teacher model.

Images