A method for measuring distance of preceding vehicles based on image features

Abstract

A method for measuring the distance to a vehicle ahead based on image features relates to the field of intelligent traffic management and control technology. A target detection model is used to detect target vehicles in an image, obtaining category information and vehicle detection frame feature information. Camera calibration is used to obtain internal parameters, and the horizontal and vertical distances of the camera relative to the vehicle are calculated based on camera imaging principles and geometric relationships. A convolutional neural network is designed to extract vehicle feature vectors. The vehicle detection frame feature information, horizontal and vertical distances, and vehicle feature vectors are integrated to generate a vehicle image feature vector. A vehicle ranging neural network is established to measure the distance to the vehicle based on the vehicle image feature vector. A feedforward neural network is established to learn the complex mapping from input vehicle feature data to output distance to the vehicle ahead. Real-time distance measurement between the vehicle and the target vehicle ahead is achieved based on image features. The vehicle detection frame feature information and vehicle image features are combined to improve ranging accuracy. The method is applicable to various complex weather environments.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent traffic management and control, and in particular to a method for measuring the distance of a preceding vehicle based on image features. Background Art

[0002] With the development of deep learning-based computer vision algorithms in traffic management and control, advanced driver assistance systems (ADAS) are now able to perceive the vehicle's surroundings using cameras, sensors, radar, lasers, and other devices. Vehicle ranging is a key component of these systems, helping to improve collision avoidance, hazard warning, and automatic cruise control, thereby effectively reducing the incidence of traffic accidents.

[0003] There are two main vehicle ranging methods: non-visual ranging and visual ranging. Non-visual ranging generally uses a vehicle-mounted LiDAR or ultrasonic sensor to measure the distance to surrounding obstacles. LiDAR provides high-resolution 3D point cloud data with high precision and accuracy, but it is expensive to manufacture and consumes a lot of energy, and its field of view is affected by the geometry and reflectivity of objects. Ultrasonic sensors, while lower in cost and size than LiDAR, are more suitable for close-range measurements and obstacle detection, but have lower accuracy for measuring distant or high-speed targets.

[0004] In contrast, visual ranging, as a foundational technology in computer vision, is more widely used, primarily encompassing monocular vision, binocular vision, and structured light vision. Considering both cost-effectiveness and practicality, monocular cameras offer advantages such as low cost, ease of maintenance, a wide field of view, and intuitiveness. However, the optical characteristics of cameras make them difficult to achieve all-weather detection in complex weather conditions. Therefore, utilizing monocular vision to achieve real-time ranging of vehicles ahead based on image features, while improving measurement accuracy and environmental adaptability, is of vital importance for intelligent traffic management and control. Summary of the Invention

[0005] To address the shortcomings of the background technology, the present invention provides a method for measuring the distance to the vehicle ahead based on image features. It establishes a feedforward neural network to learn the complex mapping from input vehicle feature data to output the distance to the vehicle ahead. It achieves real-time distance measurement between the vehicle and the target vehicle ahead based on image features, combines vehicle detection frame feature information and vehicle image features to improve ranging accuracy, and is applicable to various complex weather environments.

[0006] To achieve the above object, the present invention adopts the following technical solution: a method for measuring the distance of a vehicle ahead based on image features, comprising the following steps:

[0007] S1. Vehicle identification and detection

[0008] Detect the front target vehicle in the image through the target detection model, and obtain the category information and vehicle detection frame feature information of the front target vehicle;

[0009] The vehicle recognition detection is implemented by combining the S11 road vehicle image acquisition module, the S12 data set production module, the S13 data set training module and the S14 front vehicle image recognition and detection module; wherein the S11 road vehicle image acquisition module uses an on-board image set and self-built captured images, covering a variety of road traffic scenes under various weather conditions; the S12 data set production module classifies the image by annotating the category information of the front target vehicle and produces a data set for network reading; the S13 data set training module uses the target detection model to train the data set and saves the detection optimal weight file; the S14 front vehicle image recognition and detection module uses the detection optimal weight file to recognize and detect the input front target vehicle image, and outputs the category information of the front target vehicle and the vehicle detection frame feature information. The vehicle detection frame feature information calculation formula is as follows:

[0010]

[0011] h=v max -v min

[0012] W=u max -u min

[0013] s=h×w

[0014] Where, (u min， v min ) represents the coordinates of the upper left corner of the vehicle detection frame, (u max , v max ) represents the coordinates of the vertex of the lower right corner of the vehicle detection frame, and then the coordinates of the center point of the bottom edge of the vehicle detection frame p(u, v), the height h, width w and area s of the vehicle detection frame are obtained;

[0015] S2. Camera imaging geometry calculation

[0016] Camera calibration obtains the camera's internal parameters. Based on the camera's imaging principle and geometric relationship, the vehicle's lateral and longitudinal distance calculation formula is derived to calculate the lateral and longitudinal distances of the camera relative to the target vehicle in front.

[0017] The camera imaging geometry calculation is implemented by combining the S21 camera calibration module, the S22 geometry derivation module, and the S23 geometry calculation module. The S21 camera calibration module obtains the camera internal parameters. The S22 geometry derivation module derives the vehicle's lateral and longitudinal distance geometry calculation formula based on the camera imaging principle as follows:

[0018]

[0019] Where LD represents the longitudinal distance of the vehicle relative to the camera, TD represents the lateral distance of the vehicle relative to the camera, H represents the preset height of the camera, α represents the pitch angle of the camera, f represents the focal length of the camera, and f y represents the pixel describing the focal length in the y-axis direction, m represents the vertical distance from point a to the origin O of the image coordinate system, m = (v-v0)dy, t represents the horizontal distance from point a to the center point p of the bottom edge of the vehicle detection frame, t = (u-u0)dx, point a is the intersection of the vertical line of the center point p of the bottom edge of the vehicle detection frame and the y-axis in the image coordinate system; the S23 geometric calculation module substitutes the parameter values ​​required by the vehicle horizontal and vertical distance geometric calculation formula into the calculation to obtain the horizontal and vertical distances of the camera relative to the target vehicle in front;

[0020] S3. Vehicle feature extraction

[0021] Design a convolutional neural network to extract vehicle feature vectors;

[0022] The vehicle feature extraction is implemented by combining the S31 vehicle image preprocessing module and the S32 vehicle feature extraction module; wherein the S31 vehicle image preprocessing module crops the front target vehicle image according to the coordinates of the two corner vertices of the vehicle detection frame, and adjusts the image to a uniform size through preprocessing so that the shape of the vehicle in the image and the image pixels have characteristics; the S32 vehicle feature extraction module adopts a convolutional neural network, after inputting the preprocessed image, uses convolution and pooling operations to gradually reduce the image size, applies the ReLU activation function after the convolution operation, so that the convolution layer can retain the different levels of image features, uses the maximum pooling layer after the activation function, and reduces the image size by half to reduce the computational complexity, and finally generates the vehicle feature vector through flattening processing;

[0023] S4. Vehicle image feature fusion

[0024] The vehicle detection frame feature information, the horizontal and vertical distances of the camera relative to the target vehicle in front, and the vehicle feature vector are integrated to generate the vehicle image feature vector;

[0025] The vehicle image feature fusion is implemented in conjunction with the S41 image feature fusion module; the S41 image feature fusion module fuses the obtained vehicle detection frame feature information, the horizontal and vertical distances of the camera relative to the target vehicle in front, and the vehicle feature vector into a vehicle image feature vector;

[0026] S5. Neural Network Odometry

[0027] Establish a vehicle ranging neural network to measure the distance between the camera and the target vehicle in front based on the fused vehicle image feature vector;

[0028] The neural network ranging is implemented by combining the S51 front vehicle image and distance acquisition module, the S52 ranging data set preprocessing module and the S53 neural network ranging module; wherein, the S51 front vehicle image and distance acquisition module obtains multiple groups of front target vehicle images with a vehicle-mounted camera of preset height and angle, and simultaneously measures the actual distance to the front target vehicle, and forms a ranging data set by corresponding the front target vehicle images and the actual distances one by one. The front target vehicle images in the ranging data set are passed through the S14 front vehicle image recognition and detection module, the S23 geometric calculation module, the S31 vehicle image preprocessing module, the S32 vehicle feature extraction module and the S41 image feature fusion module to obtain a vehicle image feature vector; the S52 ranging data set preprocessing module divides the ranging data set into two parts: a ranging data training set and a ranging data test set, saves the vehicle image feature vector of the ranging data training set in the feature_train.txt file format, and saves the corresponding actual distance in the distance_train.txt file format; saves the vehicle image feature vector of the ranging data test set in the feature_test.txt file format, and saves the corresponding actual distance in the feature_test.txt file format. The data is saved in the distance_test.txt file format; the S53 neural network ranging module establishes a feedforward neural network, including six fully connected layers, and extracts features by processing data through linear transformation of six-dimensional fully connected layers. The ReLU activation function is applied between the fully connected layers. The front-end fully connected layer serves as the input layer to input the fused vehicle image feature vector, and the terminal fully connected layer contains only one neuron as the output layer to output the predicted distance; the S53 neural network ranging module is divided into two stages: training and inference; in the training stage, the ranging data training set obtained by the S52 ranging data set preprocessing module is used for training the feedforward neural network, the feature_train.txt file is the vehicle image feature vector data as the feedforward neural network input, and the distance_train.txt file is the actual distance as the true value of the feedforward neural network; in the inference stage, the feature_test.txt file of the ranging data test set is input into the feedforward neural network, and the input data is passed layer by layer through forward propagation based on the ranging optimal weight file generated by the feedforward neural network in the training stage, and finally the predicted distance from the target vehicle in front to the camera is obtained in the output layer.

[0029] Furthermore, in step S21, the camera calibration module selects a chessboard calibration plate with a specification of 12×9 and a cell side length of 30 mm, and obtains the camera internal parameters through Zhang's calibration method and OpenCV calibration toolbox.

[0030] Compared with the prior art, the present invention has the following beneficial effects: the present invention uses an on-board camera to collect images in real time, obtains category information of the target vehicle in front and vehicle detection frame feature information through a target detection model, derives the horizontal and vertical distances of the camera relative to the target vehicle in front based on the camera imaging principle and geometric relationship, extracts vehicle feature vectors through a convolutional neural network, and can adjust the network structure according to the needs of specific application scenarios to establish a feedforward neural network, learn the complex mapping from input vehicle feature data to output front vehicle distance, and realize real-time distance measurement between the vehicle and the target vehicle in front based on image features. The vehicle detection frame feature information and vehicle image features make up for the low ranging accuracy problem caused by traditional monocular ranging relying solely on geometric algorithms. The present invention is also applicable to various complex weather environments, realizes high-precision, high-stability, and real-time front target ranging, and helps to improve the level of vehicle perception technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a block diagram of the steps of the method of the present invention;

[0032] Figure 2 It is a module flow chart of the method of the present invention;

[0033] Figure 3 Schematic diagram of the geometric relationship between the camera and the target vehicle in front in the method of the present invention;

[0034] Figure 4 Schematic diagram of the convolutional neural network of the vehicle feature extraction module in the method of the present invention;

[0035] Figure 5 Schematic diagram of the feedforward neural network of the neural network ranging module in the method of the present invention. DETAILED DESCRIPTION

[0036] The technical solutions of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0037] like Figures 1 to 5 As shown, a method for measuring the distance of the vehicle ahead based on image features, the process is combined with Figure 1 As shown, the following steps are included:

[0038] S1. Vehicle identification and detection

[0039] The target detection model is used to detect the front target vehicle in the image and obtain the category information and vehicle detection frame feature information of the front target vehicle.

[0040] Combine Figure 2 As shown, the vehicle recognition detection is implemented by combining the S11 road vehicle image acquisition module, the S12 data set production module, the S13 data set training module and the S14 front vehicle image recognition and detection module. Among them, the S11 road vehicle image acquisition module uses a vehicle-mounted image set and self-built shooting images, which should cover a variety of road traffic scenes such as urban roads and road intersections under various weather conditions such as sunny days, cloudy days, rainy days, and snowy days. Images can be randomly extracted and data enhanced by cropping, color jittering and Gaussian noise to ensure the subsequent training effect; the S12 data set production module uses the labelImg annotation tool to manually set the category information of the front target vehicle for classification. After the manual annotation is completed, an xml file is generated and made into a data set for network reading; the S13 data set training module uses the target detection model to train the data set and save the detection optimal weight file; the S14 front vehicle image recognition and detection module uses the detection optimal weight file to detect the input front target vehicle image, and outputs the category information of the front target vehicle and the vehicle detection box feature information (u, v, h, w, s). The vehicle detection box feature information calculation formula is as follows:

[0041]

[0042] h=v max -v min

[0043] W=u max -u min

[0044] s=h×w

[0045] Where, (u min , v min ) represents the coordinates of the upper left corner of the vehicle detection frame, (u max , v max ) represents the coordinates of the vertex of the lower right corner of the vehicle detection frame, and then the coordinates of the center point of the bottom edge of the vehicle detection frame p(u, v), the height h, width w and area s of the vehicle detection frame are obtained.

[0046] S2. Camera imaging geometry calculation

[0047] Camera calibration obtains the camera's internal parameters. Based on the camera imaging principle and geometric relationship, the vehicle's lateral and longitudinal distance calculation formula is derived to calculate the lateral and longitudinal distances of the camera relative to the target vehicle in front.

[0048] Combine Figure 2As shown in the figure, the camera imaging geometry calculation is implemented by combining the S21 camera calibration module, the S22 geometry derivation module and the S23 geometry calculation module. Among them, the S21 camera calibration module uses a chessboard calibration plate with a specification of 12×9 and a cell side length of 30mm, and obtains the camera internal parameters through Zhang's calibration method and OpenCV calibration toolbox; the S22 geometry derivation module is based on the camera imaging principle and combined with Figure 3 As shown, using the world coordinate system O W -X W Y W Z W , camera coordinate system O C -X C Y C Z C , the geometric relationship between the image coordinate system O-xy and the pixel coordinate system O0-uv is established, and the geometric calculation formula for the vehicle's horizontal and vertical distance is derived as follows:

[0049]

[0050] Where LD represents the longitudinal distance of the vehicle relative to the camera, TD represents the lateral distance of the vehicle relative to the camera, H represents the preset height of the camera, α represents the pitch angle of the camera, f represents the focal length of the camera, and f y Represents the pixel describing the focal length in the y-axis direction, m represents the vertical distance from point a to the origin O of the image coordinate system, m = (v-v0)dy, t represents the horizontal distance from point a to the center point p of the bottom edge of the vehicle detection frame, t = (u-u0)dx, point a is the intersection of the perpendicular line between the center point p of the bottom edge of the vehicle detection frame and the y-axis in the image coordinate system; the S23 geometric calculation module substitutes the parameter values ​​required by the vehicle's lateral and longitudinal distance geometric calculation formula to calculate the lateral and longitudinal distances (TD, LD) of the camera relative to the target vehicle in front.

[0051] S3. Vehicle feature extraction

[0052] Design a convolutional neural network to extract vehicle feature vectors.

[0053] Combine Figure 2 As shown, the vehicle feature extraction is implemented by combining the S31 vehicle image preprocessing module and the S32 vehicle feature extraction module. Among them, the S31 vehicle image preprocessing module crops the front target vehicle image according to the coordinates of the two corner vertices of the vehicle detection frame. The length, width, pixel size and presentation characteristics of the front target vehicle image will change with the distance. Therefore, the image must be adjusted to a uniform size through preprocessing so that the shape of the vehicle in the image and the image pixels have characteristics; the S32 vehicle feature extraction module adopts a convolutional neural network. The convolutional neural network can adjust the number of convolution kernels according to the different types of the front target vehicles to extract the optimal vehicle feature vector. Figure 4As shown in the figure, after inputting the preprocessed image, the image size is gradually reduced by using convolution and pooling operations. The ReLU activation function is applied after the convolution operation so that the convolution layer can retain the different levels of image features. The 2×2 maximum pooling layer is used after the activation function to halve the image size to reduce the computational complexity. Finally, the vehicle feature vector is generated by flattening.

[0054] S4. Vehicle image feature fusion

[0055] The vehicle detection frame feature information, the horizontal and vertical distances of the camera relative to the target vehicle in front, and the vehicle feature vector are fused to generate a vehicle image feature vector.

[0056] Combine Figure 2 As shown in FIG, the vehicle image feature fusion is implemented in conjunction with the S41 image feature fusion module. The S41 image feature fusion module fuses the obtained vehicle detection frame feature information (u, v, h, w, s), the horizontal and vertical distances (TD, LD) of the camera relative to the target vehicle in front, and the vehicle feature vector into a vehicle image feature vector.

[0057] S5. Neural Network Odometry

[0058] A vehicle ranging neural network is established to measure the distance between the camera and the target vehicle in front based on the fused vehicle image feature vector.

[0059] Combine Figure 2 As shown, the neural network ranging is implemented by combining S51 front vehicle image and distance acquisition module, S52 ranging data set preprocessing module and S53 neural network ranging module. Among them, the S51 front vehicle image and distance acquisition module uses a vehicle-mounted camera with a preset height and angle to obtain multiple groups of front target vehicle images, and at the same time measures the actual distance to the front target vehicle, and the front target vehicle image and the actual distance are matched one by one to form a ranging data set. The front target vehicle image in the ranging data set is passed through the S14 front vehicle image recognition and detection module, the S23 geometric calculation module, the S31 vehicle image preprocessing module, the S32 vehicle feature extraction module and the S41 image feature fusion module to obtain a vehicle image feature vector; the S52 ranging data set preprocessing module divides the ranging data set into two parts: a ranging data training set and a ranging data test set, and saves the vehicle image feature vector of the ranging data training set in the feature_train.txt file format, and the corresponding actual distance in the distance_train.txt file format; the vehicle image feature vector of the ranging data test set is saved in the feature_test.txt file format, and the corresponding actual distance in the distance_test.txt file format; the S53 neural network ranging module establishes a feedforward neural network, combined with Figure 5As shown, it includes six fully connected layers, which process data through linear transformation of six-dimensional fully connected layers to extract high-dimensional and low-dimensional features. The ReLU activation function is applied between fully connected layers. The front-end fully connected layer inputs the fused vehicle image feature vector as the input layer to provide raw data for subsequent feature extraction. The terminal fully connected layer contains only one neuron as the output layer to output the predicted distance. In addition, the S53 neural network ranging module is divided into two stages: training and inference. During the training stage, the ranging data training set obtained by the S52 ranging data set preprocessing module is used for the training of the feedforward neural network. The feature_train.txt file is the vehicle image feature vector data as the feedforward neural network input, and the distance_train.txt file is the actual distance as the true value of the feedforward neural network. The training phase of the feedforward neural network involves forward propagation and backpropagation. Forward propagation calculates the output of neurons in each layer, from the input layer to the output layer. Backpropagation calculates the loss function to measure the error between the predicted distance and the actual distance. The error is propagated from the output layer to the input layer, and the gradient of the error is calculated layer by layer. The weights and biases of each connection are updated based on the error gradient to reduce the overall error. This learning achieves the complex mapping relationship between the input vehicle image feature vector data and the actual distance. The internal weight parameters are continuously adjusted, and finally an optimal distance measurement weight file is generated and saved, recording the optimal weights and biases learned by the feedforward neural network during training. During the inference phase, the feature_test.txt file of the distance measurement data test set is input into the feedforward neural network. Based on the optimal distance measurement weight file generated during the training phase, the input data is passed layer by layer through forward propagation. Using the trained optimal weights and biases, the weighted sum of neurons in each layer and the ReLU activation function are processed to obtain the predicted distance of the target vehicle in front from the camera at the output layer, realizing the mapping from input features to output results.

[0060] Example

[0061] S1. Detect the front target vehicle in the image through the target detection model to obtain the category information and vehicle detection frame feature information of the front target vehicle.

[0062] The S11 road vehicle image acquisition module uses the BDD 100K on-board image set and self-captured images, totaling 3,800 images. These images cover various road traffic scenarios, including urban roads and road intersections, in various weather conditions such as sunny, cloudy, rainy, and snowy days. Randomly selected images are then cropped, color jittered, and Gaussian noise added for data enhancement, expanding the number of images to 5,200 to ensure the effectiveness of subsequent training.

[0063] The S12 dataset production module manually annotates images using the labelImg annotation tool, sets the category information of four types of vehicles: car, truck, van, and bus. After the manual annotation is completed, an XML file is generated and made into a dataset for network reading.

[0064] The S13 dataset training module uses the target detection model to train the dataset. The conventional target detection model mainly includes a feature extraction network and a detection network. The feature extraction network usually uses a pre-trained convolutional neural network to extract the features of the input image. The detection network is the convolution layer and the fully connected layer after the feature extraction network, which predicts the position and category of the bounding box. This embodiment optimizes and improves the target detection model. On the basis of the conventional target detection model, an attention mechanism is added to the feature extraction network structure, a small target detection layer is added, and the loss function is replaced. The EMA attention mechanism (Efficient Multi-Scale Attention) is used to enhance the target detection model's attention to the overall language. The algorithm reduces the target detection model's attention to local noise in the input sequence, improves robustness and generalization ability, reduces computational complexity, and adapts to large-scale data and high-performance computing environments in real-time processing. The small target detection layer enhances the small target detection performance and accuracy of the target detection model. The Wise-IoU loss function is used to achieve multi-level feature fusion and multi-scale detection, adapt to multi-scale changes, and solve the problem of detecting occluded targets. The optimized target detection model is more adaptable to various complex weather environments, provides accurate detection box feature information, and provides precise data for vehicle ranging. The dataset is trained in the optimized target detection model and the optimal weight file best.pt is saved.

[0065] The S14 front vehicle image recognition and detection module uses the optimal weight file best.pt to identify and detect the input front target vehicle image test.jpg, outputting the front target vehicle category information (car), type confidence (96%), and the coordinates of the upper left corner vertex of the vehicle detection box (1069.0, 495.0) and the lower right corner vertex coordinates (1173.0, 566.0). The vehicle detection box feature information is calculated using the following formula:

[0066]

[0067] h=566.0-495.0=71.0

[0068] w=1173.0-1069.0=104.0

[0069] s=71.0×104.0=7384.0

[0070] The coordinates of the center point of the bottom edge of the vehicle detection frame p (1121.0, 566.0), the height h = 71.0, the width w = 104.0 and the area s = 7384.0 of the vehicle detection frame are obtained, that is, the feature information (u, v, h, w, s) of the vehicle detection frame is (1121.0, 566.0, 71.0, 104.0, 7384.0).

[0071] S2. Camera calibration obtains the camera's internal parameters. Based on the camera imaging principle and geometric relationship, the horizontal and vertical distance calculation formulas are derived to calculate the horizontal and vertical distances of the camera relative to the target vehicle in front.

[0072] The S21 camera calibration module uses a Sony HDR-CX680 camera with a lens focal length of 1.9mm. A 12×9 chessboard calibration board with a cell side length of 30mm is selected. The camera position is fixed, and the chessboard orientation and angle are changed to capture images, capturing 20 images. The camera's internal parameters are obtained using Zhang's calibration method and the OpenCV calibration toolbox. The OpenCV calibration toolbox obtains the coordinates of the inner corner points of the image and passes these coordinates to the CalibrateCamera function, establishing a correspondence between 3D coordinate points and 2D coordinate points. The 2D image coordinates are mapped to 3D world coordinates to obtain the camera's internal parameters, f. x is 1422.86 pixels, f y is 1422.73 pixels, u0 is 953.46 pixels, and v0 is 542.27 pixels.

[0073] The S22 geometry derivation module is based on the camera imaging principle and combines Figure 3 As shown, using the world coordinate system O W -X W Y W Z W , camera coordinate system O C -X C Y C Z C , the conversion relationship between the image coordinate system O-xy and the pixel coordinate system O0-uv establishes a geometric relationship. In this embodiment, the preset height of the camera is 1.4m, the focal length is 1.9mm, the pitch angle of the camera is 2°, the ranging feature point of the target vehicle in front is P, and the point mapped on the image is p(1121.0, 566.0).

[0074] The S23 geometry calculation module substitutes the required parameter values ​​into the vehicle's horizontal and vertical distance geometry calculation formula to perform the following calculation:

[0075]

[0076] The horizontal and vertical distances between the camera and the target vehicle in front are (3.08099, 27.9150), the actual measured distances are (3.0, 28.0), and the calculated errors are (0.0809, 0.0850).

[0077] S3. Design a convolutional neural network to extract vehicle feature vectors.

[0078] The S31 vehicle image preprocessing module crops the image of the target vehicle in front according to the coordinates of the two corner vertices of the vehicle detection frame. The length, width, pixel size and presentation characteristics of the image of the target vehicle in front will change with the distance. The image is adjusted to a uniform size of 128×128 through preprocessing to make the shape and image pixels of the vehicle in the image have characteristics.

[0079] The S32 vehicle feature extraction module uses convolutional neural networks to establish five 3×3 convolutional neural networks, combined with Figure 4 As shown in the figure, after inputting the preprocessed image, conv1 contains 64 convolution kernels and performs convolution operations to extract image edge and texture features. conv2 extracts high-level features, generating a 128-channel feature map to mine complex features. conv3 and conv4 perform dimensionality reduction operations, reducing the number of channels and minimizing network complexity. conv5 generates a single-channel feature map. Finally, flattening is performed to combine all subsets into a one-dimensional array to generate a 16-dimensional vehicle feature vector. A ReLU activation function is applied after the convolution operation, enabling the convolution layer to preserve features at different levels of the image. A 2×2 max pooling layer is used after the activation function to halve the image size to reduce computational complexity.

[0080] S4. Fusion of vehicle detection frame feature information, the horizontal and vertical distances of the camera relative to the target vehicle in front, and the vehicle feature vector to generate a vehicle image feature vector.

[0081] The S41 image feature fusion module fuses the obtained vehicle detection frame feature information (u, v, h, w, s), the horizontal and vertical distances of the camera relative to the target vehicle in front (TD, LD), and the 16-dimensional vehicle feature vector into a 23-dimensional vehicle image feature vector. Specifically: the obtained vehicle detection frame feature information (1121.0, 566.0, 71.0, 104.0, 7384.0), the horizontal and vertical distances of the camera relative to the target vehicle in front (3.0 8099, 27.9150), 16-dimensional vehicle feature vector (0.039285928, 0.039868373, 0.03859806, 0.043999564, 0.034194592, 0.03373158, 0.03706052, 0.0332656, 0.032308687, 0.03551898, 0.034619715, 0. 0.035736978, 0.032287486, 0.034737445, 0.03438149, 0.03596463, fused into a 23-dimensional vehicle image feature vector (0.039285928, 0.039868373, 0.03859806, 0.043999564, 0.034194592, 0.03373158, 0.03706052 , 0.0332656, 0.032308687, 0.03551898, 0.034619715, 0.035736978, 0.032287486, 0.034737445, 0.03438149, 0.03596463, 1121.0, 566.0, 71.0, 104.0, 7384.0, 3.08099, 27.9150).

[0082] S5. Establish a vehicle ranging neural network to measure the distance between the camera and the target vehicle in front based on the fused vehicle image feature vector.

[0083] The S51 front vehicle image and distance acquisition module uses a vehicle-mounted camera with a preset height of 1.4m and an angle of 2° to capture images of the target vehicle in front and measure the actual distance to it within a lateral distance of 12m and a longitudinal distance of 50m, obtaining a total of 61 sets of ranging data sets. The front target vehicle images in the ranging data sets are processed by the S14 front vehicle image recognition and detection module, the S23 geometric calculation module, the S31 vehicle image preprocessing module, the S32 vehicle feature extraction module and the S41 image feature fusion module to obtain the vehicle image feature vector.

[0084] The S52 ranging data set preprocessing module divides the 61 ranging data sets into 50 ranging data training sets and 11 ranging data test sets. The vehicle image feature vectors of the ranging data training set are saved in the feature_train.txt file format, and the corresponding actual distances are saved in the distance_train.txt file format. The vehicle image feature vectors of the ranging data test set are saved in the feature_test.txt file format, and the corresponding actual distances are saved in the distance_test.txt file format.

[0085] S53 neural network ranging module establishes a feedforward neural network, combined with Figure 5As shown, the system consists of six fully connected layers. The input layer inputs a 23-dimensional vehicle image feature vector, providing raw data for subsequent feature extraction. Fully connected layer F1 contains 32 neurons, F2 contains 48 neurons, F3 contains 16 neurons, and F4 contains 8 neurons. ReLU activation functions are applied between fully connected layers to capture complex modulo features in the data. The output layer contains only one neuron. Data is processed through linear transformations in the six fully connected layers to extract high- and low-dimensional features. During the training phase, 50 sets of distance measurement data training sets are input into the feedforward neural network for training. The batch_size is set to 1, the Adam optimizer is used, and the learning rate is set to 0.0001. During the training phase, the feedforward neural network calculates the output of neurons in each layer through forward propagation and back propagation, from the input layer to the output layer, and uses the loss function to measure the error between the predicted distance and the actual distance. The error is propagated forward from the output layer to the input layer, and the gradient of the error is calculated layer by layer. The weight and bias of each connection are updated according to the error gradient. A total of 3000 rounds of training are performed to achieve the learning input feature_train.txt file. The vehicle image feature vector data and the actual distance in the distance_train.txt file are connected. The system continuously adjusts the internal weight parameters to generate and save the optimal weight file for ranging. In the inference phase, the feature_test.txt files of 11 sets of ranging data test sets are input into the feedforward neural network. Based on the optimal weight file for ranging generated in the training phase, the input data is passed layer by layer through forward propagation. Using the trained optimal weights and biases, the weighted sum of neurons in each layer and the ReLU activation function are processed to obtain the predicted distance from the target vehicle in front to the camera at the output layer, realizing the mapping from input features to output results. Finally, the predicted distance is compared with the actual distance corresponding to the distance_test.txt file to verify the feasibility of neural network ranging.Taking the above example as a set of test data in the range measurement data test set, the 23-dimensional vehicle image feature vector (0.039285928, 0.039868373, 0.03859806, 0.043999564, 0.034194592, 0.03373158, 0.03706052, 0.0332656, 0.032308687, 0.03551898, 0.034619715, 0.03 The input of the ranging neural network is 5736978, 0.032287486, 0.034737445, 0.03438149, 0.03596463, 1121.0, 566.0, 71.0, 104.0, 7384.0, 3.08099, 27.9150). The output distance from the camera to the target vehicle in front is 28.29m. The actual distance measured is 28.37m, with an error of 0.08m, which is within a reasonable range.

[0086] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other configurations without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims, not the foregoing description, and all variations coming within the meaning and range of equivalents of the claims are intended to be embraced therein. Any reference sign in a claim should not be construed as limiting the claim to which it relates.

[0087] In addition, it should be understood that although this specification is described in terms of implementation methods, not every implementation method contains only one independent technical solution. This narrative method of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other implementation methods that can be understood by those skilled in the art.

Claims

1. A method for measuring the distance to a preceding vehicle based on image features, characterized in that: The following steps are involved: S1. Vehicle identification and detection Detect the front target vehicle in the image through the target detection model, and obtain the category information and vehicle detection frame feature information of the front target vehicle; The vehicle recognition detection is implemented by combining the S11 road vehicle image acquisition module, the S12 data set production module, the S13 data set training module and the S14 front vehicle image recognition and detection module; wherein the S11 road vehicle image acquisition module uses an on-board image set and self-built captured images, covering a variety of road traffic scenes under various weather conditions; the S12 data set production module classifies the image by annotating the category information of the front target vehicle and produces a data set for network reading; the S13 data set training module uses the target detection model to train the data set and saves the detection optimal weight file; the S14 front vehicle image recognition and detection module uses the detection optimal weight file to recognize and detect the input front target vehicle image, and outputs the category information of the front target vehicle and the vehicle detection frame feature information. The vehicle detection frame feature information calculation formula is as follows: Where, Indicates the coordinates of the upper left corner of the vehicle detection frame. Represents the coordinates of the vertex in the lower right corner of the vehicle detection frame, and then obtains the coordinates of the center point of the bottom edge of the vehicle detection frame , the height of the vehicle detection frame ,width and area ; S2. Camera imaging geometry calculation Camera calibration obtains the camera's internal parameters, and the vehicle's lateral and longitudinal distance calculation formula is derived based on the camera's imaging principle and geometric relationship to calculate the lateral and longitudinal distances of the camera relative to the target vehicle in front. The camera imaging geometry calculation is implemented by combining the S21 camera calibration module, the S22 geometry derivation module, and the S23 geometry calculation module. The S21 camera calibration module obtains the camera internal parameters. The S22 geometry derivation module derives the vehicle's lateral and longitudinal distance geometry calculation formula based on the camera imaging principle as follows: Where, represents the longitudinal distance of the vehicle relative to the camera, represents the lateral distance of the vehicle relative to the camera, H Indicates the preset height of the camera. represents the pitch angle of the camera, represents the focal length of the camera, Description The focal length in pixels along the axis, express a Point to image coordinate system origin O The vertical distance, , express a Click to the center point of the bottom edge of the vehicle detection frame The horizontal distance, , a Point is the center point of the bottom edge of the vehicle detection frame In the image coordinate system The S23 geometric calculation module substitutes the parameter values ​​required by the vehicle horizontal and vertical distance geometric calculation formula into the calculation to obtain the horizontal and vertical distance of the camera relative to the target vehicle in front; S3. Vehicle feature extraction Design a convolutional neural network to extract vehicle feature vectors; The vehicle feature extraction is implemented by combining the S31 vehicle image preprocessing module and the S32 vehicle feature extraction module; wherein, the S31 vehicle image preprocessing module crops the front target vehicle image according to the coordinates of the two corner vertices of the vehicle detection frame, and adjusts the image to a uniform size of 128×128 through preprocessing, so that the shape and image pixels of the vehicle in the image have characteristics; the S32 vehicle feature extraction module adopts a convolutional neural network, which includes 5 3×3 convolutional layers, wherein the conv1 layer contains 64 convolution kernels to perform convolution operations, and the conv2 layer generates a 128-channel feature map. The conv3 and conv4 layers use 32 and 8 convolution kernels, respectively, for dimensionality reduction. The conv5 layer uses 4 convolution kernels to generate a 4×4 single-channel feature map. Finally, all subsets are combined into a one-dimensional array through flattening to generate a 16-dimensional vehicle feature vector. After inputting the preprocessed image, the image size is gradually reduced through convolution and pooling operations. The ReLU activation function is applied after the convolution operation to enable the convolution layer to retain the different levels of image features. A 2×2 maximum pooling layer is used after the activation function to halve the image size to reduce computational complexity. Finally, the vehicle feature vector is generated through flattening. S4. Vehicle image feature fusion The vehicle detection frame feature information, the horizontal and vertical distances of the camera relative to the target vehicle in front, and the vehicle feature vector are integrated to generate a 23-dimensional vehicle image feature vector. The vehicle image feature fusion is implemented in conjunction with the S41 image feature fusion module; the S41 image feature fusion module fuses the obtained vehicle detection frame feature information, the horizontal and vertical distances of the camera relative to the target vehicle in front, and the vehicle feature vector into a vehicle image feature vector; S5. Neural Network Odometry Establish a vehicle ranging neural network to measure the distance between the camera and the target vehicle in front based on the fused vehicle image feature vector; The neural network ranging is implemented by combining the S51 front vehicle image and distance acquisition module, the S52 ranging data set preprocessing module and the S53 neural network ranging module; wherein, the S51 front vehicle image and distance acquisition module uses a vehicle-mounted camera at a preset height and angle to obtain multiple sets of front target vehicle images, and simultaneously measures the actual distance to the front target vehicle, and the front target vehicle images and the actual distances are mapped one-to-one to form a ranging data set. The front target vehicle images in the ranging data set are passed through the S14 front vehicle image recognition and detection module, the S23 geometric calculation module, the S31 vehicle image preprocessing module, the S32 vehicle feature extraction module and the S41 image feature fusion module to obtain a vehicle image feature vector; The S52 distance measurement data set preprocessing module divides the distance measurement data set into two parts: a distance measurement data training set and a distance measurement data test set. The vehicle image feature vector of the distance measurement data training set is saved in the feature_train.txt file format, and the corresponding actual distance is saved in the distance_train.txt file format. The vehicle image feature vector of the distance measurement data test set is saved in the feature_test.txt file format, and the corresponding actual distance is saved in the distance_test.txt file format. The S53 neural network distance measurement module establishes a feedforward neural network, including six fully connected layers, wherein the input layer receives a 23-dimensional vehicle image feature vector. , the fully connected layer F1 contains 32 neurons, F2 contains 48 neurons, F3 contains 16 neurons, F4 contains 8 neurons, and the output layer contains 1 neuron. The data is processed by linear transformation of the six-dimensional fully connected layer to extract features. The ReLU activation function is applied between the fully connected layers. The Adam optimizer is used for training with a learning rate of 0.0001. The front-end fully connected layer is used as the input layer to input the fused vehicle image feature vector, and the terminal fully connected layer contains only one neuron as the output layer to output the predicted distance; the S53 neural network ranging module is divided into two stages: training and inference. The training stage updates the weights through forward propagation and backpropagation. The inference stage is based on the optimal weight file through Forward propagation outputs the predicted distance; in the training phase, the ranging data training set obtained by the S52 ranging data set preprocessing module is used to train the feedforward neural network. The feature_train.txt file is the vehicle image feature vector data as the feedforward neural network input, and the distance_train.txt file is the actual distance as the true value of the feedforward neural network; in the inference phase, the feature_test.txt file of the ranging data test set is input into the feedforward neural network. Based on the ranging optimal weight file generated by the feedforward neural network in the training phase, the input data is passed layer by layer through forward propagation, and finally the predicted distance from the target vehicle in front to the camera is obtained at the output layer.

2. The method for measuring the distance to a preceding vehicle based on image features according to claim 1, wherein: In step S2, the camera calibration module S21 selects a chessboard calibration plate with a specification of 12×9 and a cell side length of 30 mm, and obtains the camera internal parameters through Zhang's calibration method and OpenCV calibration toolbox.

Images