A self-supervised monocular vehicle distance detection method

Abstract

This invention discloses a self-supervised monocular vehicle distance detection method, including step S1: downloading and processing the KITTI dataset, randomly dividing the dataset into training and testing sets; step S2: constructing an autoencoder structure for feature extraction, with the original image as input, calculating the photometric error in the extracted feature map, and dynamically adjusting parameters to minimize the error between the reconstructed image and the original image; step S3: constructing a multi-dimensional model, building a pose network and a depth prediction network, and adding an attention mechanism; step S4: converting the relative depth output by the model into absolute depth through scale recovery. This invention improves the monodepth2 network structure by adding a CRPBlock chained residual pooling module and introducing an attention module to the decoder network, making the model focus more on important feature regions, thereby improving model performance.

Description

Technical Field

[0001] This invention belongs to the field of vehicle distance estimation technology, and specifically relates to a self-supervised monocular vehicle distance detection method. Background Technology

[0002] In the fields of vehicle driving and traffic safety, accurately estimating the distance between a vehicle and a camera is crucial for driving assistance systems and intelligent transportation systems. With the rapid development of computer vision and deep learning, monocular image-based vehicle distance detection methods have become a solution with broad application potential. Traditional supervised learning methods require large amounts of labeled data and struggle to obtain accurate vehicle distance information. Furthermore, the data labeling and time costs during training are also limiting factors. However, existing self-supervised algorithms cannot handle scenes with transparency, reflection, and low-texture regions. These scenes lack explicit depth cues, making it difficult for depth estimation algorithms to accurately infer the depth of these regions. Therefore, we need a self-supervised learning-based monocular vehicle distance detection method to achieve accurate vehicle distance estimation, thereby improving the performance of driving assistance systems and traffic safety. Further research and technological innovation are needed to address these challenges. Summary of the Invention

[0003] To overcome the shortcomings of the prior art, the present invention provides a self-supervised monocular vehicle distance detection method.

[0004] To achieve the aforementioned objectives of the invention, the technical solution adopted to solve its technical problems is as follows:

[0005] A self-supervised monocular vehicle distance detection method includes the following steps:

[0006] Step S1: Download and process the KITTI dataset, randomly dividing the dataset into training and test sets;

[0007] Step S2: Construct an autoencoder structure for feature extraction. The input is the original image, and the photometric error is calculated in the extracted feature map. The parameters are dynamically adjusted to minimize the error between the reconstructed image and the original image.

[0008] Step S3: Construct a multi-dimensional model, build a pose network and a deep prediction network, and add an attention mechanism;

[0009] Step S4: Scale recovery is performed on the relative depth output by the model to convert it into absolute depth.

[0010] Furthermore, step S1 includes the following:

[0011] The downloaded KITTI dataset was augmented by transforming and expanding the data to generate diverse training samples. The KITTI dataset uses real-shot images to construct the vehicle distance detection training and test datasets. The original size of the dataset was 1242*375 pixels. After image preprocessing, it was compressed to 320*1024 pixels. The dataset was divided according to the ratio of train:val:test = 8:1:1.

[0012] Furthermore, step S1 includes the following:

[0013] 389 pairs of stereo images and optical flow maps, 29.2 km visual ranging sequences, 9300 RGBD training samples and depth maps, and over 200K images of 3D labeled objects, with images sampled at a frequency of 10 Hz and synchronized point cloud data.

[0014] Furthermore, step S2 includes the following:

[0015] Step S2-1: In the traditional U-Net sampling network, the upsampling part replaces the original transposed convolution with deconvolution operation, and adds 16 times the sampling layer on the basis of the U-Net sampling layer multiplication factor of 2 times, 4 times and 8 times;

[0016] Step S2-2: Add key modules. Based on the structure described in step S2-1, add a CRPBlock chained residual pooling module to the decoder network to fuse residual connections with learnable weights. Add a max pooling layer to the Encoder part to constrain the size of the feature map.

[0017] Step S2-3: Calculate photometric loss. Based on the structure described in step S2-2, calculate the photometric error on the output feature map. Using single-view reconstruction to learn feature representation will be helpful for the discrimination of textureless regions and surfaces with light reflection.

[0018] Furthermore, step S3 includes the following:

[0019] An attention module is introduced into the existing Monodepth2 network model. An attention module is added at the end of the Monodepth2 backbone network, that is, between the last feature extraction module and the skip connection module in the backbone network. The Self-Attention module is inserted to adaptively learn the correlation between different locations in the image.

[0020] Furthermore, step S4 includes the following:

[0021] The depth map output by the model is stored in uint16 format. Scale restoration is performed, and the data read from the depth map is compared with 256 to obtain the true distance value.

[0022] Compared with existing technologies, this invention, by adopting the above technical solution, has the following advantages and positive effects:

[0023] This invention presents a self-supervised monocular vehicle distance detection method, primarily addressing the issue of inaccurate distance estimation when using a monocular camera. Previous methods often exhibit estimation errors for textureless regions and reflective surfaces. When moving objects or motion blur are present in the image, the model suffers from inaccurate depth estimation. By adding an additional encoder-decoder structure to capture semantic information from the input image across multiple dimensions, the optimal photometric error is calculated, enabling the network to learn consistent feature representations. This optimizes the estimation of inaccurate distances on textureless and reflective surfaces. Furthermore, an adaptive attention mechanism is incorporated into the network structure to dynamically adjust attention weights, focusing the network on the detected vehicle and enhancing the model's robustness and generalization ability. After model selection, the KITTI dataset is downloaded for training. The trained model is then used for inference and compared with real depth data containing depth values ​​to further optimize model performance and meet the needs of practical applications. Attached Figure Description

[0024] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. In the drawings:

[0025] Figure 1 This is a flowchart of a self-supervised monocular vehicle distance detection method according to the present invention;

[0026] Figure 2 This is a schematic diagram of the network model structure of the present invention;

[0027] Figure 3 This is a schematic diagram of the CRPBlock structure of the present invention. Detailed Implementation

[0028] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0029] like Figure 1 As shown, this embodiment discloses a self-supervised monocular vehicle distance detection method, including the following steps:

[0030] Step S1: Download and process the KITTI dataset, randomly dividing the dataset into training and test sets;

[0031] Furthermore, step S1 includes the following:

[0032] The downloaded KITTI dataset was augmented by transforming and expanding the data to generate diverse training samples. The KITTI dataset uses real-shot images to construct the vehicle distance detection training and test datasets. The original size of the dataset was 1242*375 pixels. After image preprocessing, it was compressed to 320*1024 pixels. The dataset was divided according to the ratio of train:val:test = 8:1:1.

[0033] Furthermore, step S1 includes the following:

[0034] 389 pairs of stereo images and optical flow maps, 29.2 km visual ranging sequences, 9300 RGBD training samples and depth maps, and over 200K images of 3D labeled objects, with images sampled at a frequency of 10 Hz and synchronized point cloud data.

[0035] Step S2: Construct an autoencoder structure for feature extraction. The input is the original image, and the photometric error is calculated in the extracted feature map. The parameters are dynamically adjusted to minimize the error between the reconstructed image and the original image.

[0036] Furthermore, step S2 includes the following:

[0037] Step S2-1: In the traditional U-Net sampling network, the upsampling part replaces the original transposed convolution with deconvolution operation, and adds 16 times the sampling layer on the basis of the U-Net sampling layer multiplication factor of 2 times, 4 times and 8 times;

[0038] Step S2-2: Add key modules. Based on the structure described in step S2-1, add a CRPBlock chained residual pooling module to the decoder network to fuse residual connections with learnable weights. Add a max pooling layer to the Encoder part to constrain the size of the feature map.

[0039] Step S2-3: Calculate photometric loss. Based on the structure described in step S2-2, calculate the photometric error on the output feature map. Using single-view reconstruction to learn feature representation will be helpful for the discrimination of textureless regions and surfaces with light reflection.

[0040] Step S3: Construct a multi-dimensional model, build a pose network and a deep prediction network, and add an attention mechanism;

[0041] An attention module is introduced into the existing Monodepth2 network model. An attention module is added at the end of the Monodepth2 backbone network, that is, between the last feature extraction module and the skip connection module in the backbone network. The Self-Attention module is inserted to adaptively learn the correlation between different locations in the image.

[0042] Furthermore, step S3 includes the following:

[0043] A deep residual network is used as the backbone for feature extraction. The residual network can achieve skip connections, which can effectively transmit information from the upper network to the lower network.

[0044] The input image is convolved by a 7x7 convolutional layer with 64 channels and a stride of 2 to extract feature information. Next, the resulting feature map is downsampled. First, a downsampling layer with 128 output channels is used to reduce the spatial size of the feature map while preserving important feature information. After the downsampling layer, three blocks are added, containing several residual blocks and one more downsampling layer, progressively reducing the spatial size of the feature map.

[0045] After the last convolutional layer of the Encoder, a Self-Attention module is inserted to enhance the accuracy of the feature representation. This involves two stages:

[0046] Stage I: A 7x7 convolution kernel is used to convolve the original image, and then projected into three 1x1 convolutions to obtain an intermediate feature set containing 3xN feature maps.

[0047] Stage II: The intermediate features are clustered into N groups, each containing three feature maps, serving as the query, key, and value, respectively, following the traditional multi-head self-attention model. A lightweight fully connected layer and grouped convolutions are used for processing, ultimately yielding N feature maps, which are then used as one of the feature maps output by the Encoder.

[0048] The Decoder section consists of four blocks, several deconvolutional layers, convolutional layers, and Skip Connections. The first block contains one deconvolutional layer with 256 output channels; the other three blocks have 128, 64, and 32 output channels respectively; and the last layer is a 1x1 convolutional layer with 1 output channel, used to predict the depth map.

[0049] Step S4: Scale recovery is performed on the relative depth output by the model to convert it into absolute depth.

[0050] Furthermore, step S4 includes the following:

[0051] The depth map output by the model is stored in uint16 format. Scale restoration is performed, and the data read from the depth map is compared with 256 to obtain the true distance value.

[0052] Compared with existing technologies, this invention improves the network structure of monodepth2 by adding a CRPBlock chained residual pooling module and introducing a self-attention module to the decoder network. This allows the model to focus more on important feature regions, thereby improving model performance. The introduction of multiple encoder-decoder structures optimizes for estimation errors in textureless regions and surfaces with reflective lighting, as well as for inaccurate depth estimation when moving objects or motion blur are present in the image. Ultimately, this improves the accuracy of distance estimation and increases the model's robustness.

[0053] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A self-supervised monocular vehicle distance detection method, characterized in that, Includes the following steps: Step S1: Download and process the KITTI dataset, randomly dividing the dataset into training and test sets; Step S2: Construct an autoencoder structure for feature extraction. The input is the original image, and the photometric error is calculated in the extracted feature map. The parameters are dynamically adjusted to minimize the error between the reconstructed image and the original image. Step S2 includes the following: Step S2-1: In the traditional U-Net sampling network, the upsampling part replaces the original transposed convolution with deconvolution operation, and adds 16 times the sampling layer on the basis of the U-Net sampling layer multiplication factor of 2 times, 4 times and 8 times; Step S2-2: Add key modules. Based on the structure described in step S2-1, add a CRPBlock chained residual pooling module to the decoder network to fuse residual connections with learnable weights. Add a max pooling layer to the Encoder part to constrain the size of the feature map. Step S2-3: Calculate photometric loss. Based on the structure described in step S2-2, calculate the photometric error of the output feature map. Using single-view reconstruction to learn feature representation will be helpful for the discrimination of textureless regions and surfaces with light reflection. Step S3: Construct a multi-dimensional model, build a pose network and a deep prediction network, and add an attention mechanism; Step S3 includes the following: An attention module is introduced into the existing Monodepth2 network model. An attention module is added at the end of the Monodepth2 backbone network, that is, between the last feature extraction module and the skip connection module in the backbone network. The Self-Attention module is inserted to adaptively learn the correlation between different locations in the image. Step S4: Scale recovery is performed on the relative depth output by the model to convert it into absolute depth.

2. The self-supervised monocular vehicle distance detection method according to claim 1, characterized in that, Step S1 includes the following: The downloaded KITTI dataset was augmented by transforming and expanding the data to generate diverse training samples. The KITTI dataset uses real-shot images to construct the vehicle distance detection training and test datasets. The original size of the dataset was 1242*375 pixels. After image preprocessing, it was compressed to 320*1024 pixels. The dataset was divided according to the ratio of train:val:test= 8:1:

1.

3. The self-supervised monocular vehicle distance detection method according to claim 2, characterized in that, Step S1 includes the following: 389 pairs of stereo images and optical flow maps, 29.2 km visual ranging sequences, 9300 RGBD training samples and depth maps, and over 200K images of 3D labeled objects, with images sampled at a frequency of 10 Hz and synchronized point cloud data.

4. The self-supervised monocular vehicle distance detection method according to claim 1, characterized in that, Step S4 includes the following: The depth map output by the model is stored in uint16 format. Scale recovery is performed, and the data read from the depth map is compared with 256 to obtain the true distance value.

Images