Vehicle driving obstacle distance detection method and system based on vehicle-mounted depth camera

Abstract

The invention provides a vehicle driving obstacle distance detection method and system based on a vehicle-mounted depth camera, and relates to the technical field of vehicle obstacle detection. Comprising the following steps: acquiring RGB image information, depth point cloud information and 3D coordinate information of a to-be-identified obstacle during vehicle driving; geometric features of key points in the RGB image information are extracted, and obstacles in the key points are recognized; aligning and integrating the 3D coordinate information and the depth point cloud information according to a timestamp, and combining the identified obstacle to obtain a 3D coordinate with the timestamp of the obstacle; and learning spatial features of different scales of the 3D coordinates with the timestamps of the obstacle, and performing feature extraction to obtain the distance between the obstacle and the vehicle body. According to the method, the RGB image information, the depth point cloud information and the 3D coordinate information of the to-be-recognized obstacle in the running process of the vehicle are acquired by using the depth camera installed on the vehicle body, and classification and regression are performed after different feature information is extracted by using the hybrid network model, so that the effects of efficiently and accurately recognizing the obstacle and predicting the distance between the obstacle and the vehicle body can be considered.

Description

Technical Field

[0001] The present invention belongs to the technical field of vehicle obstacle detection, and in particular relates to a method and system for detecting the distance of a vehicle to an obstacle based on a vehicle-mounted depth camera. Background Art

[0002] At present, the challenges faced by vehicle-mounted obstacle detection technology are mainly reflected in the limitations of its various components and the detection subject itself, such as:

[0003] Although ultrasonic radar is inexpensive, it has limited ability to identify obstacles at long distances and in complex environments, and its response to high-speed moving objects is not fast enough, which may lead to lag or misjudgment. Ultrasonic radar is also affected by factors such as the angle of the wave, the location of the layout, and the shape. In actual use, the ultrasonic radar envelope is cut by software means, which makes the effective utilization of the sensor not maximized.

[0004] Although lidar provides high-precision three-dimensional spatial information, its high equipment cost, large size and requirement for high-performance computer resources make its deployment and maintenance complicated, and it is inefficient when processing large-scale data.

[0005] Although millimeter-wave radar has strong penetration ability and certain medium-range detection capabilities, its accuracy is relatively low. Its recognition of target shape and size is not as accurate as that of lidar, and there may be blind spots in certain directions.

[0006] Infrared thermal imagers have unique advantages at night and in low-temperature environments, but their resolution is low and they mainly focus on detecting temperature differences, so their ability to identify object details and capture color information is limited.

[0007] In summary, when the prior art uses the above-mentioned device to detect vehicle obstacles, there are some deficiencies due to the limitations of the device itself, and it is difficult to simultaneously meet the purposes of efficient and accurate identification. Summary of the invention

[0008] In order to overcome the shortcomings of the above-mentioned prior art, the present invention provides a method and system for detecting the distance of obstacles to be identified while the vehicle is traveling based on a vehicle-mounted depth camera. The depth camera installed on the vehicle body is used to obtain RGB image information, depth point cloud information and 3D coordinate information of obstacles to be identified while the vehicle is traveling. A hybrid network model is used to extract different feature information and then classify and regress them, which can take into account the effects of efficient and accurate obstacle identification and obstacle and vehicle distance prediction.

[0009] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions:

[0010] A first aspect of the present invention provides a method for detecting a distance to an obstacle while a vehicle is traveling based on a vehicle-mounted depth camera.

[0011] The method for detecting the distance of a vehicle obstacle based on a vehicle-mounted depth camera comprises the following steps:

[0012] Obtain RGB image information, depth point cloud information and 3D coordinate information of obstacles to be identified while the vehicle is driving;

[0013] Extract the geometric features of key points in RGB image information and identify obstacles;

[0014] Align and integrate the 3D coordinate information and depth point cloud information according to the timestamp, and combine them with the identified obstacles to obtain the 3D coordinates of the obstacles with timestamps;

[0015] The spatial features of the obstacle’s 3D coordinates with timestamps at different scales are learned and feature extracted to obtain the distance between the obstacle and the vehicle body.

[0016] As a further technical solution, after obtaining the distance between the obstacle and the vehicle body, the following is also included:

[0017] Pre-set multiple warning distance levels between obstacles and the vehicle body;

[0018] Determine the specific alarm distance level that the distance between the obstacle and the vehicle body falls into, and issue a corresponding alarm message.

[0019] As a further technical solution, a depth camera installed on the vehicle body is used to obtain RGB image information, depth point cloud information and 3D coordinate information of obstacles to be identified while the vehicle is driving.

[0020] As a further technical solution, the geometric features of key points in the RGB image information are extracted to identify obstacles therein, specifically including:

[0021] Transposed convolution is used to process RGB images to achieve supersampling;

[0022] The super-sampling information of the RGB image is input into the graph neural network model to extract the geometric features of the key points; the residual block is introduced into the graph neural network model to solve the gradient vanishing and exploding problems in the deep neural network;

[0023] The geometric features of the key points in the extracted RGB image information are input into the residual convolutional neural network model to identify and classify the RGB image information to obtain the obstacle recognition results.

[0024] As a further technical solution, the 3D coordinate information and the depth point cloud information are aligned and integrated according to the timestamp, and combined with the identified obstacles to obtain the 3D coordinates of the obstacles with timestamps, specifically including:

[0025] Convert 3D coordinate information and depth point cloud information to the same coordinate system, and use point cloud alignment technology to achieve timestamp alignment operation;

[0026] Use the optical flow fusion algorithm to fuse the 3D coordinate information and the depth point cloud information together, and perform filtering operations on the fused data;

[0027] Based on the filtered data and combined with the specific location of the identified obstacle, the 3D coordinates of the obstacle with a timestamp are obtained.

[0028] As a further technical solution, the spatial features of the obstacle's 3D coordinates with timestamps at different scales are learned and feature extracted to obtain the distance between the obstacle and the vehicle body, including:

[0029] The 3D coordinates of the obstacle with timestamps are input into the temporal convolutional network model. First, DilatedConvolutions is used to process the image time series to increase the receptive field. A residual module is added between network layers to identify and classify RGB image information, so that the network can learn deeper features. A bidirectional TCN is added in the middle of the network layer, and the past and future information of the sequence is used to improve the prediction ability of the model and obtain the obstacle recognition result.

[0030] As a further technical solution, it also includes:

[0031] Through the conversion between world coordinates, the obtained camera coordinates of the obstacle are converted into two-dimensional pixel coordinates, and the corresponding obstacle visual warning information is displayed on the two-dimensional RGB image.

[0032] A second aspect of the present invention provides a vehicle obstacle distance detection system based on a vehicle-mounted depth camera.

[0033] The vehicle obstacle distance detection system based on the vehicle-mounted depth camera includes:

[0034] The information acquisition module is configured to: acquire RGB image information, depth point cloud information and 3D coordinate information of obstacles to be identified during vehicle driving;

[0035] The obstacle recognition module is configured to: extract geometric features of key points in the RGB image information and recognize obstacles therein;

[0036] The integration module is configured to: align and integrate the 3D coordinate information and the depth point cloud information according to the timestamp, and combine the identified obstacles to obtain the 3D coordinates of the obstacles with the timestamp;

[0037] The feature learning module is configured to learn the spatial features of different scales of the 3D coordinates of the obstacle with timestamps, and perform feature extraction to obtain the distance between the obstacle and the vehicle body.

[0038] A third aspect of the present invention provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps in the method for detecting the distance to a vehicle obstacle based on a vehicle-mounted depth camera as described in the first aspect of the present invention.

[0039] The fourth aspect of the present invention provides an electronic device, including a memory, a processor, and a program stored in the memory and executable on the processor. When the processor executes the program, the steps in the method for detecting the distance of a vehicle obstacle based on a vehicle-mounted depth camera as described in the first aspect of the present invention are implemented.

[0040] One or more of the above technical solutions have the following beneficial effects:

[0041] The present invention provides a method and system for detecting the distance of obstacles to be identified while the vehicle is traveling based on a vehicle-mounted depth camera. The method uses a depth camera installed on the vehicle body to obtain RGB image information, depth point cloud information and 3D coordinate information of obstacles to be identified while the vehicle is traveling. A hybrid network model is used to extract different feature information and then classify and regress them. This method can achieve both efficient and accurate obstacle identification and prediction of the distance between the obstacle and the vehicle body.

[0042] The hybrid network model proposed in the present invention includes a graph neural network model, a residual convolutional neural network model and a temporal convolutional network model. The graph neural network model is used to extract the geometric features of key points in RGB image information; the geometric features of key points in the extracted RGB image information are input into the residual convolutional neural network model, the RGB image information is identified and classified, and the obstacle recognition result is obtained; the 3D coordinates of the obstacle with timestamps are input into the temporal convolutional network model, and the 3D coordinates of the obstacle with timestamps are extracted through multiple stacked temporal convolutional layers to learn spatial features of different scales. Through the hybrid network model, more useful information can be integrated to accurately predict the distance of obstacles.

[0043] This paper applies the idea of ​​residual network to traditional graph convolutional neural network. The residual block can solve the gradient vanishing and explosion problems in deep neural network, so that the network can learn deeper features more effectively. Before the image data is input for recognition, the algorithm will first use Transposed convolution to process the image. This function can achieve supersampling, so that the network can generate high-resolution output.

[0044] The present invention fuses multiple time series convolutional networks. First, Dilated Convolutions is used to process the image time series to increase the receptive field, so that the model can capture longer time dependencies; residual modules are added between network layers to help solve the gradient vanishing problem in deep networks, so that the network can learn deeper features. In addition, the idea of ​​adding a bidirectional TCN network in the middle of the network layer is to add a bidirectional TCN in the middle of the network layer and use the past and future information of the sequence at the same time, thereby improving the prediction ability of the model.

[0045] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0047] Figure 1 This is a flow chart of the method of the first embodiment. DETAILED DESCRIPTION

[0048] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0049] It should be noted that the terms used herein are for describing specific embodiments only and are not intended to be limiting of exemplary embodiments according to the present invention.

[0050] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0051] Embodiment 1

[0052] Before explaining in detail the method for preventing rear-end collision of an automobile provided in the embodiment of the present application, the application scenario provided in the embodiment of the present application is first introduced.

[0053] Vehicle-mounted obstacle detection is an important aspect of vehicle intelligent identification. The existing methods of using ultrasonic radar, lidar, millimeter-wave radar and infrared thermal imager for vehicle-mounted obstacle detection often fail to achieve high efficiency, accuracy and cost savings due to some limitations of the equipment itself.

[0054] Based on such an application scenario, an embodiment of the present application provides a method for detecting the distance of obstacles when a vehicle is traveling based on a vehicle-mounted depth camera. The method uses a vehicle-mounted depth camera installed on the vehicle body to collect information, and uses a hybrid network model to process the collected information to obtain the distance between the obstacle and the vehicle body, and based on this, an alarm is issued to provide a certain warning to the driver.

[0055] Next, the vehicle obstacle distance detection method based on the vehicle-mounted depth camera provided in the embodiment of the present application will be explained in detail with reference to the accompanying drawings.

[0056] like Figure 1 As shown, the vehicle obstacle distance detection method based on the vehicle-mounted depth camera includes the following steps:

[0057] Obtain RGB image information, depth point cloud information and 3D coordinate information of obstacles to be identified while the vehicle is driving;

[0058] Extract the geometric features of key points in RGB image information and identify obstacles;

[0059] Align and integrate the 3D coordinate information and depth point cloud information according to the timestamp, and combine them with the identified obstacles to obtain the 3D coordinates of the obstacles with timestamps;

[0060] The spatial features of the obstacle’s 3D coordinates with timestamps at different scales are learned and feature extracted to obtain the distance between the obstacle and the vehicle body.

[0061] In this application, a hybrid network model is designed, including a graph neural network model, a residual convolutional neural network model, and a temporal convolutional network model, to extract features and identify obstacles from the RGB image information, depth point cloud information, and 3D coordinate information obtained by the vehicle-mounted depth camera, and finally obtain accurate distance information between the obstacle and the vehicle body. Specifically, it includes:

[0062] In some embodiments, after obtaining the distance between the obstacle and the vehicle body, the method further includes:

[0063] Pre-set multiple warning distance levels between obstacles and the vehicle body;

[0064] Determine the specific alarm distance level that the distance between the obstacle and the vehicle body falls into, and issue a corresponding alarm message.

[0065] It can be understood that a depth camera installed on the vehicle body can be used to obtain RGB image information, depth point cloud information and 3D coordinate information of obstacles to be identified while the vehicle is driving.

[0066] In some embodiments, extracting geometric features of key points in RGB image information and identifying obstacles therein specifically includes:

[0067] Transposed convolution is used to process RGB images to achieve supersampling;

[0068] The super-sampling information of the RGB image is input into the graph neural network model to extract the geometric features of the key points; the residual block is introduced into the graph neural network model to solve the gradient vanishing and exploding problems in the deep neural network;

[0069] The geometric features of the key points in the extracted RGB image information are input into the residual convolutional neural network model to identify and classify the RGB image information to obtain the obstacle recognition results.

[0070] In some embodiments, the 3D coordinate information and the depth point cloud information are aligned and integrated according to the timestamp, and combined with the identified obstacles to obtain the 3D coordinates of the obstacles with timestamps, specifically including:

[0071] Convert 3D coordinate information and depth point cloud information to the same coordinate system, and use point cloud alignment technology to achieve timestamp alignment operation;

[0072] Use the optical flow fusion algorithm to fuse the 3D coordinate information and the depth point cloud information together, and perform filtering operations on the fused data;

[0073] Based on the filtered data and combined with the specific location of the identified obstacle, the 3D coordinates of the obstacle with a timestamp are obtained.

[0074] In some embodiments, the spatial features of different scales of the 3D coordinates of the obstacle with timestamps are learned and feature extraction is performed to obtain the distance between the obstacle and the vehicle body, specifically including:

[0075] The 3D coordinates of the obstacle with timestamps are input into the temporal convolutional network model. First, DilatedConvolutions is used to process the image time series to increase the receptive field. A residual module is added between network layers to identify and classify RGB image information, so that the network can learn deeper features. A bidirectional TCN is added in the middle of the network layer, and the past and future information of the sequence is used to improve the prediction ability of the model and obtain the obstacle recognition result.

[0076] In some embodiments, it also includes:

[0077] Through the conversion between world coordinates, the obtained camera coordinates of the obstacle are converted into two-dimensional pixel coordinates, and the corresponding obstacle visual warning information is displayed on the two-dimensional RGB image.

[0078] All the above optional technical solutions can be combined in any way to form optional embodiments of the present application, and the embodiments of the present application will not be described in detail one by one.

[0079] More specifically:

[0080] (1) The improvement of the graph neural network model structure in this embodiment is that the idea of ​​residual network is applied to the traditional graph convolutional neural network. The residual block can solve the gradient vanishing and explosion problems in deep neural networks, so that the network can learn deeper features more effectively. In addition, before the image data is input for recognition, the algorithm will first use Transposed convolution to process the image. This function can achieve supersampling, so that the network can generate high-resolution output.

[0081] (2) The improvement of the temporal convolutional network model lies in the fusion of multiple temporal convolutional networks. The specific structure includes: first, using Dilated Convolutions to process the image time series to increase the receptive field, so that the model can capture longer time dependencies. Similarly, adding residual modules between network layers helps solve the gradient vanishing problem in deep networks, so that the network can learn deeper features. In addition, the idea of ​​adding a bidirectional TCN network in the middle of the network layer is to add a bidirectional TCN in the middle of the network layer and use the past and future information of the sequence at the same time, thereby improving the prediction ability of the model.

[0082] The 3D coordinates of the obstacles with timestamps are input into the temporal convolutional network model. The features of the 3D coordinates of the obstacles with timestamps are extracted through multiple stacked temporal convolutional layers to learn spatial features of different scales. The data processing process is to use different convolution kernels to perform convolution operations on the image to obtain feature attributes.

[0083] (3) Align and integrate the 3D coordinate information and the depth point cloud information according to the timestamp, and combine them with the identified obstacles to obtain the 3D coordinates of the obstacles with timestamps, where:

[0084] Integrate the 3D coordinate information and the depth point cloud information. Specifically, first, convert the 3D coordinate information and the depth point cloud information into the same coordinate system.

[0085] Secondly, point cloud alignment technology is used to utilize the transformation matrix between the two data sets, which can transform one data set into the reference coordinate system of another data set. Then, the optical flow fusion algorithm is used to fuse the 3D coordinate information and the depth point cloud information together. The fused data contains noise, so two filtering algorithms, Gaussian filtering and median filtering, are needed to improve the data quality.

[0086] Finally, the fused data can be visualized and analyzed using 3D visualization tools to extract useful information or perform further processing.

[0087] This embodiment captures RGB image information and depth point cloud information through a depth camera installed on the car. The graph convolutional neural network can extract key point geometric features from the RGB information in the image, and recognize and classify the image information in combination with the residual convolutional neural network structure. In the data preprocessing stage, the obtained 3D coordinates, depth information and timestamps are integrated to improve the diversity and completeness of the data set.

[0088] This embodiment uses a mixture of multiple network models to extract the obtained information, and uses GNN to perform dense sampling evenly at different positions of the image;

[0089] Convolution is used to predict and extract test results of different feature maps. The homogeneous transformation matrix obtained by convolution can establish the relationship between the two-dimensional mask and the three-dimensional point. This method uses the camera projection matrix algorithm to project the center of each voxel feature onto the image plane.

[0090] The temporal convolutional network (TCN) is used to extract features of 3D coordinates with timestamps. By stacking multiple TCN layers, the network can learn spatial features of different scales.

[0091] After using the hybrid network model to extract different feature information, classification and regression are performed. After the target is confirmed, the NMS algorithm is used to filter out the prediction boxes with large overlaps. The remaining prediction boxes are the target locations we have identified.

[0092] Finally, the algorithm defines the need to classify and divide the frequency and level of obstacle distance alarms. The camera coordinates of the obstacles are converted into two-dimensional pixel coordinates through the conversion between world coordinates, and the corresponding obstacle visual alarm information is obtained on the two-dimensional RGB image. When the obstacles are within different detection ranges, different levels of alarms are issued on the monitoring map. At the same time, the distance between the vehicle and the obstacle is displayed on the image through the 3D coordinate information.

[0093] Embodiment 2

[0094] This embodiment discloses a vehicle obstacle distance detection system based on a vehicle-mounted depth camera.

[0095] The vehicle obstacle distance detection system based on the vehicle-mounted depth camera includes:

[0096] The information acquisition module is configured to: acquire RGB image information, depth point cloud information and 3D coordinate information of obstacles to be identified during vehicle driving;

[0097] The obstacle recognition module is configured to: extract geometric features of key points in the RGB image information and recognize obstacles therein;

[0098] The integration module is configured to: align and integrate the 3D coordinate information and the depth point cloud information according to the timestamp, and combine the identified obstacles to obtain the 3D coordinates of the obstacles with the timestamp;

[0099] The feature learning module is configured to learn the spatial features of different scales of the 3D coordinates of the obstacle with timestamps, and perform feature extraction to obtain the distance between the obstacle and the vehicle body.

[0100] Embodiment 3

[0101] The purpose of this embodiment is to provide a computer-readable storage medium.

[0102] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the method for detecting the distance to an obstacle while the vehicle is traveling based on a vehicle-mounted depth camera as described in Embodiment 1 of the present disclosure.

[0103] Embodiment 4

[0104] The purpose of this embodiment is to provide an electronic device.

[0105] An electronic device includes a memory, a processor, and a program stored in the memory and executable on the processor. When the processor executes the program, the steps in the method for detecting the distance of a vehicle obstacle based on a vehicle-mounted depth camera as described in Example 1 of the present disclosure are implemented.

[0106] The steps involved in the apparatuses of the above embodiments 2, 3 and 4 correspond to the method embodiment 1, and the specific implementation methods can refer to the relevant description part of embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood to include any medium that can store, encode or carry an instruction set for execution by a processor and enable the processor to execute any method in the present invention.

[0107] Those skilled in the art should understand that the modules or steps of the present invention described above can be implemented by a general-purpose computer device, or alternatively, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. The present invention is not limited to any specific combination of hardware and software.

[0108] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A vehicle obstacle distance detection method based on a vehicle-mounted depth camera, characterized in that: The following steps are involved: Obtain RGB image information, depth point cloud information and 3D coordinate information of obstacles to be identified while the vehicle is driving; Extract the geometric features of key points in RGB image information and identify obstacles; Align and integrate the 3D coordinate information and depth point cloud information according to the timestamp, and combine them with the identified obstacles to obtain the 3D coordinates of the obstacles with timestamps; The spatial features of the obstacle’s 3D coordinates with timestamps at different scales are learned and feature extracted to obtain the distance between the obstacle and the vehicle body.

2. The method for detecting the distance of a vehicle obstacle based on a vehicle-mounted depth camera according to claim 1, characterized in that: After obtaining the distance between the obstacle and the vehicle, it also includes: Pre-set multiple warning distance levels between obstacles and the vehicle body; Determine the specific alarm distance level that the distance between the obstacle and the vehicle body falls into, and issue a corresponding alarm message.

3. The method for detecting the distance of a vehicle obstacle based on a vehicle-mounted depth camera according to claim 1, characterized in that: A depth camera installed on the vehicle body is used to obtain RGB image information, depth point cloud information and 3D coordinate information of obstacles to be identified while the vehicle is driving.

4. The method for detecting the distance of a vehicle obstacle based on a vehicle-mounted depth camera according to claim 1, characterized in that: Extract the geometric features of key points in the RGB image information and identify obstacles, including: Transposed convolution is used to process RGB images to achieve supersampling; The super-sampling information of the RGB image is input into the graph neural network model to extract the geometric features of the key points; the residual block is introduced into the graph neural network model to solve the gradient vanishing and exploding problems in the deep neural network; The geometric features of the key points in the extracted RGB image information are input into the residual convolutional neural network model to identify and classify the RGB image information to obtain the obstacle recognition results.

5. The method for detecting the distance of a vehicle obstacle based on a vehicle-mounted depth camera according to claim 1, characterized in that: The 3D coordinate information and the depth point cloud information are aligned and integrated according to the timestamp, and combined with the identified obstacles to obtain the 3D coordinates of the obstacles with timestamps, including: Convert 3D coordinate information and depth point cloud information to the same coordinate system, and use point cloud alignment technology to achieve timestamp alignment operation; Use the optical flow fusion algorithm to fuse the 3D coordinate information and the depth point cloud information together, and perform filtering operations on the fused data; Based on the filtered data and combined with the specific location of the identified obstacle, the 3D coordinates of the obstacle with a timestamp are obtained.

6. The method for detecting the distance of a vehicle obstacle based on a vehicle-mounted depth camera according to claim 1, characterized in that: Learn the spatial features of different scales of the 3D coordinates of the obstacle with timestamps, and perform feature extraction to obtain the distance between the obstacle and the vehicle body, including: The 3D coordinates of the obstacle with timestamps are input into the temporal convolutional network model. First, DilatedConvolutions is used to process the image time series to increase the receptive field. A residual module is added between network layers to identify and classify RGB image information, so that the network can learn deeper features. A bidirectional TCN is added in the middle of the network layer, and the past and future information of the sequence is used to improve the prediction ability of the model and obtain the obstacle recognition result.

7. The method for detecting the distance of a vehicle obstacle based on a vehicle-mounted depth camera according to claim 1, characterized in that: Also includes: Through the conversion between world coordinates, the obtained camera coordinates of the obstacle are converted into two-dimensional pixel coordinates, and the corresponding obstacle visual warning information is displayed on the two-dimensional RGB image.

8. A vehicle obstacle distance detection system based on a vehicle-mounted depth camera, characterized in that: include: The information acquisition module is configured to: acquire RGB image information, depth point cloud information and 3D coordinate information of obstacles to be identified during vehicle driving; The obstacle recognition module is configured to: extract geometric features of key points in the RGB image information and recognize obstacles therein; The integration module is configured to: align and integrate the 3D coordinate information and the depth point cloud information according to the timestamp, and combine the identified obstacles to obtain the 3D coordinates of the obstacles with the timestamp; The feature learning module is configured to learn the spatial features of different scales of the 3D coordinates of the obstacle with timestamps, and perform feature extraction to obtain the distance between the obstacle and the vehicle body.

9. A computer-readable storage medium having a program stored thereon, characterized in that: When the program is executed by a processor, the steps in the method for detecting the distance to an obstacle while the vehicle is traveling based on a vehicle-mounted depth camera as described in any one of claims 1 to 7 are implemented.

10. An electronic device comprising a memory, a processor, and a program stored in the memory and executable on the processor, wherein: The feature is that when the processor executes the program, the steps in the vehicle obstacle distance detection method based on a vehicle-mounted depth camera as described in any one of claims 1 to 7 are implemented.

Images