Monocular distance measurement method, device, equipment, medium, and vehicle for intelligent vehicle driving

Abstract

The present invention relates to the technical field of vehicle intelligent driving, in particular to a monocular distance measurement method for vehicle intelligent driving, which includes the steps of: acquiring image information, inputting the image information into a first convolutional neural network model for detection, and obtaining a segmentation map and a depth map including an object detection frame; extracting a rectangular frame corresponding to each object detection frame, dividing the segmentation map by the rectangular frame, acquiring a region of interest map corresponding to the object, and inputting the region of interest map into a second convolutional neural network model for detection, and obtaining information of the object; processing the segmentation map and the depth map, and calculating a first distance by a clustering calculation model; combining the information of the object, and calculating a second distance by a geometric distance measurement model; and performing a fusion processing calculation on the first distance and the second distance to obtain a final distance to the object in real space. The monocular distance measurement method for vehicle intelligent driving of the present invention has high measurement accuracy, no limitation in use, and low implementation cost.

Description

[Technical Field]

[0001] The present invention relates to the technical field of intelligent vehicle driving, and in particular to a monocular distance measurement method, device, equipment, medium and vehicle for intelligent vehicle driving. [Background technology]

[0002] Advanced driver assistance systems (ADAS) use sensors installed in the vehicle to constantly sense the surrounding environment, acquire data, and recognize, detect, and track static and dynamic objects, while also combining this with map data from the navigator to perform systematic calculations and analysis to alert the driver to potential hazards in advance, effectively improving the comfort and safety of driving. Measuring the distance to the vehicle ahead is a basic requirement for ADAS, and is often used for functions such as adaptive cruise control (ACC) and automatic emergency braking (AEB). Therefore, high accuracy is required when calculating the distance to the vehicle ahead.

[0003] Conventional distance measurement methods based on binocular vision systems mainly estimate the distance to an obstacle in a scene based on calculating the disparity between two cameras. This method requires a large amount of calculation, and while ensuring accuracy, it also requires high computing power from the processing unit. It also requires the two cameras to be horizontally coaxial, with parallel optical axes, synchronized acquisition, and consistent exposure parameters, which greatly limits its practical application. Furthermore, the cost of installing two cameras and a processing unit with high computing power is high. Summary of the Invention [Problem to be solved by the invention]

[0004] The technical problem that the present invention aims to solve is that the distance measurement methods in the prior art cannot simultaneously consider measurement accuracy, application limitations, and cost. In order to solve this technical problem, the present invention provides a monocular distance measurement method for intelligent vehicle driving, which has high measurement accuracy, no application limitations, and low implementation costs. [Means for solving the problem]

[0005] The technical means adopted by the present invention to solve the technical problem are: Step S1: reading image information for each frame from a video stream captured by an on-board monocular camera; Step S2: inputting the image information into a first convolutional neural network model for detection, and obtaining a vehicle instance segmentation map and a depth map including at least one object detection frame; S3: extracting a rectangular frame corresponding to each of the object detection frames, dividing the vehicle instance segmentation map by the rectangular frame, obtaining a region of interest map corresponding to the object, and inputting the region of interest map into a second convolutional neural network model for detection, thereby obtaining information about the object, including the object category, actual height, and height of the object in the image; Step S4: extracting a depth ROI (region of interest) map using the object detection frame and the depth map, calculating the minimum distance of pixel points in the depth ROI map using a clustering calculation model, and acquiring coordinates s(u,v) of the pixel points, where the minimum distance of the pixel points is a first distance D1 from the vehicle-mounted monocular camera to the object in real space; Step S5: calculating a second distance D2 from the vehicle-mounted monocular camera to the object in real space by linking the information about the object with the coordinates s(u,v) of the pixel point using a geometric distance measurement model; and step S6 of performing a fusion processing calculation for the first distance D1 and the second distance D2 to obtain a final distance from the vehicle-mounted monocular camera to the object in real space.fruit, Specifically, the calculation by the clustering calculation model in step S4 is Suppose there are n pixel points in the depth ROI map, and in the initial calculation, each pixel point is set to a cluster S, and the depth value corresponding to the pixel point is set to dn in step S41; Step S42: calculating the distance between each pair of clusters and combining the two closest clusters into one cluster; Step S43: repeating step S42 until the distance between every two clusters becomes greater than a threshold T; Step S44: Calculating the number of all pixel points in the two clusters, respectively, to obtain the cluster SMax with the largest number of pixel points, reading the depth value corresponding to each pixel point in the cluster SMax, obtaining the minimum depth value dmin in the cluster SMax and the pixel point corresponding to the minimum depth value, and setting the minimum depth value dmin as the minimum distance between the pixel points; When the number of the minimum depth value dmin is 1, obtain the coordinates s(u,v) of the pixel point corresponding to the minimum depth value dmin; Alternatively, when the number of the minimum depth values ​​dmin is plural, there are plural coordinates of pixel points, the coordinate of the intersection point m in the pixel coordinate system o-uv of the optical axis of the in-vehicle monocular camera is (u0, v0), the Euclidean distance from each of the pixel points to the intersection point m is calculated, and when the Euclidean distance is the minimum value, the coordinate s(u, v) of the pixel point corresponding to the minimum depth value dmin is obtained; Specifically, in step S6, the standard deviation σ1 of the first distance D1 and the standard deviation σ2 of the second distance D2 are obtained, and the distance fusion standard deviation is calculated using the following formula:

number

number

[0006] More specifically, extracting rectangular frames corresponding to the object detection frames in step S3 is performed by: Step S31 of creating a pixel coordinate system o-uv; The upper boundary point (u t ,v t ), lower boundary point (u d ,v d ), left boundary point (u l ,v l ) and the right boundary point (u r ,v r ) in step S32; The rectangular frame of the object detection frame is drawn using the coordinates of the upper boundary point, the lower boundary point, the left boundary point, and the right boundary point, and h=v d -v t and step S33 of setting the height of the object in the image to be the height of the object in the image.

[0007] More specifically, the calculation using the geometric distance measurement model in step S5 is Step S51: creating a camera coordinate system O-XYZ with the on-board monocular camera as the origin, and defining the direction perpendicular to the pixel coordinate system o-uv plane as the Z-axis direction of the camera coordinate system; In the pixel coordinate system o-uv, the pixel point coordinate s(u,v) and the intersection point m(u0,v0) of the optical axis in the pixel coordinate system o-uv are known, the actual height of the object is H, the height of the object in the image is h, S(x,y,z) is the closest point of the object in the camera coordinate system, the SMN plane is parallel to the imaging plane, the intersection point of the optical axis and the SMN plane is M, and the M coordinate in the camera coordinate system is (0,0,z), and the following calculation formula is used:

number

[0008] More specifically, a Kalman filter algorithm is used to process the distance D3 from the vehicle-mounted monocular camera to the object in the final real space.

[0009] Furthermore, in step S6, the standard deviation σ1 of the first distance D1 and the standard deviation σ2 of the second distance D2 are obtained, and a distance fusion standard deviation calculation formula, i.e., k=σ2 2 / (σ1 2 +σ2 2 ) calculating the distance fusion standard deviation by The following calculation formula, i.e., D3=D1+k(D2-D1) and calculating a final real-space distance D3 from the vehicle-mounted monocular camera to the object based on the distance fusion standard deviation.

[0010] a processor; a memory used to store executable commands; The processor is a computing device used to read the executable command from the memory, execute the executable command, and realize the above-mentioned monocular distance measurement method for intelligent vehicle driving.

[0011] A computer-readable storage medium stores a computer program, which, when executed by a processor, causes the processor to realize the above-described monocular distance measurement method for intelligent vehicle driving.

[0012] The vehicle is equipped with the above-described monocular distance measurement device for intelligent vehicle driving. [Effects of the Invention]

[0013] (1) Compared to conventional technologies that estimate the distance to obstacles in a scene based on calculating the parallax between two cameras, the present application is not limited in use and has low implementation costs due to the video stream captured by the on-board monocular camera. (2) After processing the original image information, the first distance is calculated using a clustering algorithm model, which can effectively eliminate external interference and achieve high accuracy in the first distance result; by combining the acquired actual data and calculating the second distance using a geometric distance measurement model, the accuracy in the second distance result is high; and as a result of performing fusion calculations on the first distance and the second distance, the distance to the vehicle ahead in real space is obtained. Compared with the monocular distance measurement of the prior art, the present invention has high measurement accuracy, relatively low requirements on the computing power of the processing unit, and lower implementation costs.

[0014] The present invention will now be further described with reference to the accompanying drawings and examples. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a flowchart of a first embodiment of the present invention. [Figure 2] FIG. 2 is a schematic diagram of image information according to the first embodiment of the present invention. [Figure 3] FIG. 2 is a schematic diagram of a vehicle instance segmentation map according to the first embodiment of the present invention; [Figure 4] FIG. 2 is a schematic diagram of a depth map according to the first embodiment of the present invention; [Figure 5] FIG. 2 is a schematic diagram of a detection frame of an object in a pixel coordinate system according to the first embodiment of the present invention. [Figure 6] 1 is a partial depth ROI map according to the first embodiment of the present invention; [Figure 7] FIG. 2 is a schematic diagram of calculating the distance between two clusters in Example 1 of the present invention. [Figure 8] FIG. 2 is a schematic diagram illustrating the principle of a geometric distance measurement model in the first embodiment of the present invention. [Figure 9] FIG. 10 is a schematic diagram of the configuration of Example 2 of the present invention. [Figure 10] FIG. 10 is a schematic diagram of a hardware configuration according to a third embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0016] The present invention will now be described in more detail with reference to the accompanying drawings, all of which are simplified schematic diagrams that only show the basic configuration of the present invention, and therefore only show the configuration related to the present invention.

[0017] In describing the present invention, it should be understood that orientations or positional relationships indicated by terms such as "center," "longitudinal," "lateral," "length," "width," "thickness," "up," "down," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," "circumferential," and the like, are based on the drawings and are intended solely to facilitate and simplify the description of the present invention, and should not be construed as limiting the present invention, as they do not expressly or imply that the devices or elements described necessarily have, are configured, or operate in a particular orientation. Furthermore, features qualified as "first" or "second" may expressly or implicitly include one such feature or more such features. In describing the present invention, unless otherwise specified, "plurality" means two or more than two.

[0018] In the description of the present invention, it should be understood that unless otherwise clearly defined or limited, the terms "attach," "couple," and "connect" should be understood in a broad sense, for example, to mean fixedly connected, detachably connected, integrally connected, mechanically connected, electrically connected, directly connected, or indirectly connected via an intermediate medium, or to allow the interiors of two elements to communicate with each other. Those skilled in the art will be able to understand the specific meanings of the above terms in the present invention according to specific circumstances.

[0019] Embodiment 1: The embodiment of the present application provides a monocular distance measurement method for intelligent vehicle driving, including the following steps S1 to S6.

[0020] Step S1: Compared with the prior art, which reads image information frame by frame from the video stream captured by the on-board monocular camera and estimates the distance to an obstacle in the scene based on the calculation of the parallax between the two cameras, as shown in Figure 2, the video stream captured by the on-board monocular camera allows the present application to be used without any limitations, has a wide range of application, and has low implementation costs.

[0021] Step S2: Input the image information into a first convolutional neural network model for detection, and obtain a vehicle instance segmentation map and a depth map including at least one object detection frame.

[0022] The first convolutional neural network may use, but is not limited to, a SOLOv2 segmentation network configuration to obtain a vehicle instance segmentation map. The first convolutional neural network has a depth map prediction branch to obtain a depth map, and the first convolutional neural network achieves high accuracy and speed in image segmentation. When there is only one vehicle in the road ahead, there is only one object detection frame in the vehicle instance segmentation map. When there are two vehicles in the road ahead, there are two object detection frames in the vehicle instance segmentation map. The number of object detection frames in the vehicle instance segmentation map is the same as the number of vehicles in the road ahead, as shown in FIG. 3. Depth represents the distance of each point in the road ahead scene relative to the on-board monocular camera. That is, each pixel value in the depth map represents the distance of a point in the road ahead scene to the on-board monocular camera. A depth map is an image formed by using the vertical depth values ​​of an object instead of the grayscale of a grayscale image, and the length and width of the depth map are the same as the length and width of the image information. Figure 4 is a schematic diagram of the depth map of this embodiment, and in Figure 4, the darker the color, the closer the distance to the vehicle, and the lighter the color, the farther the distance to the vehicle.

[0023] Step S3: Extract a rectangular frame corresponding to each object detection frame, divide the vehicle instance segmentation map using the rectangular frame, obtain a region of interest map corresponding to the object, and input the region of interest map into a second convolutional neural network model for detection to obtain information about the object, including the object category, actual height, and height in the image of the object.

[0024] In this embodiment, as shown in FIG. 5, extracting rectangular frames corresponding to each object detection frame in step S3 specifically includes the following steps S31 to S33. S31: A pixel coordinate system o-uv is created. S32: Obtain the upper boundary point (ut, vt), lower boundary point (ud, vd), left boundary point (ul, vl) and right boundary point (ur, vr) of the object detection frame, and the object detection frame is a polygon, but is not limited to this. S33: A rectangular frame of the object detection frame is drawn using the coordinates of the upper boundary point, lower boundary point, left boundary point, and right boundary point, and h=vd-vt is set as the height of the object in the image.

[0025] In this embodiment, the vehicle instance segmentation map is segmented using a rectangular frame drawn based on an image segmentation algorithm to obtain a region of interest (ROI) map corresponding to the object, and the object type is obtained by detection using a second convolutional neural network model. The second convolutional neural network uses, but is not limited to, a ShuffleNetv2 network configuration, which improves the accuracy of object classification and speeds up processing. The object categories include, but are not limited to, passenger cars, SUVs, and buses. Furthermore, the actual height corresponding to the object is obtained according to the object category. For example, the actual height of a passenger car is 1.5m, the actual height of an SUV is 1.7m, and the height of a bus is 3m, which further improves the accuracy of the calculation results.

[0026] Step S4: Extract a depth ROI map using the object detection frame and the depth map, calculate the minimum distance of pixel points in the depth ROI map using the clustering calculation model, obtain the coordinates s(u,v) of the pixel points corresponding to the minimum distance of the pixel points, and the minimum distance of the pixel points is the first distance D1 from the vehicle-mounted monocular camera to the object in real space. Specifically, extract a depth ROI map using the object detection frame and the depth map, and combine the coordinates of the object detection frame in the vehicle instance segmentation map with the depth map to obtain a depth ROI (region of interest) map.

[0027] In this embodiment, the calculation using the clustering calculation model in step S4 includes the following steps S41 to S44. S41: Assume that there are n pixel points in the depth ROI map, and at the time of initial calculation, each pixel point is assigned to its own cluster S, and the depth value corresponding to the pixel point is set to dn. S42: Calculate the distance between every two clusters, and combine the two closest clusters into one cluster. Specifically, as shown in Figure 7, let the two clusters be SP and SQ, respectively. Here, cluster SP has AP pixel points, and cluster SQ has AQ pixel points. The difference between the depth value dp1 corresponding to pixel point AP1 and the depth value dQ1 corresponding to pixel point AQ1 is calculated, and the result dPQ1 is set as the distance between pixel point AP1 and pixel point AQ1. The distance between the two clusters, L(SP, SQ) = min(dpQ), is obtained. S43: Step S42 is repeated until the distance between the two clusters becomes greater than the threshold T. S44: Calculate the number of all pixel points in the two clusters, obtain the cluster SMax with the largest number of pixel points, read the depth value corresponding to each pixel point in the cluster SMax, obtain the minimum depth value dmin in the cluster SMax and the pixel point corresponding to the minimum depth value, and determine the minimum depth value dmin as the minimum distance between the pixel points. Here, when the number of minimum depth values ​​dmin is 1, the coordinate s(u,v) of the pixel point corresponding to the minimum depth value dmin is obtained; or when the number of minimum depth values ​​dmin is multiple, there are multiple coordinates of the pixel point, and the coordinate of intersection point m in the pixel coordinate system o-uv of the optical axis of the on-board monocular camera is (u0,v0). The Euclidean distance from each pixel point to intersection point m is calculated, and when the Euclidean distance is minimum, the coordinate s(u,v) of the pixel point corresponding to the minimum depth value dmin is obtained. After performing a fusion process between the coordinates of the object detection frame in the vehicle instance segmentation map and the depth map, the minimum distance of the pixel point is obtained, which is the shortest distance from the forward vehicle to the host vehicle. The coordinate of this pixel point in pixel coordinates is obtained using the minimum distance of the pixel point, which facilitates the calculation of subsequent steps, reduces calculation deviation, and improves the accuracy of the calculation results.

[0028] When an on-board monocular camera captures an image, the contours of nearby objects may be relatively complete, but the contours of distant objects may be missing due to the perspective relationship between the objects. The depth can be used to obtain the context of the objects, which can then be used to calculate the distance from the on-board monocular camera to a distant object. Taking the case of occlusion in the depth map as an example, Figure 6 shows a 16*16 partial depth ROI map, where the pixels in region A represent the depth of nearby objects, the pixels in region B represent the depth of distant objects, and the black pixels represent noise. When calculating the distance to distant objects, it is necessary to avoid the calculation interference caused by noise and nearby objects. Using the clustering calculation model of this embodiment, three clusters, namely, cluster SA in region A, cluster SB in region B, and abnormal cluster SY, are obtained. The number of pixels in each cluster is calculated, and cluster SA has the most pixels. The minimum depth value in cluster SA is the minimum distance, which is the first distance from the on-board monocular camera to the object in real space. The clustering calculation model can eliminate the interference that nearby objects and noise (including both nearby and distant noise) have on the calculation of the distance to the object, improving the accuracy and stability of the calculation results. Furthermore, accurate calculations can be performed regardless of the vehicle's orientation, position, or whether it is obstructed, making it applicable in a wide range of applications.

[0029] Step S5: The object information and the pixel point coordinates s(u, v) are linked together to calculate a second distance D2 from the vehicle-mounted monocular camera to the object in real space using a geometric distance measurement model.

[0030] In this embodiment, as shown in FIG. 8, the calculation using the geometric distance measurement model in step S5 is as follows: Step S51: creating a camera coordinate system O-XYZ with the on-board monocular camera as the origin, and setting the direction perpendicular to the pixel coordinate system o-uv plane as the Z-axis direction of the camera coordinate system; In the pixel coordinate system o-uv, the pixel point coordinate s(u,v) and the intersection point m(u0,v0) of the optical axis in the pixel coordinate system o-uv are known, the actual height of the object is H, the height of the object in the image is h, S(x,y,z) is the closest point of the object in the camera coordinate system, the SMN plane is parallel to the imaging plane, the intersection point of the optical axis and the SMN plane is M, and the M coordinate in the camera coordinate system is (0,0,z), and the following calculation formula is used:

number

[0031] Step S6: Fusion processing calculation is performed for the first distance D1 and the second distance D2 to obtain the final distance from the vehicle-mounted monocular camera to the object in real space.

[0032] Specifically, the standard deviation σ1 of the first distance D1 and the standard deviation σ2 of the second distance D2 are obtained, and the distance fusion standard deviation is calculated using the formula:

number

number

[0033] In this embodiment, the Kalman filter algorithm 3 is further used to process the final distance D from the vehicle-mounted monocular camera to the object in real space, and the distance obtained after processing further improves the measurement accuracy of the distance measurement method.

[0034] Specifically, the standard deviation σ1 of the first distance D1 and the standard deviation σ2 of the second distance D2 are obtained, and the distance fusion standard deviation is calculated using the formula: k=σ2 2 / (σ1 2 +σ2 2 ) The distance fusion standard deviation is calculated using the following formula: D3=D1+k(D2-D1) Based on the distance fusion standard deviation, the final real-space distance D3 from the vehicle-mounted monocular camera to the object is calculated.

[0035] In this embodiment, the greater the distance to the object, the greater the distance measurement error. In the experimental scene environment, a high-precision laser rangefinder is used to measure the actual distance DZS to the object, and 1000 sets of image information samples are obtained in steps S1 to S5 of this embodiment, and the first distance D1 and the second distance D2 are calculated respectively to obtain 1000 sets of [DZS, D1, D2] data, and X1 and X2 are the relative distance ratios of D1 and D2 respectively, and the calculation formula for the relative distance ratio is:

number

number

[0036] The monocular distance measurement method for intelligent vehicle driving of the present invention, compared to the prior art that estimates the distance to an obstacle in a scene based on calculating the parallax between two cameras, relies on a video stream captured by an on-board monocular camera, which is not limited in use and has low implementation costs. After processing the original image information, the first distance is calculated using a clustering algorithm model, which can effectively eliminate external interference and achieve high accuracy of the first distance result. By combining the acquired actual data and calculating the second distance using a geometric distance measurement model, the accuracy of the second distance result is also high. Furthermore, the distance to the vehicle ahead in real space is obtained as a result of fusion calculation of the first distance and the second distance. Compared to the prior art monocular distance measurement, the present invention has high measurement accuracy, relatively low requirements on the computing power of the processing unit, and lower implementation costs.

[0037] Embodiment 2: The embodiment of the present application is a monocular distance measurement device for intelligent vehicle driving using the above-mentioned monocular distance measurement method for intelligent vehicle driving, as shown in FIG. a first acquisition module 200 for reading image information frame by frame from a video stream captured by an on-board monocular camera; a first detection module 201 for inputting image information into a first convolutional neural network model for detection, and obtaining a vehicle instance segmentation map and a depth map including at least one object detection frame; a second detection module 202 for extracting a rectangular frame corresponding to each object detection frame, dividing the vehicle instance segmentation map by the rectangular frame, obtaining a region of interest map corresponding to the object, inputting the region of interest map into a second convolutional neural network model for detection, and obtaining information about the object, including the object category, actual height, and height in the image of the object; a first calculation module 203 that extracts a depth ROI map using the object detection frame and the depth map, calculates the minimum distance of pixel points in the depth ROI map using a clustering calculation model, and obtains the coordinates s(u,v) of the pixel points, where the minimum distance of the pixel points is a first distance D1 from the vehicle-mounted monocular camera to the object in real space; a second calculation module 204 that calculates a second distance D2 from the vehicle-mounted monocular camera to the object in real space by linking information about the object with the coordinates s(u,v) of the pixel point using a geometric distance measurement model; A monocular distance measurement device for intelligent vehicle driving is provided, which is equipped with a fusion calculation module 205 that performs fusion processing on the first distance D1 and the second distance D2 to obtain the final distance from the on-board monocular camera to the object in real space.

[0038] The first acquisition module 200 and the second detection module 202 are both connected to the first detection module 201, the first calculation module 203 is connected to the first detection module 201, the first calculation module and the second detection module are both connected to the second calculation module 204, and the first calculation module 203 and the second calculation module 204 are both connected to the fusion calculation module 205.

[0039] In the device provided in the above embodiments, the functions are realized by differentiating the above-mentioned functional modules only as an example, and in actual applications, the above functions may be completed by allocating them to different functional modules as needed, that is, it should be noted that the internal configuration of the device may be divided into different functional modules to achieve all or part of the above-described functions. Furthermore, the device provided in the above embodiments belongs to the same concept as the method embodiments, and the details of the specific realization process may be referred to the method embodiments, and are not repeated here.

[0040] Example 3: An example of the present application provides a computer device comprising a processor and a memory storing at least one command or at least one program segment, wherein the at least one command or the at least one program segment is loaded and executed by the processor to realize the monocular distance measurement method provided in the above method example.

[0041] FIG. 10 is a schematic diagram of the hardware configuration of a device for implementing the monocular distance measurement method provided in the embodiments of the present application. The device may be configured to constitute or include the device or system provided in the embodiments of the present application. As shown in FIG. 10, the computer device 10 may include one or more processors 1002 (the processor may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA), a memory 1004 for storing data, and a transmission device 1006 for communication functions. Other components may include a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that the configuration shown in FIG. 10 is merely schematic and does not limit the configuration of the electronic device. For example, the computer device 10 may include more or fewer components than those shown in FIG. 10 or may have a different arrangement than that shown in FIG. 10.

[0042] It should be noted that the one or more processors and / or other data processing circuits may be generally referred to herein as "data processing circuitry." This data processing circuitry may be implemented in whole or in part as software, hardware, firmware, or any other combination. It should be noted that the data processing circuitry may be a single, independent processing module, or may be incorporated in whole or in part into any one of the other elements of the computing device 10 (or mobile device). For example, as described in the embodiments herein, this data processing circuitry may act as a processor to control (e.g., select the path of a variable resistance terminal connected to an interface).

[0043] The memory 1004 may be configured to store software programs and modules for operating the software. For example, in a program command / data storage device corresponding to the monocular distance measurement method of an embodiment of the present application, the processor executes the software programs and modules stored in the memory 1004 to perform various function applications and data processing, i.e., to realize the above-described method. The memory 1004 may include high-speed random access memory or non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some embodiments, the memory 1004 may further include memory located remotely from the processor, which may be connected to the computer device 10 via a network. Examples of such networks include, but are not limited to, the Internet, a corporate intranet, a local area network, a mobile communication network, and combinations thereof.

[0044] The transmission device 1006 is for transmitting and receiving data via a network, which may include a wireless network provided by the communication provider of the computer device 10. In one example, the transmission device 1006 includes a network interface controller (NIC) that can connect to other network devices via a base station and communicate with the Internet. In one example, the transmission device 1006 may be a radio frequency (RF) module for communicating with the Internet wirelessly.

[0045] The display may be, for example, a touchscreen liquid crystal display (LCD) that allows a user to interact with the user interface of the computing device 10 (or mobile device).

[0046] Example 4: An embodiment of the present application further provides a computer-readable storage medium that can be installed on a server and stores at least one command or at least one program segment for realizing the monocular distance measurement method in the method embodiment, wherein the at least one command or the at least one program segment is loaded and executed by the processor to realize the monocular distance measurement method provided in the above method embodiment.

[0047] Optionally, in this embodiment, the storage medium may be located in at least one of a plurality of network servers of a computer network. Optionally, in this embodiment, the storage medium may include various media capable of storing program code, such as, but not limited to, a USB memory, a read-only memory (ROM), a random access memory (RAM), a removable hard disk, a magnetic disk, or an optical disk.

[0048] Embodiment 5: An embodiment of the present invention further provides a computer program product or a computer program including computer commands, the computer commands being stored in a computer-readable storage medium, wherein a processor of a computing device reads the computer commands from the computer-readable storage medium and executes the computer commands, thereby causing the computing device to perform the monocular distance measurement methods provided in the various alternative embodiments described above.

[0049] Embodiment 6: The embodiment of the present invention further provides a vehicle equipped with the above-mentioned monocular distance measurement device for vehicle intelligent driving.

[0050] It should be noted that the order of the above-described embodiments of the present application is for illustrative purposes only and does not represent the superiority or inferiority of the embodiments. Also, specific embodiments of the present application have been described above. Other embodiments are within the scope of the appended claims. The actions or steps recited in the claims may be performed in a different order than the examples and still achieve the desired results. Also, the steps depicted in the accompanying figures do not necessarily require the particular order or sequential order shown to achieve the desired results. Multitasking and parallel processing are also possible or may be beneficial in some embodiments.

[0051] The embodiments in this application will be described step by step, and the same or similar parts between the embodiments may be mutually referred to, and the description of each embodiment will focus on the differences from other embodiments. As for the embodiments of the device, apparatus, and storage medium, they are basically similar to the method embodiments, so the description will be relatively simple, and the relevant parts may be referred to in the description of the method embodiments.

[0052] As will be understood by those skilled in the art, all or part of the steps in the above embodiments may be realized by hardware, or by a program that issues commands to relevant hardware. The program may be stored in a computer-readable storage medium, and the storage medium may be a read-only memory, a magnetic disk, an optical disk, etc.

[0053] The above preferred embodiments of the present invention are merely illustrative, and it is understood that those skilled in the art may make various changes and modifications based on the above description without departing from the technical concept of the present invention. The technical scope of the present invention is not limited to the content of the specification, but should be determined by the claims. [Explanation of symbols]

[0054] 20 Monocular distance measurement device 200 First Acquisition Module 201 First Detection Module 202 Second Detection Module 203 First Calculation Module 204 Second Calculation Module 205 Fusion Computing Module 10. Computer Equipment 1002 processor 1004 memory 1006 Transmission equipment

Claims

1. Step S1: reading image information for each frame from a video stream captured by an on-board monocular camera; Step S2: inputting the image information into a first convolutional neural network model for detection, and obtaining a vehicle instance segmentation map and a depth map including at least one object detection frame; S3: extracting a rectangular frame corresponding to each of the object detection frames, dividing the vehicle instance segmentation map by the rectangular frame, obtaining a region of interest map corresponding to the object, and inputting the region of interest map into a second convolutional neural network model for detection, thereby obtaining information about the object, including the object category, actual height, and height of the object in the image; Step S4: extracting a depth ROI map using the object detection frame and the depth map, calculating a minimum distance between pixel points in the depth ROI map using a clustering calculation model, and acquiring coordinates s(u, v) of the pixel points, wherein the minimum distance between the pixel points is a first distance D1 from the vehicle-mounted monocular camera to the object in real space; a step S5 of calculating a second distance D2 from the on-board monocular camera to the object in real space using a geometric distance measurement model based on geometric dimensions by linking information about the object with the coordinates s(u, v) of the pixel point; and step S6 of performing a fusion processing calculation on the first distance D1 and the second distance D2 to obtain a final distance from the vehicle-mounted monocular camera to the object in real space, The calculation by the clustering calculation model in step S4 is Suppose there are n pixel points in the depth ROI map, and in the initial calculation, each pixel point is assigned to a cluster S, and the depth value corresponding to the pixel point is set to dn in step S41; Step S42: calculating the distance between each pair of clusters by calculating the difference in depth value between each cluster for pixel points in the depth ROI map, calculating the smallest difference among them as the distance between each pair of clusters, and combining the two closest clusters into one cluster; Step S43: repeating step S42 until the distance between every two clusters becomes greater than a threshold T; and a step S44 of calculating the number of all pixel points in the two clusters, respectively, to obtain a cluster SMax having the largest number of pixel points, reading a depth value corresponding to each pixel point in the cluster SMax, obtaining a minimum depth value dmin in the cluster SMax and a pixel point corresponding to the minimum depth value, and setting the minimum depth value dmin as the minimum distance between the pixel points, When the number of the minimum depth value dmin is 1, obtain the coordinates s(u, v) of the pixel point corresponding to the minimum depth value dmin; Alternatively, when the number of the minimum depth values ​​dmin is plural, there are plural coordinates of pixel points, the coordinate of the intersection point m in the pixel coordinate system o-uv of the optical axis of the on-vehicle monocular camera is (u0, v0), the Euclidean distance from each of the pixel points to the intersection point m is calculated, and when the Euclidean distance is the minimum value, the coordinate s(u, v) of the pixel point corresponding to the minimum depth value dmin is obtained; Specifically, in step S6, the standard deviation σ1 of the first distance D1 and the standard deviation σ2 of the second distance D2 are obtained, and the distance fusion standard deviation is calculated using the formula k=σ2 2 / (σ1 2 +σ2 2 ). calculating the distance fusion standard deviation by The following calculation formula: D3 = D1 + k (D2 - D1) and calculating a final distance D3 from the on-board monocular camera to the object in real space based on the distance fusion standard deviation.

2. Specifically, extracting rectangular frames corresponding to the object detection frames in step S3 is performed by: Step S31 of creating a pixel coordinate system o-uv; Step S32: acquiring an upper boundary point (ut, vt), a lower boundary point (ud, vd), a left boundary point (ul, vl), and a right boundary point (ur, vr) in the detection frame of the object; The monocular distance measurement method for intelligent driving of a vehicle as described in claim 1, characterized in that it includes a step S33 of drawing a rectangular frame of the object detection frame using the coordinates of the upper boundary point, the lower boundary point, the left boundary point, and the right boundary point, and setting h = vd - vt as the height of the object in the image.

3. In step S5, the calculation using the geometric distance measurement model is Step S51: creating a camera coordinate system O-XYZ with the on-board monocular camera as the origin, and setting the direction perpendicular to the pixel coordinate system o-uv plane as the Z-axis direction of the camera coordinate system; In the pixel coordinate system o-uv, the coordinates s (u, v) of the pixel point and the intersection m (u0, v0) of the optical axis in the pixel coordinate system o-uv are known, the actual height of the object is H, the height of the object in the image is h, S (x, y, z) is the closest point of the object in the camera coordinate system, the SMN plane on which the closest point of the object in the camera coordinate system is located is parallel to the imaging plane, the intersection point of the optical axis and the SMN plane is M, and the M coordinate in the camera coordinate system is (0, 0, z), and the following calculation formula, i.e., [Equation 9] and a step S52 of calculating a second distance D2 from the vehicle-mounted monocular camera to the object in real space by where dz is the length distance of OM, dx is the distance from S to the OYZ plane, dy is the distance from S to the OXZ plane, fx and fy are internal parameters representing the internal characteristics of the vehicle-mounted monocular camera, the internal parameters include focal length, coordinates of the principal point, and distortion coefficient, △u is the difference in distance between the pixel point s and the intersection point m in the u-axis direction, and △v is the difference in distance between the pixel point s and the intersection point m in the v-axis direction.

4. The monocular distance measurement method for intelligent vehicle driving described in claim 1 further comprises using a Kalman filter algorithm to process the final distance D3 from the on-board monocular camera to the object in real space.

5. A monocular distance measurement device for intelligent vehicle driving using the monocular distance measurement method for intelligent vehicle driving according to any one of claims 1 to 4, a first acquisition module (200) for reading image information frame by frame from a video stream captured by an on-board monocular camera; a first detection module (201) that inputs the image information into a first convolutional neural network model for detection, and obtains a vehicle instance segmentation map and a depth map including at least one object detection frame; a second detection module (202) that extracts a rectangular frame corresponding to each of the object detection frames, segments the vehicle instance segmentation map using the rectangular frame, obtains a region of interest map corresponding to the object, and inputs the region of interest map into a second convolutional neural network model for detection, thereby obtaining information about the object, including the object category, actual height, and height of the object in the image; a first calculation module (203) that extracts a depth ROI map using the object detection frame and the depth map, calculates the minimum distance of pixel points in the depth ROI map using a clustering calculation model, and acquires coordinates s(u, v) of the pixel points, where the minimum distance of the pixel points is a first distance D1 from the vehicle-mounted monocular camera to the object in real space; a second calculation module (204) that calculates a second distance D2 from the vehicle-mounted monocular camera to the object in real space by linking information about the object and the coordinates s(u, v) of the pixel point using a geometric distance measurement model; A monocular distance measurement device for intelligent vehicle driving, characterized by comprising: a fusion calculation module (205) that performs fusion processing on the first distance D1 and the second distance D2 to obtain the final distance from the on-board monocular camera to the object in real space.

6. a processor; a memory used to store executable commands; The processor reads the executable command from the memory, executes the executable command, and is used to realize the monocular distance measurement method for intelligent vehicle driving described in any one of claims 1 to 4.

7. A computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor realizes the monocular distance measurement method for intelligent vehicle driving described in any one of claims 1 to 4.

8. A vehicle comprising the monocular distance measurement device for intelligent vehicle driving according to claim 5.

Images