Low obstacle detection method and system based on binocular parallax and electronic equipment

Abstract

The invention discloses a low obstacle detection method and system based on binocular parallax and electronic equipment, and the method comprises the steps: judging whether there is an obstacle in a vehicle driving region based on a current frame reference image, and if yes, cutting the current frame reference image according to the vehicle driving region to obtain a cut image; edge extraction is carried out on the current frame of reference image after cutting to obtain an obstacle edge coordinate; according to the barrier edge coordinates, carrying out statistics on a barrier edge parallax value in the current frame parallax image, and calculating a barrier edge height based on a parallax algorithm and the barrier edge parallax value; judging whether the maximum value of the height of the edge of the obstacle is smaller than or equal to a preset height threshold value, if yes, judging that the obstacle in the vehicle driving area is a low obstacle, and if not, judging that the obstacle in the vehicle driving area is a dangerous obstacle. According to the technical scheme, the accuracy of obstacle detection in the vehicle driving track range can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of autonomous driving, and in particular to a low obstacle detection method based on binocular parallax, a low obstacle detection system based on binocular parallax, and an electronic device. Background Art

[0002] With the increasing popularity of autonomous driving technology, improving the safety of autonomous vehicles is a pressing issue. Autonomous Emergency Braking (AEB) is one of the core technologies that ensures autonomous driving safety. This technology proactively brakes the vehicle when it detects an obstacle ahead and the distance is less than the safe distance, ensuring driving safety.

[0003] Existing autonomous driving solutions typically use camera-based image detection and recognition for object detection and recognition, combined with LiDAR for obstacle ranging. Image-based object detection and recognition typically requires the annotation of numerous objects (such as vehicles, pedestrians, and traffic signs) to train deep learning networks. Consequently, trained deep learning models are often only able to recognize objects that were annotated during training.

[0004] Existing image annotation methods typically treat the target as a whole and use the distance measurement of this entire object as the final distance measurement result, which is then fed into the vehicle's autonomous driving system. This results in some special scenarios, such as roads under construction, where obstacles that are relatively low and do not affect normal vehicle traffic (such as the edge of a dirt pile) can still trigger the vehicle's warning system (such as AEB), affecting the driving experience. Summary of the Invention

[0005] The purpose of this application is to improve the accuracy of obstacle detection within the vehicle's driving trajectory, thereby improving the reliability of the automatic emergency braking system.

[0006] The technical solution of the first aspect of the present application is: providing a low obstacle detection method based on binocular disparity, the method comprising: judging whether there is an obstacle in the vehicle driving area based on the current frame reference image; if so, cropping the current frame reference image according to the vehicle driving area, and performing edge extraction on the cropped current frame reference image to obtain the obstacle edge coordinates; according to the obstacle edge coordinates, counting the obstacle edge disparity value in the current frame disparity map, and calculating the obstacle edge height based on the disparity algorithm and the obstacle edge disparity value; wherein the current frame disparity map corresponds to the current frame reference image; judging whether the maximum value in the obstacle edge height is less than or equal to a preset height threshold; if so, judging that the obstacle in the vehicle driving area is a low obstacle; if not, judging that the obstacle in the vehicle driving area is a dangerous obstacle.

[0007] In any of the above technical solutions, further, it is determined whether there are obstacles in the vehicle driving area, specifically including: dividing the space where the vehicle is located based on a preset size to generate a bird's-eye view grid map; projecting based on the current frame disparity map to generate a V disparity map, and generating a fitted road surface based on the V disparity map; obtaining a disparity point in any row of the current frame disparity map whose disparity value is greater than the corresponding disparity value of the fitted road surface, recording it as a target disparity point, and mapping the target disparity point to the bird's-eye view grid map based on the binocular camera parameters; obtaining the bird's-eye view grid occupied by the vehicle driving area in the bird's-eye view grid map, and judging whether there are obstacles in the vehicle driving area based on whether the occupied bird's-eye view grid contains the target disparity point, wherein when the occupied bird's-eye view grid contains the target disparity point, it is determined that there is an obstacle in the vehicle driving area; otherwise, it is determined that there is no obstacle in the vehicle driving area.

[0008] In any of the above technical solutions, the method further includes: when it is determined that the vehicle is in a turning state, calculating the inner wheel difference of the vehicle based on the vehicle turning information; superimposing the inner wheel difference to the vehicle driving area, and judging whether there is an obstacle based on the superimposed vehicle driving area.

[0009] In any of the above technical solutions, the method further includes: extracting historical data to obtain historical frame reference images and historical frame disparity maps and historical vehicle positions corresponding to the historical frame reference images; determining the position offset between the vehicle position at the current moment and the historical vehicle position; performing image processing on the historical frame reference image based on the position offset, and judging whether there are obstacles in the superimposed vehicle driving area based on the processed historical frame reference image, wherein the image processing includes at least any one of translation, rotation, and scaling.

[0010] In any of the above technical solutions, the method further includes: extracting historical data frame by frame based on a preset step size, wherein the historical data includes a historical frame reference image, a corresponding historical frame disparity map, and a historical vehicle position when the historical frame reference image is obtained; based on the historical vehicle position, calculating the historical edge coordinates of the obstacle edge coordinates in the historical frame reference image by means of coordinate inverse operation; based on the historical edge coordinates, counting the historical obstacle edge disparity values ​​in the historical frame disparity map, and calculating the historical obstacle edge height based on the disparity algorithm and the historical obstacle edge disparity values; based on the historical obstacle edge height, updating the obstacle edge height by a weighted manner, and judging whether the maximum value of the updated obstacle edge height is less than or equal to the preset height threshold.

[0011] In any of the above technical solutions, the method further includes: determining an edge optimization range based on a set edge margin and with any obstacle edge coordinate as the center; optimizing the obstacle edge disparity value based on the disparity value within the edge optimization range, wherein the optimized obstacle edge disparity value is the mean or median of the disparity values ​​within the edge optimization range.

[0012] The technical solution of the second aspect of the present application is: providing a low obstacle detection system based on binocular parallax, the system including: an edge determination unit, the edge determination unit is configured to determine whether there is an obstacle in the vehicle driving area based on the current frame reference image, if so, the current frame reference image is cropped according to the vehicle driving area, and the edge of the cropped current frame reference image is extracted to obtain the obstacle edge coordinates; a height calculation unit, the height calculation unit is configured to count the obstacle edge disparity value in the current frame disparity map according to the obstacle edge coordinates, and calculate the obstacle edge height based on the disparity algorithm and the obstacle edge disparity value; wherein the current frame disparity map corresponds to the current frame reference image; a judgment unit, the judgment unit is configured to determine whether the maximum value of the obstacle edge height is less than or equal to a preset height threshold, if so, the obstacle in the vehicle driving area is determined to be a low obstacle, if not, the obstacle in the vehicle driving area is determined to be a dangerous obstacle.

[0013] In any of the above technical solutions, further, the edge determination unit is also configured to: divide the space where the vehicle is located based on a preset size to generate a bird's-eye view grid map; perform projection based on the current frame disparity map to generate a V disparity map, and generate a fitted road surface based on the V disparity map; obtain a disparity point in any row of the current frame disparity map whose disparity value is greater than the corresponding disparity value of the fitted road surface, record it as a target disparity point, and map the target disparity point to the bird's-eye view grid map based on the binocular camera parameters; obtain the bird's-eye view grid occupied by the vehicle's driving area in the bird's-eye view grid map, and judge whether there is an obstacle in the vehicle's driving area based on whether the occupied bird's-eye view grid contains the target disparity point, wherein when the occupied bird's-eye view grid contains the target disparity point, it is determined that there is an obstacle in the vehicle's driving area; otherwise, it is determined that there is no obstacle in the vehicle's driving area.

[0014] In any of the above technical solutions, further, the edge determination unit is also configured to: when it is determined that the vehicle is in a turning state, calculate the inner wheel difference of the vehicle based on the vehicle turning information; superimpose the inner wheel difference to the vehicle driving area, and based on the superimposed vehicle driving area, determine whether there is an obstacle.

[0015] The technical solution of the third aspect of the present application is: to provide an electronic device, which includes: a processor; a memory for storing executable instructions of the processor; wherein the processor is configured to implement the low obstacle detection method based on binocular parallax in any one of the above-mentioned technical solutions of the first aspect when executing the executable instructions.

[0016] The beneficial effects of this application are:

[0017] The technical solution in this application clips the obstacles within the driving area of ​​the vehicle in front of the vehicle, maps the coordinates of the obtained part of the obstacles to the disparity map, and then calculates the obstacle edge disparity value corresponding to this range, thereby obtaining the obstacle height of this part of the obstacle within the vehicle's driving area, which is used as the basis for judging whether the vehicle can pass safely, avoiding directly using the overall height of the obstacle, so as to realize the detection of low obstacles / low areas of obstacles, improve the accuracy of obstacle detection within the vehicle's driving trajectory, and thus improve the reliability of the automatic emergency braking system.

[0018] When judging whether there is an obstacle in the driving area in front of the vehicle, the present application does not need to use a network model. Instead, the disparity map is first projected onto a bird's-eye view grid map, and then the grid occupied by the vehicle's driving area is judged to determine whether there is an obstacle in the vehicle's driving area. This helps to realize general obstacle detection, reduces the computing power occupied by the method, and reduces the implementation cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The advantages of the above and / or additional aspects of the present application will become apparent and readily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0020] Figure 1 It is a schematic diagram of a bird's-eye view of obstacles ahead during the vehicle's autonomous driving process;

[0021] Figure 2 is a schematic flow chart of a low obstacle detection method based on binocular parallax according to an embodiment of the present application;

[0022] Figure 3 is a schematic diagram of cropping a reference image according to a vehicle driving area according to an embodiment of the present application;

[0023] Figure 4 is a schematic diagram of a bird's-eye view grid map according to one embodiment of the present application;

[0024] Figure 5 This is a schematic block diagram of low obstacle detection based on binocular parallax according to an embodiment of the present application. DETAILED DESCRIPTION

[0025] In order to more clearly understand the above-mentioned objectives, features and advantages of the present application, the present application is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features therein can be combined with each other in the absence of conflict.

[0026] In the following description, many specific details are set forth to facilitate a full understanding of the present application. However, the present application may also be implemented in other ways different from those described herein. Therefore, the scope of protection of the present application is not limited to the specific embodiments disclosed below.

[0027] Example 1:

[0028] like Figure 1 As shown, using a bird's-eye view autonomous driving solution as an example, when the vehicle control system determines that obstacle 102 is located on driving trajectory 103 ahead of vehicle 101 and its distance is less than a safety threshold, it issues an alarm to ensure safe driving. However, the area within driving trajectory 103 where obstacle 102 lies may be entirely or partially low, such as a pile of sand or dirt. In this case, obstacle 102 does not affect the passage of vehicle 101. Therefore, such an alarm is a false alarm and should be avoided.

[0029] Therefore, this embodiment, based on binocular disparity technology, proposes a method for low obstacle detection in bird's-eye-view scenarios. When an obstacle is determined to be within the vehicle's driving trajectory, especially if only a portion of the obstacle is present, the edges of the obstacle image within the driving trajectory are obtained through cropping and edge extraction. The extracted edges are then mapped to the disparity map, and the spatial information (such as height) of the obstacle edge area is obtained through disparity calculation. If the height of the extracted edge area is below a height threshold, it can be determined to be a low obstacle, and the vehicle can safely pass without triggering an alarm or AEB braking.

[0030] like Figure 2 As shown, this embodiment provides a low obstacle detection method based on binocular parallax, the method comprising:

[0031] Step 100: Based on the current frame reference image, determine whether there is an obstacle in the vehicle's driving area. If so, crop the current frame reference image according to the vehicle's driving area, and perform edge extraction on the cropped current frame reference image to obtain the obstacle edge coordinates. If not, there is no need to perform subsequent steps, and the corresponding historical data can be stored.

[0032] Specifically, during vehicle driving, the vehicle's trajectory in its current state can be calculated using kinematic equations based on the vehicle's driving information (such as steering angle), the vehicle's own information (such as body size), and other relevant information.

[0033] Then, combined with the preset safety distance, such as the vehicle's maximum braking distance, the vehicle's driving area can be obtained. If there is an obstacle in this area, the vehicle's driving needs to be controlled, such as steering and braking, to avoid the risk of collision.

[0034] In this embodiment, after the binocular camera is mounted on a vehicle and calibrated, a binocular image of the vehicle's forward field of view (left and right images) can be obtained. Furthermore, the vehicle's driving area can be mapped onto the image to determine the image coordinates of the vehicle's driving area.

[0035] Then, using either image as a reference image (e.g., the left image), a binocular disparity algorithm can be used to calculate the disparity map corresponding to the reference image. At the same time, a network model can be used for object detection. The reference image is fed into an object detection network model to detect objects in the image and obtain the pixel coordinates of each object in the image.

[0036] Finally, by combining the internal and external parameters and disparity map of the binocular camera, the spatial information of each target in the reference image can be calculated. This spatial information can include the target's spatial coordinates, distance from the vehicle, target size, etc., where the above spatial information corresponds one-to-one with the target detection results.

[0037] It should be noted that the detection results of the target detection model are determined by the labels during network training. They can be conventional labels such as vehicles, pedestrians, traffic cones, etc., or some non-standard general labels such as piles of earth, piles of stones, and objects scattered on the road.

[0038] Therefore, after obtaining the vehicle's driving area, we can determine whether the obstacle is completely or partially located in the driving area based on the spatial information of the obstacle (target). If not, the vehicle can pass through normally; if it is, we need to further determine whether it will affect the vehicle's driving.

[0039] The following is an example of the existence of some obstacles in the vehicle's driving area.

[0040] like Figure 3 As shown, the straight arrow points in the direction of vehicle 301's travel. The pixel coordinates of obstacle 302 (e.g., target frame coordinates) are compared with the image coordinates of vehicle travel area 303. If the right side of obstacle 302 is determined to be within vehicle travel area 303 (i.e., there is an overlap in coordinates), the reference image can be cropped according to the image coordinates of vehicle travel area 303 in the reference image. Edge extraction (e.g., using the Canny edge detection algorithm) is then performed to obtain the pixel coordinates of the right side of obstacle 302 in the reference image, i.e., the obstacle edge coordinates.

[0041] Step 200: Counting the obstacle edge disparity values ​​in the current frame disparity map according to the obstacle edge coordinates, and calculating the obstacle edge height based on the disparity algorithm and the obstacle edge disparity values; wherein the current frame disparity map corresponds to the current frame reference image.

[0042] Specifically, since the pixels in the reference image correspond one-to-one to the disparity points in the disparity map, the corresponding disparity points and disparity values ​​of the edge coordinates of the right side of the obstacle 302 in the disparity map, i.e., the obstacle edge disparity values, can be obtained by coordinate mapping.

[0043] Afterwards, a disparity algorithm may be used, such as converting the disparity points into three-dimensional space coordinates, to calculate the spatial information of the right side of the obstacle 302 , and further obtain the corresponding obstacle edge height.

[0044] It should be noted that when calculating the height of the obstacle edge, the vanishing point in the image can also be used for calculation. The corresponding calculation formula is:

[0045]

[0046] Road_y=Road_k*Disp+Road_b

[0047] Where H is the height of the pixel point / disparity point from the road surface, Road_y is the vertical coordinate of the road surface at the disparity point in the image, pixel_y is the horizontal coordinate of the pixel coordinate corresponding to the disparity point, F is the focal length of the binocular camera, Road_k ​​is the slope of the road surface equation, Disp is the disparity value of the disparity point, and Road_b is the intercept of the road surface equation. The slope Road_k ​​and intercept Road_b can be obtained by fitting the road surface in the V-disparity map after projecting the disparity map.

[0048] It should be noted that when calculating the obstacle edge height, you can also: based on the set edge margin, determine the edge optimization range with any obstacle edge coordinate as the center, and optimize the obstacle edge disparity value based on the disparity value within the edge optimization range, wherein the optimized obstacle edge disparity value can be the mean or median of the disparity values ​​within the edge optimization range.

[0049] Specifically, when calculating the obstacle edge height, an edge margin can be set, such as ±1% of the image resolution. Taking any obstacle edge coordinate as the center, the coordinates of the pixel row where it is located are offset in both directions to determine the edge optimization range. The disparity values ​​within this range are then counted. Finally, the mean or median of these disparity values ​​is selected as the optimized obstacle edge disparity value.

[0050] Step 300: Determine whether the maximum value of the obstacle edge height is less than or equal to a preset height threshold. If so, the obstacle in the vehicle's driving area is determined to be a low obstacle, and there is no need to trigger the alarm of the automatic driving system (such as AEB), and the vehicle can pass normally; if not, the obstacle in the vehicle's driving area is determined to be a dangerous obstacle, triggering the alarm of the automatic driving system, controlling the vehicle's braking, and avoiding a collision.

[0051] In any of the above embodiments, further, in order to reduce the amount of calculation and the computing power occupied by the network model, this embodiment first projects the disparity map onto the bird's-eye view grid map, and then determines whether there are obstacles in the vehicle's driving area by determining whether the grid occupied by the vehicle's driving area contains the projected coordinates, thereby achieving universal obstacle detection, which helps and improves the accuracy of determining whether there are obstacles in the vehicle's driving area. Therefore, in step 100, determining whether there are obstacles in the vehicle's driving area specifically includes:

[0052] Step 101: Divide the space where the vehicle is located based on a preset size and generate a bird's-eye view grid map;

[0053] Step 102: Projecting the current frame disparity map to generate a V disparity map, and generating a fitted road surface based on the V disparity map;

[0054] Step 103: Obtain a disparity point in any row of the disparity map of the current frame whose disparity value is greater than the disparity value corresponding to the fitted road surface, record it as a target disparity point, and map the target disparity point to the bird's-eye view grid map based on the binocular camera parameters;

[0055] Step 104: Obtain the bird's-eye view grids occupied by the vehicle's driving area in the bird's-eye view grid map, and determine whether there are obstacles in the vehicle's driving area based on whether the occupied bird's-eye view grids contain the target parallax point.

[0056] When the occupied bird's-eye view grid contains the target disparity point, it is determined that there is an obstacle in the vehicle's driving area; otherwise, it is determined that there is no obstacle in the vehicle's driving area.

[0057] Specifically, such as Figure 4 As shown, first, a spatial coordinate system is established with the center of vehicle 401 as the origin, the vehicle's driving direction as the positive direction of the longitudinal axis (Z axis), and the horizontal right direction of the vehicle as the positive direction of the transverse axis (X axis). The space where the vehicle is located is divided with 10 cm as the preset size to generate a bird's-eye view grid map.

[0058] It should be noted that the preset size may also be other values, and its value may be within the range of 5-50 cm.

[0059] Next, the disparity map calculated by the binocular camera is projected along the vertical axis to obtain a V-disparity map. The horizontal axis of the V-disparity map is the disparity value, and the vertical axis is the number of image rows. Through linear fitting, a straight line with a negative slope is obtained, which is the fitted road surface.

[0060] Therefore, for any row of disparity points in the disparity map, if the disparity value of the disparity point is greater than the disparity value of the fitted road surface at that row, the disparity point is located above the road surface and is recorded as the target disparity point.

[0061] Then, based on the binocular camera parameters and the pixel coordinates corresponding to the target disparity point, the spatial coordinates of the disparity point in the bird's-eye view can be calculated. The corresponding calculation formula is:

[0062]

[0063] Where Z is the longitudinal distance of the target disparity point, K is the preset coefficient, B is the baseline of the binocular camera, F is the focal length of the binocular camera, Disp is the disparity value of the target disparity point, X is the lateral distance of the target disparity point, pixel_x is the lateral coordinate in the pixel coordinates corresponding to the target disparity point, and vanish_x is the lateral coordinate of the vanishing point of the binocular camera.

[0064] Therefore, after obtaining the longitudinal distance Z and the lateral distance X of the target disparity point, it can be mapped to the bird's-eye view grid map.

[0065] Finally, the bird's-eye view grids occupied by vehicle driving area 402 in the bird's-eye view grid map are obtained. Although vehicle driving area 402 only occupies a small portion of the area, bird's-eye view grid 403 is still marked as an occupied bird's-eye view grid. If an occupied bird's-eye view grid contains the target disparity point, it is determined that an obstacle exists within the vehicle driving area. Otherwise, it is determined that no obstacle exists within the vehicle driving area.

[0066] It should be noted that if any bird's-eye view grid contains only one disparity point or the number of disparity points is less than the filtering threshold, the disparity points in the bird's-eye view grid will be filtered out.

[0067] In any of the above embodiments, further, considering that the vehicle is a rigid body, especially for a large vehicle, there is an inner wheel difference during a turn. Therefore, in order to ensure the safety of the vehicle, the method further includes:

[0068] When it is determined that the vehicle is in a turning state, the inner wheel difference of the vehicle is calculated based on the vehicle turning information, such as the arc radius of the inner and outer wheels of the vehicle;

[0069] The inner wheel difference is superimposed on the vehicle's driving area, and based on the superimposed vehicle driving area, it is determined whether there is an obstacle.

[0070] In any of the above embodiments, further, Figure 4 As shown, after the binocular camera is installed on the vehicle, due to its field of view, the inner wheel difference area of ​​the vehicle will inevitably contain a certain range of blind spots. In order to detect low obstacles in this area, this embodiment performs operations such as translation, rotation, and scaling on the image at the historical moment based on the position offset between the vehicle at the historical moment and the current moment, transforming it into a predicted image at the current moment, thereby completing the information of the above-mentioned blind spots. Therefore, the method also includes:

[0071] The historical data is extracted to obtain the historical frame reference image and the historical frame disparity map and historical vehicle position corresponding to the historical frame reference image; wherein the extracted historical data can be one frame or multiple frames, and each time processing is performed, a single frame is processed.

[0072] Determine the position offset between the current vehicle position and the historical vehicle position; wherein the position offset is the difference between the position coordinates of the two moments.

[0073] Based on the position offset, the historical frame reference image is processed, and based on the processed historical frame reference image, it is determined whether there is an obstacle in the superimposed vehicle driving area, and step 100 is executed, wherein the image processing includes at least any one of translation, rotation, and scaling.

[0074] In any of the above embodiments, further, considering that the detection range of the AEB system is generally smaller than the effective field of view of the binocular camera, for example, the detection range of the AEB system is 1-20 meters, while the effective field of view of the binocular camera is 0.5-50 meters, to ensure the accuracy of low obstacle detection, data within the range of 20-50 meters may be selected as enhanced data to enhance the data within the detection range of the AEB system. The method further includes:

[0075] Based on a preset step size, historical data is extracted frame by frame, wherein the historical data includes a historical frame reference image, a corresponding historical frame disparity map, and a historical vehicle position when the historical frame reference image is obtained;

[0076] Among them, the preset step size can be set according to actual needs. The selected historical frame reference images can be 5-10 frames, and the historical data can be data stored 3-5 seconds ago, including historical frame reference images, historical frame disparity maps, and historical vehicle positions can be determined by satellite positioning signals, odometers, etc.

[0077] Based on the historical vehicle position, the historical edge coordinates of the obstacle edge in the historical frame reference image are calculated by coordinate inversion operation;

[0078] Specifically, for a stationary obstacle, its spatial coordinates are fixed values. Therefore, if the obstacle's current position in a coordinate system with the vehicle center as the origin and the vehicle's position change are known, the obstacle's position in a coordinate system with the vehicle center as the origin at a historical moment can be calculated through inverse calculation, i.e., the historical edge coordinates.

[0079] Based on the historical edge coordinates, the historical obstacle edge disparity values ​​in the historical frame disparity map are counted, and the historical obstacle edge height is calculated based on the disparity algorithm and the historical obstacle edge disparity values;

[0080] Based on the historical obstacle edge heights, the obstacle edge heights are updated in a weighted manner, and it is determined whether the maximum value of the updated obstacle edge heights is less than or equal to a preset height threshold.

[0081] Example 2:

[0082] like Figure 5 As shown, this embodiment provides a low obstacle detection system 500 based on binocular parallax, and the system 500 includes:

[0083] The edge determination unit 501 is configured to determine whether there is an obstacle in the vehicle's driving area based on the current frame reference image. If there is an obstacle, the current frame reference image is cropped according to the vehicle's driving area, and edge extraction is performed on the cropped current frame reference image to obtain the obstacle edge coordinates.

[0084] A height calculation unit 502 is configured to calculate the obstacle edge disparity value in the current frame disparity map according to the obstacle edge coordinates, and calculate the obstacle edge height based on the disparity algorithm and the obstacle edge disparity value; wherein the current frame disparity map corresponds to the current frame reference image;

[0085] The judgment unit 503 is configured to judge whether the maximum value of the obstacle edge height is less than or equal to a preset height threshold. If so, the obstacle in the vehicle driving area is judged to be a low obstacle. If not, the obstacle in the vehicle driving area is judged to be a dangerous obstacle.

[0086] In any of the above technical solutions, further, the edge determination unit 501 is also configured to: divide the space where the vehicle is located based on a preset size to generate a bird's-eye view grid map; perform projection based on the current frame disparity map to generate a V disparity map, and generate a fitted road surface based on the V disparity map; obtain a disparity point in any row of the current frame disparity map whose disparity value is greater than the corresponding disparity value of the fitted road surface, record it as a target disparity point, and map the target disparity point to the bird's-eye view grid map based on the binocular camera parameters; obtain the bird's-eye view grid occupied by the vehicle's driving area in the bird's-eye view grid map, and judge whether there is an obstacle in the vehicle's driving area based on whether the occupied bird's-eye view grid contains the target disparity point, wherein when the occupied bird's-eye view grid contains the target disparity point, it is determined that there is an obstacle in the vehicle's driving area; otherwise, it is determined that there is no obstacle in the vehicle's driving area.

[0087] In any of the above technical solutions, further, the edge determination unit 501 is also configured to: when it is determined that the vehicle is in a turning state, calculate the inner wheel difference of the vehicle based on the vehicle turning information; superimpose the inner wheel difference to the vehicle driving area, and based on the superimposed vehicle driving area, determine whether there is an obstacle.

[0088] In any of the above embodiments, further, the edge determination unit 501 is also configured to: extract historical data, obtain a historical frame reference image and a historical frame disparity map and a historical vehicle position corresponding to the historical frame reference image; determine the position offset between the vehicle position at the current moment and the historical vehicle position; based on the position offset, perform image processing on the historical frame reference image, and based on the processed historical frame reference image, determine whether there is an obstacle in the superimposed vehicle driving area, wherein the image processing includes at least any one of translation, rotation, and scaling.

[0089] In any of the above embodiments, further, the system 500 further includes:

[0090] A data extraction unit is configured to extract historical data frame by frame based on a preset step size, wherein the historical data includes a historical frame reference image, a corresponding historical frame disparity map, and a historical vehicle position when the historical frame reference image is obtained;

[0091] The coordinate calculation unit is configured to calculate the historical edge coordinates of the obstacle edge in the historical frame reference image by means of coordinate inverse operation based on the historical vehicle position;

[0092] The height calculation unit 502 is further configured to: based on the historical edge coordinates, calculate the historical obstacle edge disparity values ​​in the historical frame disparity map, and calculate the historical obstacle edge height based on the disparity algorithm and the historical obstacle edge disparity values;

[0093] The judgment unit 503 is further configured to update the obstacle edge height in a weighted manner based on the historical obstacle edge heights, and judge whether the maximum value of the updated obstacle edge heights is less than or equal to a preset height threshold.

[0094] In any of the above embodiments, the system 500 further includes an optimization unit configured to:

[0095] Based on the set edge margin, the edge optimization range is determined with the coordinates of any obstacle edge as the center;

[0096] Based on the disparity value within the edge optimization range, optimize the disparity value of the obstacle edge,

[0097] The optimized obstacle edge disparity value is the mean or median of the disparity values ​​within the edge optimization range.

[0098] Example 3:

[0099] This embodiment provides an electronic device, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to implement the low obstacle detection method based on binocular parallax of any one of the above-mentioned embodiments 1 when executing the executable instructions.

[0100] It should be noted that the number of processors can be one or more. Furthermore, the electronic device in the embodiments of the present application may also include an input device and an output device. The processor, memory, input device, and output device may be connected via a bus or other means, which are not specifically limited herein.

[0101] So far, the various embodiments of the present application have been described in detail. To avoid obscuring the concept of the present application, some details well known in the art have not been described. Based on the above description, those skilled in the art can fully understand how to implement the technical solutions disclosed herein.

[0102] Although some specific embodiments of the present application have been described in detail through examples, those skilled in the art should understand that the above examples are only for illustration and are not intended to limit the scope of the present application.

[0103] The steps in this application can be adjusted in order, combined, and deleted according to actual needs.

[0104] Although the present application is disclosed in detail with reference to the accompanying drawings, it should be understood that these descriptions are merely exemplary and are not intended to limit the application of the present application. The scope of protection of the present application is defined by the appended claims and may include various modifications, alterations and equivalents made to the invention without departing from the scope and spirit of the present application.

Claims

1. A low obstacle detection method based on binocular parallax, characterized in that: The method comprises: Based on the current frame reference image, determine whether there is an obstacle in the vehicle driving area, and if so, crop the current frame reference image according to the vehicle driving area, and perform edge extraction on the cropped current frame reference image to obtain the obstacle edge coordinates; According to the obstacle edge coordinates, counting the obstacle edge disparity values ​​in the current frame disparity map, and calculating the obstacle edge height based on the disparity algorithm and the obstacle edge disparity values; wherein the current frame disparity map corresponds to the current frame reference image; It is determined whether the maximum value of the obstacle edge heights is less than or equal to a preset height threshold. If so, it is determined that the obstacle in the vehicle driving area is a low obstacle. If not, it is determined that the obstacle in the vehicle driving area is a dangerous obstacle.

2. The low obstacle detection method based on binocular parallax according to claim 1, characterized in that: The determining whether there is an obstacle in the vehicle driving area specifically includes: Divide the space where the vehicle is located based on preset dimensions and generate a bird's-eye view grid map; Performing projection based on the current frame disparity map to generate a V disparity map, and generating a fitted road surface based on the V disparity map; Obtain a disparity point in any row of the disparity map of the current frame whose disparity value is greater than the disparity value corresponding to the fitted road surface, record it as a target disparity point, and map the target disparity point to the bird's-eye view grid map based on binocular camera parameters; Obtaining the bird's-eye view grid occupied by the vehicle driving area in the bird's-eye view grid map, and judging whether the obstacle exists in the vehicle driving area based on whether the occupied bird's-eye view grid contains the target parallax point, When the occupied bird's-eye view grid contains the target disparity point, it is determined that the obstacle exists in the vehicle driving area; otherwise, it is determined that the obstacle does not exist in the vehicle driving area.

3. The low obstacle detection method based on binocular parallax as claimed in claim 1, characterized in that: The method further comprises: When it is determined that the vehicle is in a turning state, the inner wheel difference of the vehicle is calculated according to the vehicle turning information; The inner wheel difference is superimposed on the vehicle driving area, and based on the superimposed vehicle driving area, it is determined whether the obstacle exists.

4. The low obstacle detection method based on binocular parallax as claimed in claim 3, characterized in that: The method further comprises: Extracting historical data to obtain a historical frame reference image and a historical frame disparity map corresponding to the historical frame reference image, and a historical vehicle position; Determine a position offset between a current vehicle position and the historical vehicle position; Based on the position offset, the historical frame reference image is processed, and based on the processed historical frame reference image, it is determined whether the obstacle exists in the superimposed vehicle driving area, The image processing includes at least one of translation, rotation and scaling.

5. The low obstacle detection method based on binocular parallax as claimed in claim 1, characterized in that: The method further comprises: Based on a preset step size, extracting historical data by frame, wherein the historical data includes a historical frame reference image, a corresponding historical frame disparity map, and a historical vehicle position when the historical frame reference image is obtained; Based on the historical vehicle position, calculating the historical edge coordinates of the obstacle edge coordinates in the historical frame reference image by means of coordinate inversion operation; Based on the historical edge coordinates, counting historical obstacle edge disparity values ​​in the historical frame disparity map, and calculating historical obstacle edge heights based on a disparity algorithm and the historical obstacle edge disparity values; Based on the historical obstacle edge heights, the obstacle edge heights are updated in a weighted manner, and it is determined whether a maximum value of the updated obstacle edge heights is less than or equal to the preset height threshold.

6. The low obstacle detection method based on binocular parallax according to any one of claims 1 to 5, characterized in that: The method further comprises: Based on the set edge margin, an edge optimization range is determined with any obstacle edge coordinate as the center; Based on the disparity value within the edge optimization range, optimizing the disparity value of the obstacle edge, The optimized obstacle edge disparity value is the mean or median of the disparity values ​​within the edge optimization range.

7. Low obstacle detection system based on binocular parallax, characterized in that: The system comprises: an edge determination unit, the edge determination unit being configured to determine whether there is an obstacle in the vehicle driving area based on the current frame reference image, and if so, to crop the current frame reference image according to the vehicle driving area, and to perform edge extraction on the cropped current frame reference image to obtain the obstacle edge coordinates; A height calculation unit, wherein the height calculation unit is configured to count the obstacle edge disparity values ​​in the current frame disparity map according to the obstacle edge coordinates, and calculate the obstacle edge height based on a disparity algorithm and the obstacle edge disparity values; wherein the current frame disparity map corresponds to the current frame reference image; A judgment unit, wherein the judgment unit is configured to judge whether the maximum value among the obstacle edge heights is less than or equal to a preset height threshold; if so, the obstacle in the vehicle driving area is judged to be a low obstacle; if not, the obstacle in the vehicle driving area is judged to be a dangerous obstacle.

8. The low obstacle detection system based on binocular parallax as claimed in claim 7, characterized in that: The edge determination unit is further configured to: Divide the space where the vehicle is located based on preset dimensions and generate a bird's-eye view grid map; Performing projection based on the current frame disparity map to generate a V disparity map, and generating a fitted road surface based on the V disparity map; Obtain a disparity point in any row of the disparity map of the current frame whose disparity value is greater than the disparity value corresponding to the fitted road surface, record it as a target disparity point, and map the target disparity point to the bird's-eye view grid map based on binocular camera parameters; Obtaining the bird's-eye view grid occupied by the vehicle driving area in the bird's-eye view grid map, and judging whether the obstacle exists in the vehicle driving area based on whether the occupied bird's-eye view grid contains the target parallax point, When the occupied bird's-eye view grid contains the target disparity point, it is determined that the obstacle exists in the vehicle driving area; otherwise, it is determined that the obstacle does not exist in the vehicle driving area.

9. The low obstacle detection system based on binocular parallax as claimed in claim 7, characterized in that: The edge determination unit is further configured to: When it is determined that the vehicle is in a turning state, the inner wheel difference of the vehicle is calculated according to the vehicle turning information; The inner wheel difference is superimposed on the vehicle driving area, and based on the superimposed vehicle driving area, it is determined whether the obstacle exists.

10. An electronic device, characterized in that The electronic device comprises: processor; a memory for storing instructions executable by the processor; The processor is configured to implement the low obstacle detection method based on binocular parallax described in any one of claims 1 to 6 when executing the executable instructions.

Images