Method, device, system and storage medium for determining ground parallax false detection

Abstract

The present application provides a method, device, system and storage medium for judging ground parallax misdetection, wherein the method includes: obtaining a disparity map, a V disparity map, and an obstacle target frame corresponding to the marked obstacle; selecting a road disparity point corresponding to the road surface in the V disparity map according to the disparity map to perform road surface fitting; selecting an obstacle disparity point corresponding to the obstacle in the V disparity map according to the obstacle target frame and the disparity map to perform obstacle fitting to obtain the outer contour of the obstacle; calculating the obstacle angle between the outer contour of the obstacle and the fitted road surface; when the angle is greater than a preset value, judging that the marked obstacle is a real obstacle; when the angle is less than or equal to a preset value, judging that the marked obstacle is a misdetection. By judging whether the target is a misdetection by judging the angle between the outer contour fitting straight line of the target and the road surface fitting straight line, the problem of eliminating the misdetection of obstacles caused by ground marking lines and shadow areas after a certain degradation of the binocular camera is solved.

Description

Technical Field

[0001] The present application relates to the field of binocular vision, and in particular to a method for determining ground parallax misdetection, a device for determining ground parallax misdetection, a control system, and a storage medium. Background Art

[0002] The stereo camera provides color image RGB data and disparity map data. When outputting disparity map data, the stereo camera aligns each row of pixels in the left and right images through image dedistortion and binocular correction of the left and right lenses, and performs pixel matching by row in the left and right images based on the principle of epipolar constraints and matching algorithms to calculate the disparity data corresponding to the color image of the stereo camera.

[0003] Therefore, when using a binocular camera for obstacle detection, the parallax above the ground is usually clustered, and the clustering result is output as the obstacle detection result, and the real properties of such obstacles (such as horizontal / vertical distance, size, yaw angle, etc.) are output.

[0004] However, with the use of binocular cameras, binocular cameras will degrade to varying degrees, and the position relationship between the two lenses of the binocular camera will change, which will lead to incomplete alignment of pixel rows in the left and right images when matching, especially at ground signs or shadows. At this time, when the matching algorithm is used to calculate the disparity, the pixel points at the edge of the diagonal line in the left image of the binocular camera will be matched to the upper / lower side of the corresponding pixel points at the edge of the diagonal line in the right image of the binocular camera according to the principle of epipolar constraints, resulting in the disparity at the diagonal line being larger / smaller than the true value. These disparity points with incorrect disparity value calculations are reflected as sudden changes in color in the parallax pseudo-color image, which is significantly different from the gradual color change on the ground.

[0005] Therefore, in the process of parallax clustering, when the parallax at a certain location changes suddenly compared with the theoretical parallax, the parallax cluster tends to be clustered into an obstacle on the road, resulting in false obstacle detection. Such false detections can trigger target collision alarms, false braking, or affect the vehicle's path planning. Summary of the invention

[0006] In view of this, the present application proposes a method for determining ground parallax misdetection, and the technical solution is as follows:

[0007] On the one hand, the present application proposes a method for determining ground parallax misdetection, comprising:

[0008] Obtain the disparity map, V disparity map, and obstacle target frame corresponding to the marked obstacle;

[0009] According to the disparity map, a road surface disparity point corresponding to the road surface is selected in the V disparity map to perform road surface fitting;

[0010] According to the obstacle target frame and disparity map, the obstacle disparity point corresponding to the obstacle is selected in the V disparity map to perform obstacle fitting and obtain the outer contour of the obstacle;

[0011] Calculate the obstacle angle between the outer contour of the obstacle and the fitted road surface;

[0012] When the angle is greater than a preset value, the marked obstacle is determined to be a real obstacle;

[0013] When the angle is less than or equal to the preset value, the marked obstacle is determined to be a false detection.

[0014] As an optional implementation of the present application, optionally, selecting a road disparity point corresponding to the road surface in the V disparity map according to the disparity map to perform road surface fitting includes:

[0015] According to the segmentation map corresponding to the disparity map and the vanishing point calibrated by the camera, determine whether there are left and right lane lines in the segmentation map;

[0016] When there are left and right lane lines in the segmentation map, the credible disparity points corresponding to the lane line categories in the segmentation map are projected into the V disparity map for road surface fitting;

[0017] When there are no left or right lane lines in the segmentation map, the credible disparity points corresponding to the road surface categories in the segmentation map are projected into the V disparity map for road surface fitting.

[0018] As an optional implementation scheme of the present application, optionally, when there are left and right lane lines in the segmentation map, the credible disparity points corresponding to the lane line categories in the segmentation map are projected into the V disparity map for road surface fitting, including:

[0019] When there are left and right lane lines in the segmentation map, traverse each pixel point p in the segmentation map and the label category of the pixel point p to determine whether the label category of any pixel point p is a lane line category;

[0020] When the label category of the pixel point is the lane line category, determine whether the disparity value dsp of the disparity point in the disparity map corresponding to the pixel point p is greater than 0;

[0021] When the disparity value dsp is greater than 0, the disparity point is projected into the V disparity map with the coordinates (dsp, y) as a credible disparity point of the road surface for road surface fitting.

[0022] As an optional implementation scheme of the present application, optionally, when there are no left and right lane lines in the segmentation map, the credible disparity points corresponding to the lane line categories in the segmentation map are projected into the V disparity map for road surface fitting, including:

[0023] When there are no left or right lane lines in the segmentation map, the label category of each pixel point p in the segmentation map is traversed to determine whether the label category of any pixel point p is a road surface category;

[0024] If so, determine whether the disparity value dsp of the disparity point corresponding to the pixel point p in the disparity map is greater than 0;

[0025] If yes, project the disparity point corresponding to the pixel point into the V disparity map, and based on the point number threshold thresh, select the disparity point with the largest Y value in each column in the V disparity map, which has been projected more than the corresponding point number threshold thresh, as the credible disparity point of the road surface;

[0026] Road surface fitting is performed based on the screened credible disparity points of the road surface.

[0027] As an optional implementation of the present application, optionally, the method further includes: determining a point number threshold thresh for each column in the V disparity map, specifically including:

[0028] According to the value of the disparity value dsp, traverse the V disparity map column by column from left to right, and record the maximum number of projections at each disparity value dsp;

[0029] The value obtained by multiplying the maximum number of projections at the disparity value dsp in each column by a preset multiple is used as the point count threshold thresh of this column.

[0030] As an optional implementation scheme of the present application, optionally, according to the obstacle target frame and the disparity map, an obstacle disparity point corresponding to the obstacle is selected in the V disparity map to perform obstacle fitting to obtain the outer contour of the obstacle, including:

[0031] According to the disparity map, the obstacle target frame is detected;

[0032] Traverse each disparity point in the obstacle target frame. If the disparity point has a disparity value greater than 0 in the disparity map, project the disparity point into the V disparity map.

[0033] According to the value of the ordinate in the disparity map, count the points (max_dsp,y) at the maximum disparity value of each row in the corresponding range of the target frame in the V disparity map row by row, as the outer contour points of the obstacles in the target frame in the V disparity map;

[0034] Fit the outer contour points with the straight line equation, record the slope of the outer contour equation as KObject and get the outer contour of the obstacle.

[0035] As an optional implementation scheme of the present application, optionally, when the disparity value dsp is greater than 0, the road surface credible disparity points are screened based on the set point threshold thresh to perform road surface fitting, including:

[0036] When the disparity value dsp is greater than 0, traverse the V disparity map from left to right according to the value of the disparity value dsp, and record the maximum number of projections at each disparity value dsp;

[0037] The value obtained by multiplying the maximum number of projections at each disparity value dsp by a preset multiple is used as the point count threshold thresh of this column;

[0038] Traverse each disparity point in the V disparity map, and select the point in each column whose projection times are greater than the point number threshold thresh and whose Y value is the largest as the credible disparity point of the road surface at the disparity value;

[0039] The road surface is fitted according to the selected credible disparity points of the road surface.

[0040] As an optional implementation scheme of the present application, optionally, it also includes:

[0041] When the marked obstacle is determined to be a real obstacle, this target is retained;

[0042] When the marked obstacle is determined to be a false detection, the target is filtered out.

[0043] Another aspect of the present application provides a device for implementing the above-mentioned method for determining ground parallax misdetection, comprising:

[0044] Information acquisition module: used to obtain the disparity map, V disparity map, and obstacle target frame corresponding to the marked obstacles;

[0045] Road surface fitting module: used to select road surface disparity points corresponding to the road surface in the V disparity map according to the disparity map to perform road surface fitting;

[0046] Obstacle outer contour fitting module: used to select the obstacle disparity point corresponding to the obstacle in the V disparity map according to the obstacle target frame and disparity map to perform obstacle fitting and obtain the obstacle outer contour;

[0047] Angle calculation module: used to calculate the obstacle angle between the outer contour of the obstacle and the fitted road surface;

[0048] Obstacle judgment module: used to judge the marked obstacle as a real obstacle when the angle is greater than a preset value;

[0049] The module for judging whether an obstacle is a false detection is used to judge that the marked obstacle is a false detection when the angle is less than or equal to a preset value.

[0050] On the other hand, the present application provides a control system for implementing the above-mentioned method for determining ground parallax misdetection, comprising:

[0051] processor;

[0052] a memory for storing processor-executable instructions;

[0053] Wherein, the processor is configured to implement the aforementioned method for determining ground parallax misdetection when executing executable instructions.

[0054] On the other hand, the present application provides a non-volatile computer-readable storage medium having computer program instructions stored thereon, wherein the computer program instructions, when executed by a processor, implement the aforementioned method for determining ground parallax misdetection.

[0055] Technical effects of this application:

[0056] The method of the present application combines the disparity map, V disparity map, and obstacle target frame corresponding to the marked obstacle, and determines whether the target is a false detection by judging the relationship between the angle between the target's outer contour fitting straight line and the road surface fitting straight line and the set threshold. When the angle is less than the preset value, that is, when the angle is too small or even there is no angle, it can be explained that the detected obstacle of the target is not a real obstacle. In the present method, it is determined to be a false detection of an obstacle and is eliminated, thereby solving the problem of false detection of obstacles caused by ground marking lines and shadow areas after a certain degree of degradation of the binocular camera, and can eliminate false detection of obstacles caused by ground marking lines or shadow areas, etc., to ensure accurate detection of obstacles by the binocular camera, avoid false triggering of target collision warnings and false braking, and ensure accurate planning of vehicle paths.

[0057] Other features and aspects of the present application will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the present application and, together with the description, serve to explain the principles of the present application.

[0059] Figure 1 A flowchart of a method for determining ground parallax misdetection according to an embodiment of the present application;

[0060] Figure 2 A color image of misdetection caused by parallax matching error at a guide line according to an embodiment of the present application;

[0061] Figure 3 A pseudo-color map of disparity that is misdetected due to disparity matching error at a guide line according to an embodiment of the present application;

[0062] Figure 4 A color image when there are left and right lane lines in the segmentation image of an embodiment of the present application;

[0063] Figure 5 A pseudo-color image of the disparity when there are left and right lane lines in the segmentation image of an embodiment of the present application;

[0064] Figure 6It is a segmentation map output by the model when there are left and right lane lines in the segmentation map of an embodiment of the present application;

[0065] Figure 7 A V disparity map obtained by projecting credible disparity points when there are left and right lane lines in the segmentation map of an embodiment of the present application;

[0066] Figure 8 A color image when there are no left and right lane lines in the segmentation image of an embodiment of the present application;

[0067] Fig. 9 It is a segmentation map output by the model when there are no left and right lane lines in the segmentation map of an embodiment of the present application;

[0068] Fig.10 A V disparity map obtained by projecting the disparity map when there are no left and right lane lines in the segmentation map of an embodiment of the present application;

[0069] Fig.11 A disparity map and a corresponding V disparity map when there are no left and right lane lines in the segmentation map of an embodiment of the present application;

[0070] Fig.12 A color image of a bridge pier in front of a vehicle according to an embodiment of the present application;

[0071] Fig.13 A bit color map of the parallax of a bridge pier in front of the vehicle according to an embodiment of the present application;

[0072] Fig.14 This is a schematic diagram of detecting target No. 1 based on a disparity map when there is a bridge pier in front of a vehicle according to an embodiment of the present application;

[0073] Fig.15 A V-disparity map of a vehicle with a bridge pier in front of it according to an embodiment of the present application;

[0074] Fig.16 A color diagram of a method for determining ground parallax misdetection according to another embodiment of the present application;

[0075] Fig.17 A V disparity map of a method for determining ground disparity misdetection according to another embodiment of the present application;

[0076] Fig.18 A disparity map of a method for determining ground disparity misdetection according to another embodiment of the present application;

[0077] Fig.19 A color image of two targets detected in a method for determining ground parallax misdetection according to another embodiment of the present application;

[0078] Fig. 20 A specific embodiment of a method for determining ground parallax misdetection according to another embodiment of the present application is a V-disparity map of the outer contour of target 1 and the road surface;

[0079] Fig.21 A specific embodiment of a method for determining ground parallax misdetection according to another embodiment of the present application is a V-disparity map of the outer contour of target No. 4 and the road surface. DETAILED DESCRIPTION

[0080] Various exemplary embodiments, features and aspects of the present application will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0081] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0082] In addition, in order to better illustrate the present application, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present application can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present application.

[0083] It should be noted that the binocular camera provides color image RGB data and disparity map data. When outputting the disparity map data, the binocular camera aligns each row of pixels in the left and right images through image dedistortion and binocular correction of the left and right lenses, and matches pixels row by row in the left and right images based on the principle of epipolar constraints and matching algorithms to calculate the disparity data corresponding to the binocular camera color image. Therefore, when using a binocular camera for obstacle detection, the parallax above the ground is usually clustered, and the clustering results are output as obstacle detection results, and the true attributes of such obstacles (such as horizontal / vertical distance, size, yaw angle, etc.) are output. However, with the use of the binocular camera, the binocular camera will degrade to varying degrees, and the posture relationship between the two lenses of the binocular camera will change, which will cause the pixel rows in the left and right images to be incompletely aligned when matching, especially at ground signs or shadows. At this time, when the matching algorithm is used for disparity calculation, the pixel points at the edge of the diagonal line in the left image of the binocular camera will be matched to the upper / lower side of the corresponding pixel points at the edge of the diagonal line in the right image of the binocular camera according to the principle of epipolar constraints, resulting in the disparity at the diagonal line being larger / smaller than the true value. These disparity points with incorrect disparity value calculations are reflected as sudden changes in color in the parallax pseudo-color image, which is significantly different from the gradual color change on the ground. Therefore, in the process of parallax clustering, when the parallax at a certain location changes suddenly compared with the theoretical parallax, the parallax cluster tends to be clustered into an obstacle on the road, resulting in false obstacle detection. Such false detections can trigger target collision alarms, false braking, or affect the vehicle's path planning, such as Figure 2 This is a color image of a misdetection caused by a parallax matching error at a guide line in another embodiment of the method for determining ground parallax misdetection of the present application. Figure 3 for Figure 2 The pseudo-color image of the parallax of the misdetected parallax caused by the parallax matching error at the guide line in the application embodiment of the ground parallax misdetection judgment method. The application aims to determine whether the target is a misdetection by judging the angle between the outer contour fitting line of the target and the road surface fitting line, and solve the problem of eliminating the misdetection of obstacles caused by ground marking lines and shadow areas after a certain degradation of the binocular camera. The application is explained through specific embodiments below.

[0084] Example 1

[0085] like Figure 1-21 As shown, the present application provides, on the one hand, a method for determining ground parallax misdetection, comprising the following steps:

[0086] S1000: Obtain a disparity map, a V disparity map, and an obstacle target frame corresponding to the marked obstacle.

[0087] It should be noted that if the resolution of the RGB original image, segmentation map and disparity map is the same, the pixels correspond one to one. The segmentation model has five labels. For areas without labeled objects, the model will judge them as "empty" or "road".

[0088] S2000 , selecting a road surface disparity point corresponding to the road surface in the V disparity map according to the disparity map to perform road surface fitting.

[0089] It should be noted that there are two problems when directly using the disparity points with the category of "road surface" in the segmentation map for road surface fitting: 1. The disparity points with the label category of "road surface" may contain some disparity points calculated by unlabeled objects; 2. Due to the accuracy of the binocular camera and the degree of degradation during use, when the disparity map is projected into the V disparity map, the obtained projection points usually have a certain thickness. Therefore, when the projection points in the V disparity map are used to fit the ground, the fitting equation of the ground will deviate from the true value, so it is necessary to screen out reliable ground disparity points in the V disparity map for ground fitting.

[0090] The disparity map is projected to obtain the corresponding V disparity map. For the V disparity map, the horizontal axis is the disparity value (the value of the horizontal axis gradually increases from left to right), and the vertical axis is the row coordinate in the disparity map / RGB image (ie, the y-axis), and the value of the vertical axis gradually increases from top to bottom.

[0091] This step S2000 specifically includes three small steps S2100, S2200, and S2300.

[0092] S2100: Determine whether there are left and right lane lines in the segmentation map according to the segmentation map corresponding to the disparity map and the vanishing points calibrated by the camera.

[0093] In this step, the AI ​​model inference module is used to obtain a segmentation map of the same size as the RGB original image. In the segmentation map, pixel-level labels are used to classify each pixel in the RGB image into five categories: car, person, ground, lane line, and space. Each pixel in the image is traversed, and based on the category of the pixel in the segmentation map and the horizontal coordinates of the vanishing point, it is determined whether there are lane lines on the left and right sides of the vanishing point in the image.

[0094] It should be noted that the vanishing point refers to the vanishing point specified during the binocular camera calibration process, that is, the intersection of the extended lane lines, which can be set manually.

[0095] S2200: When there are left and right lane lines in the segmentation map, project the credible disparity points corresponding to the lane line categories in the segmentation map into the V disparity map for road surface fitting.

[0096] In this step S2200, the credible disparity point set P (abscissa dsp, ordinate y) of the road surface in the V disparity map is fitted with a straight line equation to obtain a road surface equation of Y = K * Disparity + B, where K is the slope of the road surface, Disparity is the disparity, B is the ordinate of the road surface at infinity (dsp=0), and Y is the ordinate of the road surface at the disparity Disparity. Specifically, the steps S2210, S2220, and S2230 are included:

[0097] S2210: When there are left and right lane lines in the segmentation map, traverse each pixel point p in the segmentation map and the label category of the pixel point p to determine whether the label category of any pixel point p is a lane line category.

[0098] S2220: When the label category of the pixel point is the lane line category, determine whether the disparity value dsp of the disparity point in the disparity map corresponding to the pixel point p is greater than 0.

[0099] It should be noted that when the disparity value dsp is not greater than 0, the point is discarded.

[0100] S2230, when the disparity value dsp is greater than 0, project the disparity point into the V disparity map with coordinates (dsp, y) as a credible disparity point of the road surface, and perform road surface fitting.

[0101] It should be noted that in the V-disparity map projection process, there is a situation where the disparity values ​​corresponding to the pixels in the same row and columns of the image are the same. At this time, the disparity points corresponding to these pixels will all be projected to the same point with the same coordinates in the V-disparity map.

[0102] S2300: When there are no left or right lane lines in the segmentation map, project the credible disparity points corresponding to the road surface categories in the segmentation map into the V disparity map for road surface fitting.

[0103] like Figure 8-11 , This step S2300 specifically includes four small steps S2310, S2320, S2330, and S2340:

[0104] S2310: When there are no left or right lane lines in the segmentation map, traverse the label category of each pixel point p in the segmentation map to determine whether the label category of any pixel point p is a road surface category.

[0105] S2320, when the pixel point is of the road surface category, determine whether the disparity value dsp of the disparity point corresponding to the pixel point p in the disparity map is greater than 0; when the pixel point is not of the road surface category, discard the pixel point.

[0106] S2330. When the disparity value dsp of the pixel point of the road surface category is greater than 0, the disparity point corresponding to the pixel point is projected into the V disparity map, and based on the point count threshold thresh, the disparity point with the largest Y value and the number of projections greater than the corresponding point count threshold thresh in each column of the V disparity map is selected as the credible disparity point of the road surface; when the disparity value dsp of the pixel point of the road surface category is greater than 0, the pixel point is discarded.

[0107] S2340: Perform road surface fitting based on the screened road surface credible parallax points.

[0108] Furthermore, the method in this embodiment further includes: determining a point number threshold thresh for each column in the V disparity map. The process specifically includes:

[0109] According to the value of the disparity value dsp, traverse the V disparity map column by column from left to right, and record the maximum number of projections at each disparity value dsp;

[0110] The value obtained by multiplying the maximum number of projections at the disparity value dsp in each column by a preset multiple is used as the point count threshold thresh of this column.

[0111] This step specifically includes four small steps: S2331, S2332, S2333, and S2334.

[0112] S2331. When the disparity value dsp is greater than 0, traverse the V disparity map from left to right according to the value of the disparity value dsp, and record the maximum number of projections at each disparity value dsp.

[0113] S2332. The value obtained by multiplying the maximum number of projections at each disparity value dsp by a preset multiple is used as the point count threshold thresh of this column.

[0114] It should be noted that the preset multiple in this step can be adjusted between 0.4-0.7.

[0115] S2333, traverse each disparity point in the V disparity map, and select the point in each column whose projection times are greater than the point number threshold thresh and whose Y value is the largest as the road surface credible disparity point at the disparity value.

[0116] In this step, since there may be more noise points and valid disparity points projected at the top of the V disparity map, but the bottom is the desired disparity point, the point with the largest Y value and the number of projections greater than the point threshold thresh is selected as the credible disparity point of the road surface at this disparity value.

[0117] S2334, fitting the road surface according to the selected road surface credible parallax points.

[0118] S3000: According to the obstacle target frame and the disparity map, select the obstacle disparity point corresponding to the obstacle in the V disparity map to perform obstacle fitting to obtain the outer contour of the obstacle.

[0119] This step includes four small steps: S3100, S3200, S3300, and S3400:

[0120] S3100: Detect an obstacle target frame according to the disparity map.

[0121] S3200 , traverse each disparity point in the obstacle target frame, and if the point has a disparity value greater than 0 in the disparity map, project the point into the V disparity map.

[0122] S3300, according to the value of the ordinate in the disparity map, count the points (max_dsp, y) at the maximum disparity value of each row in the corresponding range of the target frame in the V disparity map row by row, as the outer contour points of the obstacles in the target frame in the V disparity map.

[0123] S3400, fit the outer contour points with a straight line equation, record the slope of the outer contour equation as KObject, and obtain the outer contour of the obstacle.

[0124] S4000: Calculate an obstacle angle between an outer contour of the obstacle and the fitted road surface.

[0125] S5000: When the angle is greater than a preset value, determine that the marked obstacle is a real obstacle.

[0126] It should be noted that, in this embodiment, the preset value is set to 20 degrees, and can be adjusted between 10-45 degrees according to actual conditions.

[0127] It should also be noted that when the marked obstacle is determined to be a real obstacle, this target is retained.

[0128] S6000: When the angle is less than or equal to a preset value, determine that the marked obstacle is a false detection.

[0129] It should be noted that when the marked obstacle is determined to be a false detection, the target is filtered out.

[0130] In summary, the present application determines whether the target is a false detection by judging the angle between the target's outer contour fitting line and the road surface fitting line, thereby solving the problem of eliminating false detection of obstacles caused by ground markings and shadow areas after a certain degree of degradation of the binocular camera.

[0131] Example 2

[0132] like Figure 4-7 The following is a specific embodiment of a method for projecting credible disparity points corresponding to lane line categories in the segmentation map into the V disparity map for road surface fitting when there are left and right lane lines in the segmentation map.

[0133] like Figure 4 This is a color image when there are left and right lane lines in the segmentation image of the method for determining ground parallax misdetection in another embodiment of the present application. Figure 5 for Figure 4 A pseudo-color image of the parallax when there are left and right lane lines in the segmentation image of the method for determining ground parallax misdetection in the embodiment of the application, Figure 6 for Figure 4 The segmentation map output by the model when there are left and right lane lines in the segmentation map of the method for determining ground parallax misdetection in the embodiment of the application. Since there are lane lines in the map, road surface fitting is performed according to S2210, S2220, and S2230 in step S2200. Figure 6 In the example, the ground label category is 1 and the lane label category is 2.

[0134] S2210: When there are left and right lane lines in the segmentation map, traverse each pixel point p in the segmentation map and the label of the pixel point p to determine whether the pixel point is a lane line category.

[0135] S2220: When the pixel point is a lane line category, determine whether the disparity value dsp of the disparity point in the disparity map corresponding to the pixel point p is greater than 0.

[0136] like Figure 7 for Figure 4In the segmentation map of the method for determining ground parallax misdetection in the embodiment of the application, when there are left and right lane lines in the segmentation map, the V parallax map obtained by projecting the credible parallax point is as follows: according to step S2230, when the parallax value dsp is greater than 0, the parallax point is projected into the V parallax map with the coordinates (dsp, y) as the credible parallax point of the road surface, and the road surface fitting is performed. The white points in the map are the credible parallax points selected, and the credible parallax points selected are used for road surface fitting. When the parallax value dsp of the parallax point in the parallax map corresponding to the pixel point p is less than or equal to 0, the pixel point is discarded.

[0137] Example 3

[0138] like Figure 8-11 The following is a specific embodiment of a method for projecting credible disparity points corresponding to lane line categories in the segmentation map into the V disparity map for road surface fitting when there are no left and right lane lines in the segmentation map.

[0139] Figure 8 This is a color image when there are no left and right lane lines in the segmentation image of the method for judging ground parallax misdetection in another embodiment of the present application. In the image, there is a cardboard box as an unlabeled general obstacle. Fig. 9 for Figure 8 The segmentation map output by the model when there are no left and right lane lines in the segmentation map of the method for judging ground parallax false detection in the application embodiment. In this segmentation map, the area of ​​the cardboard box on the ground is also classified as the ground. If the ground is directly fitted, the parallax of the box area will affect the fitting result of the ground. The disparity map corresponding to the ground area is projected into the V disparity map. The road surface has a certain thickness and contains the parallax of the box area, which will affect the fitting result of the ground and needs to be filtered out. Since there are no left and right lane lines in the map, the road surface is fitted according to step S2300 and the sub-steps S2310, S2320, and S2330 included therein. S2310, when there are no left and right lane lines in the segmentation map, traverse each pixel point p in the segmentation map and the label of the pixel point p to determine whether the pixel point belongs to the road surface category.

[0140] like Fig.10 for Figure 8 In the segmentation map of the method for determining ground parallax misdetection in the embodiment of the application, when there are no left and right lane lines in the segmentation map, the above parallax map is projected to obtain a V parallax map, wherein the green point is the point with the maximum number of votes at the parallax value dsp, and the red point is the credible parallax point selected. Therefore, the credible parallax point is used for road surface fitting, which reduces the influence of the projection point of the parallax point on the road surface carton in the V parallax map and the road surface thickness on the road surface fitting. Fig.11 Another embodiment of the present application is a method for determining ground parallax misdetection, wherein the disparity map when there are no left and right lane lines in the segmentation map and the corresponding V disparity map are: Figure 8 In the embodiment and Fig.11In the embodiments, according to step S2320, when the pixel point is a road type, it is determined whether the disparity value dsp of the disparity point in the disparity map corresponding to the pixel point p is greater than 0, and S2330, when the disparity value dsp is greater than 0, the disparity point is projected into the V disparity map, and the road credible disparity points are screened based on the set point number threshold thresh to perform road fitting. When the disparity value dsp is less than or equal to 0, the pixel point is discarded.

[0141] Example 4

[0142] like Figure 12-15 The following is a specific embodiment of the process of selecting obstacle disparity points corresponding to obstacles in the V disparity map according to the obstacle target frame and the disparity map to perform obstacle fitting and obtain the outer contour of the obstacle.

[0143] like Fig.12 In another embodiment of the present application, a method for determining ground parallax misdetection is provided. In front of the vehicle is a color image of a bridge pier. According to step S3100, an obstacle target frame is detected based on the parallax image. Fig.13 for Fig.12 In the method for determining ground parallax misdetection in the embodiment of the application, there is a parallax color map of a bridge pier in front of the vehicle, such as Fig.14 for Fig.12 The method for determining ground parallax misdetection in the embodiment of the application is a schematic diagram of detecting target No. 1 based on the parallax map in which there is a bridge pier in front of the vehicle. Fig.15 for Fig.12 In the method for determining ground parallax misdetection in the embodiment of the application, there is a V disparity map of a bridge pier in front of the vehicle. According to step S3200, each disparity point in the obstacle target frame is traversed. If the point has a disparity value greater than 0 in the disparity map, the point is projected into the V disparity map. The disparity points in target frame No. 1 are projected into the V disparity map. The minimum disparity value of each row is represented by a blue dot, and the maximum disparity value of each row is represented by a red dot. According to step S3300, according to the value of the ordinate in the disparity map, the points (max_dsp,y) at the maximum disparity value of each row in the corresponding range of the target frame are counted row by row in the V disparity map as the outer contour points of the obstacle in the target frame in the V disparity map. According to step S3400, the outer contour points are fitted with a straight line equation, the slope of the outer contour equation is recorded as KObject, and the outer contour of the obstacle is obtained. The outer contour of the target fitted with a straight line equation is represented by a green line.

[0144] Example 5

[0145] like Figure 16-21 , the method of the present application is further illustrated below with a complete embodiment.

[0146] like Fig.16This is a color image of a method for determining ground parallax misdetection according to another embodiment of the present application. According to S1000, a disparity map, a V disparity map, and an obstacle target frame corresponding to a marked obstacle are obtained. According to S2000, a road surface disparity point corresponding to the road surface is selected in the V disparity map according to the disparity map to perform road surface fitting. Since there are clear left and right lane lines in the image, the corresponding disparity points on the lane lines are projected onto the V disparity map. Fig.17 middle, Fig.17 for Fig.16 The V disparity map of the method for judging ground disparity misdetection in the embodiment of the application is calculated through road fitting, and the road equation Y = K * Disparity + B is obtained, where K is 0.28 and B is 117.29. Fig.18 for Fig.16 The disparity map of the method for determining ground disparity misdetection in the application embodiment is as follows: Fig.19 for Fig.16 The method for determining ground parallax misdetection in the embodiment of the application detects the color images of two targets, Fig.19 In step S3000, according to the obstacle target frame and the disparity map, the obstacle disparity point corresponding to the obstacle is selected in the V disparity map to perform obstacle fitting, and the outer contour of the obstacle is obtained. Fig. 20 for Fig.16 The V disparity map of the outer contour of target No. 1 and the road surface in the method for judging ground parallax misdetection in the application embodiment, Fig.21 for Fig.16 V-parallax map of the outer contour of target No. 4 and the road surface in the method for judging ground parallax false detection in the application embodiment. According to step S4000, the obstacle angle between the outer contour of the obstacle and the fitted road surface is calculated, and the outer contour of target No. 1 is analyzed. The angle between the outer contour of the target and the road surface is 0.49 degrees. According to step S6000, when the angle is less than or equal to the preset value, the marked obstacle is determined to be a false detection, and the target is determined to be a false detection on the road surface. The outer contour of target No. 4 is analyzed. The angle between the outer contour of the target and the road surface is 3.90 degrees. Similarly, according to step S6000, when the angle is less than or equal to the preset value, the marked obstacle is determined to be a false detection, and the target is determined to be a false detection on the road surface.

[0147] Example 6

[0148] Furthermore, in another aspect, the present application provides a control system, comprising:

[0149] processor;

[0150] a memory for storing processor-executable instructions;

[0151] Wherein, the processor is configured to implement any of the above-mentioned methods for determining ground parallax misdetection when executing executable instructions.

[0152] The control system of the embodiment of the present application includes a processor and a memory for storing processor executable instructions, wherein the processor is configured to implement any of the above-mentioned ground parallax misdetection judgment methods when executing the executable instructions.

[0153] Here, it should be noted that the number of processors can be one or more. At the same time, in the control system of the embodiment of the present application, an input device and an output device may also be included. Among them, the processor, memory, input device and output device may be connected through a bus or in other ways, which are not specifically limited here.

[0154] The memory, as a computer-readable storage medium, can be used to store software programs, computer executable programs and various modules, such as the program or module corresponding to the method for determining ground parallax misdetection in the embodiment of the present application. The processor executes various functional applications and data processing of the control system by running the software programs or modules stored in the memory.

[0155] The input device can be used to receive input numbers or signals. The signal can be a key signal related to user settings and function control of the device / terminal / server. The output device can include a display device such as a display screen.

[0156] Example 7

[0157] On the other hand, the present application provides a non-volatile computer-readable storage medium having computer program instructions stored thereon, and when the computer program instructions are executed by a processor, any of the above-mentioned methods for determining ground parallax misdetection is implemented.

[0158] The present application may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present application.

[0159] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable and removable hard disk, and a plurality of other storage devices.

[0160] Programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, mechanical encoding device, such as punch card or raised structure in groove with instructions stored thereon, and any suitable combination of the above. Computer readable storage medium as used herein is not to be interpreted as a transient signal itself, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0161] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0162] The computer program instructions for performing the operation of the present application may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed entirely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be personalized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present application.

[0163] Various aspects of the present application are described herein with reference to flowcharts and / or block diagrams of methods, devices (systems) and computer program products according to embodiments of the present application. It should be understood that each box in the flowchart and / or block diagram and the combination of boxes in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0164] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0165] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0166] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of a module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special-purpose hardware and computer instructions.

[0167] The embodiments of the present application have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for determining ground parallax misdetection, characterized in that: The following steps are involved: Obtain the disparity map, V disparity map and obstacle target frame corresponding to the marked obstacle; Selecting a road surface disparity point corresponding to a road surface in the V disparity map according to the disparity map to perform road surface fitting; According to the obstacle target frame and the disparity map, an obstacle disparity point corresponding to the obstacle is selected in the V disparity map to perform obstacle fitting to obtain an outer contour of the obstacle; Calculating an obstacle angle between the outer contour of the obstacle and the fitted road surface; When the angle is greater than a preset value, the marked obstacle is determined to be a real obstacle; When the angle is less than or equal to the preset value, determining that the marked obstacle is a false detection; The step of selecting an obstacle disparity point corresponding to an obstacle in the V disparity map according to the obstacle target frame and the disparity map to perform obstacle fitting to obtain an outer contour of the obstacle includes: Detecting the obstacle target frame according to the disparity map; Traversing each disparity point in the obstacle target frame, if the disparity point has a disparity value greater than 0 in the disparity map, projecting the disparity point into the V disparity map; According to the value of the ordinate in the disparity map, count the points (max_dsp, y) at the maximum disparity value of each row within the corresponding range of the target frame in the V disparity map row by row, as the outer contour points of the obstacles in the target frame in the V disparity map; Fit the outer contour points with a straight line equation, record the slope of the outer contour equation as KObject and obtain the outer contour of the obstacle.

2. The method for determining ground parallax misdetection according to claim 1, characterized in that: The selecting a road surface disparity point corresponding to the road surface in the V disparity map according to the disparity map to perform road surface fitting includes: According to the segmentation map corresponding to the disparity map and the vanishing point calibrated by the camera, it is determined whether there are left and right lane lines in the segmentation map; When there are left and right lane lines in the segmentation map, the credible disparity points corresponding to the lane line categories in the segmentation map are projected into the V disparity map for road surface fitting; When there are no left and right lane lines in the segmentation map, the credible disparity points corresponding to the road surface categories in the segmentation map are projected into the V disparity map for road surface fitting.

3. The method for determining ground parallax misdetection according to claim 2, characterized in that: When there are left and right lane lines in the segmentation map, projecting the credible disparity points corresponding to the lane line categories in the segmentation map into the V disparity map for road surface fitting includes: When there are left and right lane lines in the segmentation map, traverse each pixel point p in the segmentation map and the label category of the pixel point p to determine whether the label category of any pixel point p is a lane line category; When the label category of the pixel point is the lane line category, determine whether the disparity value dsp of the disparity point in the disparity map corresponding to the pixel point p is greater than 0; When the disparity value dsp is greater than 0, the disparity point is projected into the V disparity map with coordinates (dsp, y) as a credible disparity point of the road surface for road surface fitting.

4. The method for determining ground parallax misdetection according to claim 2, characterized in that: When there are no left and right lane lines in the segmentation map, projecting the credible disparity points corresponding to the lane line categories in the segmentation map into the V disparity map for road surface fitting includes: When there are no left or right lane lines in the segmentation map, traverse the label category of each pixel point p in the segmentation map to determine whether the label category of any pixel point p is a road surface category; If so, determine whether the disparity value dsp of the disparity point corresponding to the pixel point p in the disparity map is greater than 0; If yes, project the disparity point corresponding to the pixel point into the V disparity map, and based on the point number threshold thresh, select the disparity point with the largest Y value in each column in the V disparity map, which has been projected more than the corresponding point number threshold thresh, as the credible disparity point of the road surface; Road surface fitting is performed based on the screened credible disparity points of the road surface.

5. The method for determining ground parallax misdetection according to claim 4, characterized in that: The method further includes: determining the point number threshold thresh of each column in the V disparity map, specifically including: According to the value of the disparity value dsp, the V disparity map is traversed column by column from left to right, and the maximum number of projections at each disparity value dsp is recorded; The value obtained by multiplying the maximum number of projections at the disparity value dsp in each column by a preset multiple is used as the point count threshold thresh of this column.

6. The method for determining ground parallax misdetection according to any one of claims 1 to 5, characterized in that: Also includes: When the marked obstacle is determined to be a real obstacle, this target is retained; When it is determined that the marked obstacle is a false detection, the target is filtered out.

7. A device for determining ground parallax false detection, used to implement the method for determining ground parallax false detection according to any one of claims 1 to 6, characterized in that: include: Information acquisition module: used to obtain the disparity map, V disparity map, and obstacle target frame corresponding to the marked obstacles; A road surface fitting module: used for selecting a road surface disparity point corresponding to a road surface in the V disparity map according to the disparity map to perform road surface fitting; The obstacle outer contour fitting module is used to select the obstacle disparity point corresponding to the obstacle in the V disparity map according to the obstacle target frame and the disparity map to perform obstacle fitting to obtain the obstacle outer contour; it includes: Detecting the obstacle target frame according to the disparity map; Traversing each disparity point in the obstacle target frame, if the disparity point has a disparity value greater than 0 in the disparity map, projecting the disparity point into the V disparity map; According to the value of the ordinate in the disparity map, count the points (max_dsp, y) at the maximum disparity value of each row within the corresponding range of the target frame in the V disparity map row by row, as the outer contour points of the obstacles in the target frame in the V disparity map; Fit the outer contour points with a straight line equation, record the slope of the outer contour equation as KObject and obtain the outer contour of the obstacle; Angle calculation module: used to calculate the obstacle angle between the outer contour of the obstacle and the fitted road surface; A module for judging whether an obstacle is real: used to judge that the marked obstacle is a real obstacle when the angle is greater than a preset value; A module for judging whether an obstacle is a false detection is used to judge whether the marked obstacle is a false detection when the angle is less than or equal to the preset value.

8. A control system for implementing a method for determining ground parallax misdetection, characterized in that: include: processor; a memory for storing processor-executable instructions; Wherein, the processor is configured to implement the method for determining ground parallax misdetection as described in any one of claims 1 to 6 when executing the executable instructions.

9. A non-volatile computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method for determining ground parallax misdetection described in any one of claims 1 to 6 is implemented.

Images