A method for identifying small obstacles in parking spaces based on visual depth estimation

Abstract

The present invention discloses a method for identifying small obstacles in parking spaces based on visual depth estimation, comprising the following steps: starting a parking space search function; searching for available pre-selected parking spaces; outputting available pre-selected parking spaces after searching; entering radar obstacle detection and parking space space fusion judgment: preliminarily determining whether the parking space is available, and if not, entering a new parking space search judgment; if available, identifying small obstacles in the parking space; determining whether the parking space status is confirmed for the first time, and if so, starting automatic parking and starting a second parking space status confirmation process during parking, that is, performing radar obstacle detection and parking space space fusion judgment again; if not, starting automatic parking, and closing the parking space search function after parking is completed. The present invention can accurately identify the status of small obstacles in parking spaces, including opening ground locks, and make parking space availability judgments without changing the hardware equipment at the parking lot end, thereby greatly improving the safety of automatic parking.

Description

Technical Field

[0001] The present invention belongs to the technical field of automatic parking, and in particular relates to a method for identifying small obstacles in parking spaces based on visual depth estimation. Background Art

[0002] In the current development of automatic parking technology, ultrasonic radar was initially used to detect parking spaces, and then it was developed to combine vision and ultrasonic sensor fusion to detect parking spaces and further determine the availability of parking spaces. However, in the actual application process, a problem will be encountered. When encountering small and low obstacles, since the ground clearance of sedans is between 110-150mm and the ground clearance of SUVs is between 200-250mm, the low recognition accuracy of vision and radar for small obstacles will affect the system's judgment of parking space availability, and thus affect the safety of automatic parking.

[0003] For example, in the identification of the ground lock status, it is not possible to determine whether the parking space ground lock is open or closed through ultrasonic sensors and vision, because when ultrasonic radar identifies low targets such as ground locks, it is difficult to identify the open or closed state of the ground lock due to sensor performance limitations, which in turn affects the judgment of parking space availability; for fusion parking combined with vision, vision can identify the existence of ground locks through deep learning target detection algorithms, but due to the wide variety of ground locks and the small difference in the open and closed visual images, it is difficult to directly classify the open and closed state of the ground lock through deep learning target detection algorithms, especially when encountering ground locks of unconventional shapes, the accuracy of visual direct judgment of the open and closed state of the ground lock is further reduced. For another example, when encountering floor tiles on brick-floor parking spaces or stones on cement-floor parking spaces that affect parking safety, it is difficult for radar to identify such small obstacles at a low height. For visual fusion parking, since such obstacles are very similar to the parking space background, the accuracy of identifying obstacles through target detection is low, and the height cannot be determined, and it is impossible to determine whether it is a small obstacle in the parking space.

[0004] At present, ultrasonic radar can identify parking spaces or combine it with visual fusion to identify parking spaces. The driver can also confirm the selection to avoid the problem of selecting parking spaces with open ground locks or other small obstacles that affect parking as target empty parking spaces. However, in automatic parking situations such as LAPA memory parking and AVP valet parking where there is no driver monitoring at all times, the inability to accurately identify small obstacles in parking spaces may cause automatic parking safety risks such as scratching the chassis. In LAPA and AVP, since there is no driver on-site to finally confirm the availability of the vehicle, the availability of the parking space requires the parking perception system to confirm it autonomously. Summary of the invention

[0005] The purpose of the present invention is to provide a method for identifying small obstacles in parking spaces based on visual depth estimation. Without changing the hardware equipment at the parking lot end, the method can accurately identify the status of small obstacles in parking spaces, including opening ground locks, and make parking space availability judgments, thereby greatly improving the safety of automatic parking.

[0006] In order to solve the above technical problems, the technical solution of the present invention is: a method for identifying small obstacles in parking spaces based on visual depth estimation, comprising a small obstacle identification system arranged in an automatic parking system, the small obstacle identification system comprising a parking space corresponding side view selection module, a corresponding side view image preprocessing module, a monocular image depth detection module, a target area depth map clipping module, a target area depth difference calculation module and a target area obstacle judgment module; the method comprises the following steps:

[0007] S1. Enable the parking space search function;

[0008] S2, searching for available pre-selected parking spaces;

[0009] S3, searching for available pre-selected parking spaces and outputting them;

[0010] S4, radar obstacle detection and parking space fusion judgment:

[0011] Preliminarily determine whether the parking space is available. If not, enter the search for a new parking space; if available, identify small obstacles in the parking space;

[0012] Small obstacle recognition: Determine whether there are small obstacles in the parking space. If so, enter the search for a new parking space; if not, determine whether the parking space is currently available;

[0013] New parking space search judgment: judge whether to continue searching for a new parking space, if yes, return to S2; if no, close the parking space search function;

[0014] S5. Determine whether the parking space status is confirmed for the first time. If so, start automatic parking and start the second parking space status confirmation process during the parking process, that is, perform radar obstacle detection and parking space fusion judgment again; if not, start automatic parking and turn off the parking space search function after parking is completed.

[0015] The steps for small obstacle recognition are:

[0016] According to the coordinates of the center point of the available parking space obtained by preliminary determination, a fisheye camera in a parking space corresponding side view selection module corresponding to the position of the available parking space is selected to output a fisheye image;

[0017] The fisheye image is preprocessed to obtain a corresponding side view image; the preprocessing is to calibrate the fisheye camera to obtain a corresponding calibration description file, perform distortion correction on the fisheye image according to the calibration description file, adjust the image brightness, perform median filtering, and remove abnormal noise to obtain the corresponding side view image;

[0018] The depth information of each pixel in the corresponding side view image is predicted by the depth estimation algorithm to obtain the side view monocular depth map;

[0019] The parking area in the side-view monocular depth map is taken as the target area and cropped to obtain a depth map of the target area;

[0020] Calculate the depth difference between each pixel in the target area depth map and its surrounding pixels to obtain the maximum depth difference D of the target area depth map. m , maximum depth difference D m The corresponding target pixel depth d black and the maximum depth difference D m The pixel depth d of the neighboring points gray ;

[0021] According to the pixel depth d of the adjacent points gray , maximum depth difference D m The obstacle height is calculated based on the installation height of the fisheye camera. When the obstacle meets the preset height condition, it is determined that there is a small obstacle that affects parking safety; otherwise, it does not affect parking safety.

[0022] The method for calculating the depth difference between each pixel in the target area depth map and its surrounding pixels is the hole adjacent depth difference calculation method, and its implementation steps are: to a certain extent, expand the receptive field of pixel depth difference calculation, adjust the receptive field according to the different hole rates, select five specific points around the target pixel as adjacent points, and calculate the depth difference between the target pixel and the adjacent points respectively, and screen to obtain the maximum depth difference D of the target area depth map. m , and calculate the maximum depth difference D m The corresponding target pixel depth d black and the maximum depth difference D m The pixel depth d of the neighboring points gray ; The specific five points are selected as: the farthest points directly above, directly to the left, and directly to the right of the target pixel in the receptive field, and the two points farthest from the target pixel.

[0023] Maximum depth difference D m and its corresponding target pixel depth d black and the maximum depth difference D m The pixel depth d of the neighboring points gray The calculation method is:

[0024] The depth differences between the target pixel and the farthest points directly above, directly to the left, and directly to the right of the target pixel in the receptive field, as well as the two points farthest from the target pixel, are calculated to be Δd1, Δd2, Δd3, Δd4, and Δd5 respectively;

[0025] At the edge of the image in the depth map of the target area, pixels are expanded around the edge of the pixel. The RGB-D value of the expanded pixel is the same as the target pixel value at the edge. Then there is the maximum pixel depth difference D at a target pixel point i on the depth map of the target area. i The calculation formula is:

[0026] D i =Max(Δd1,Δd2,Δd3,Δd4,Δd5)

[0027] The maximum depth difference D of the target area depth map m The calculation formula is:

[0028] D m =Max(D1, D2, D3, ... D i …)

[0029] Then the maximum depth difference D of the target area depth map is m , and the corresponding target pixel depth d black , the pixel depth of the adjacent point with the maximum depth difference d gray The relationship between:

[0030] D m =|d black -d gray |.

[0031] The specific calculation method of obstacle height is as follows: The installation height H of the fisheye camera v and obstacle height H ob The following calculation relationship exists:

[0032]

[0033] Then we have:

[0034]

[0035] When the obstacle height H ob When the following preset altitude conditions are met:

[0036]

[0037] It is determined that there is a small obstacle that affects parking safety; otherwise, it does not affect parking safety; where L d is the minimum height of the vehicle chassis, σ is the obstacle chassis safety factor, and σ≥1.

[0038] A parking space small obstacle recognition system based on visual depth estimation is also provided, comprising:

[0039] A parking space corresponding side view selection module is used to select a fisheye camera in the parking space corresponding side view selection module corresponding to the position of the available parking space to output a fisheye image according to the coordinates of the center point of the available parking space obtained by preliminary determination;

[0040] The corresponding side view image preprocessing module is used to preprocess the fisheye image to obtain the corresponding side view image; the preprocessing is to calibrate the fisheye camera to obtain the corresponding calibration description file, perform distortion correction on the fisheye image according to the calibration description file, adjust the image brightness, perform median filtering, and remove abnormal noise to obtain the corresponding side view image;

[0041] The monocular image depth detection module is used to predict the depth information of each pixel in the corresponding side view image through the depth estimation algorithm to obtain the side view monocular depth map;

[0042] A target area depth map cropping module, used to take the parking area in the side view monocular depth map as the target area and crop it to obtain a target area depth map;

[0043] The target area depth difference calculation module is used to calculate the depth difference between each pixel in the target area depth map and its surrounding pixels, and obtain the maximum depth difference D of the target area depth map. m , maximum depth difference D m The corresponding target pixel depth d black and the maximum depth difference D m The pixel depth d of the neighboring points gray ;

[0044] The target area obstacle judgment module is used to determine the obstacle according to the pixel depth d of the adjacent points. gray , maximum depth difference D m The obstacle height is calculated based on the installation height of the fisheye camera. When the obstacle meets the preset height condition, it is determined that there is a small obstacle that affects parking safety; otherwise, it does not affect parking safety.

[0045] The method for calculating the depth difference between each pixel in the target area depth map and its surrounding pixels is the hole adjacent depth difference calculation method, and its implementation steps are: to a certain extent, expand the receptive field of pixel depth difference calculation, adjust the receptive field according to the different hole rates, select five specific points around the target pixel as adjacent points, and calculate the depth difference between the target pixel and the adjacent points respectively, and screen to obtain the maximum depth difference D of the target area depth map. m , and calculate the maximum depth difference D m The corresponding target pixel depth d black and the maximum depth difference Dm The pixel depth d of the neighboring points gray ; The specific five points are selected as: the farthest points directly above, directly to the left, and directly to the right of the target pixel in the receptive field, and the two points farthest from the target pixel.

[0046] Maximum depth difference D m and its corresponding target pixel depth d black and the maximum depth difference D m The pixel depth d of the neighboring points gray The calculation method is:

[0047] The depth differences between the target pixel and the farthest points directly above, directly to the left, and directly to the right of the target pixel in the receptive field, as well as the two points farthest from the target pixel, are calculated to be Δd1, Δd2, Δd3, Δd4, and Δd5 respectively;

[0048] At the edge of the image in the depth map of the target area, pixels are expanded around the edge of the pixel. The RGB-D value of the expanded pixel is the same as the target pixel value at the edge. Then there is the maximum pixel depth difference D at a target pixel point i on the depth map of the target area. i The calculation formula is:

[0049] D i =Max(Δd1,Δd2,Δd3,Δd4,Δd5)

[0050] The maximum depth difference D of the target area depth map m The calculation formula is:

[0051] D m =Max(D1, D2, D3, ... D i …)

[0052] Then the maximum depth difference D of the target area depth map is m , and the corresponding target pixel depth d black , the pixel depth of the adjacent point with the maximum depth difference d gray The relationship between:

[0053] D m =|d black -d gray |.

[0054] The specific calculation method of obstacle height is as follows: The installation height H of the fisheye camera v and obstacle height H ob The following calculation relationship exists:

[0055]

[0056] Then we have:

[0057]

[0058] When the obstacle height H ob When the following preset altitude conditions are met:

[0059]

[0060] It is determined that there is a small obstacle that affects parking safety; otherwise, it does not affect parking safety; where L d is the minimum height of the vehicle chassis, σ is the obstacle chassis safety factor, and σ≥1.

[0061] A computer-readable storage medium is also provided, on which a computer program is stored, and when the computer program is executed by a processor, the steps of any of the above methods are implemented.

[0062] Compared with the prior art, the present invention has the following beneficial effects:

[0063] The present invention is based on visual depth estimation to judge the visual depth difference of the parking area, which fundamentally makes up for the performance defects of radar in identifying obstacles and solves the problems of false detection and missed detection when visual target detection identifies obstacles. It can largely solve the problem of identifying small obstacles in parking spaces, improve the accuracy of parking space availability judgment, greatly improve the safety of automatic parking in unmanned processes such as LAPA memory parking and AVP automatic parking, and reduce users' concerns about the safety of automatic parking functions.

[0064] After identifying the parking space status, the parking space ground lock may be temporarily opened, which may cause damage to the vehicle during the parking process and bring about vehicle safety issues. The secondary detection and confirmation of small obstacles during the parking process of the present invention can greatly avoid the occurrence of such situations that are likely to cause safety problems. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 A schematic diagram of a process of an embodiment of the present invention;

[0066] Figure 2 This is a schematic diagram of the information transmission sequence of each module in the system of an embodiment of the present invention;

[0067] Figure 3 Schematic diagram of the receptive field in an embodiment of the present invention;

[0068] Figure 4 A schematic top view of a vehicle and a small obstacle in an embodiment of the present invention;

[0069] Figure 5 Schematic diagram of the height relationship between a vehicle and a small obstacle in an embodiment of the present invention. DETAILED DESCRIPTION

[0070] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0071] Automatically searching for available parking spaces requires the use of surround-view fisheye cameras and ultrasonic radar sensors. To solve the above problems, this invention proposes a method for identifying small obstacles in parking spaces based on depth estimation. The main application process of this small obstacle identification module in the automatic parking system is as follows: Figure 1 shown.

[0072] The small obstacle recognition method for parking spaces mainly includes: a parking space corresponding side view selection module, a corresponding side view image preprocessing module, a monocular image depth detection module, a target area depth map clipping module, a target area depth difference calculation module, and a target area depth. The corresponding process is as follows Figure 2 shown.

[0073] In the parking space corresponding side view selection module, the available pre-selected parking spaces are searched and output by the parking space search system, and then the preliminarily determined available parking spaces are output after being integrated with the non-small obstacles detected by the radar. Based on the coordinates of the center point of the preliminarily determined available parking spaces, the fisheye camera corresponding to the side view of the parking space is determined, and the fisheye image corresponding to the side view is selected and output separately for the next step of processing.

[0074] In the corresponding side view image preprocessing module, the input of this module is the monocular fisheye image corresponding to the preliminary determination of the available parking space, and the output is the corresponding side view image processed by the preprocessing module. The preprocessing process of this module mainly includes: performing distortion correction processing on the image, because the fisheye image will have obvious distortion. After calibrating the fisheye camera, the calibration description file corresponding to the camera will be obtained. According to the corresponding calibration description file, the monocular fisheye image is subjected to distortion correction. The corrected image also needs to be adjusted in brightness according to the brightness of the image, and median filtering is performed to remove abnormal noise.

[0075] In the monocular image depth detection module, a depth estimation neural network algorithm (such as the common PackNet and other depth estimation algorithms) is used to predict the depth information of each pixel and obtain an RGB-D depth image from the RGB image. The depth estimation algorithm requires a certain truth value as the training result, so when collecting and annotating training data sets, the lidar can be used as the depth truth information for model training.

[0076] In the target area depth map cropping module, in order to reduce the amount of calculation, the parking area in the side view monocular depth map is used as the target area of ​​interest, and the target area depth map is cropped for the next step of depth difference calculation processing.

[0077] In the target area depth difference calculation module, the depth difference calculation is mainly performed between each pixel point of the target area depth map and the surrounding pixels. In order to appropriately expand the receptive field of the pixel depth difference calculation without increasing the amount of calculation, considering the characteristics of the visual depth map of small obstacles in parking spaces, the present invention proposes a hole adjacent depth difference calculation method. According to the different hole rates r, this method only needs to calculate five specific points around the target pixel point (left side, upper left, top, upper right, and right side). For example, when the hole rate r=1, the calculation receptive field of the hole adjacent depth difference calculation method is 3*5, such as Figure 3 shown.

[0078] Among them, the black dots are the target pixels that need to calculate the adjacent depth difference. When using the hole adjacent depth difference calculation method, it is only necessary to calculate the depth difference Δd1, Δd2, Δd3, Δd4, Δd5 between the pixel depth at the black dot and the pixel depth at the five gray dots. While ensuring the calculation of the adjacent receptive field range, it can greatly reduce the amount of calculation. At the edge of the image in the depth map of the target area, when using the hole adjacent depth difference calculation, the pixels around the edge of the pixel can be expanded, and the RGB-D value of the expanded pixel is the same as the target pixel value at the edge.

[0079] Then the maximum pixel depth difference D at a pixel i in the target area depth map is i The calculation formula is as follows:

[0080] D i =Max(Δd1,Δd2,Δd3,Δd4,Δd5)

[0081] Then the maximum depth difference D of the target area depth map m The calculation formula is as follows:

[0082] D m =Max(D1, D2, D3, ... D i …)

[0083] The target area depth difference calculation module outputs the maximum depth difference D of the target area depth map. m , and the corresponding target pixel depth d black , the pixel depth of the adjacent point with the maximum depth difference d gray , and their mutual relationships are as follows:

[0084] D m =|d black -d gray |

[0085] In the target area obstacle judgment module, assuming that the minimum height of the vehicle chassis is L d , the obstacle chassis safety factor is σ (σ≥1, for example, 1.3). The installation height of the single-sided fisheye camera is a known quantity H v , the height of the obstacle is the unknown quantity H ob , then Figure 4 and Figure 5 as shown.

[0086] According to the triangle similarity theorem, we can get the following relationship:

[0087]

[0088] Further find the unknown obstacle height H ob as follows:

[0089]

[0090] Where H v Known quantity, D m d gray and d black It can be obtained through the target area depth difference calculation module, so H ob It can be calculated using the above formula.

[0091] When the obstacle height H ob When the following conditions are met,

[0092]

[0093] The obstacle is a small obstacle in the parking space and will affect parking safety. Otherwise, it will not affect parking safety and can be ignored.

[0094] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for identifying small obstacles in parking spaces based on visual depth estimation, characterized in that: It includes a small obstacle recognition system arranged in the automatic parking system, and the small obstacle recognition system includes a parking space corresponding side view selection module, a corresponding side view image preprocessing module, a monocular image depth detection module, a target area depth map clipping module, a target area depth difference calculation module and a target area obstacle judgment module; The method comprises the following steps: S1. Enable the parking space search function; S2, searching for available pre-selected parking spaces; S3, searching for available pre-selected parking spaces and outputting them; S4, radar obstacle detection and parking space fusion judgment: Preliminarily determine whether the parking space is available. If not, enter the search for a new parking space; if available, identify small obstacles in the parking space; Small obstacle recognition: Determine whether there are small obstacles in the parking space. If so, enter the search for a new parking space; if not, determine whether the parking space is currently available; New parking space search judgment: judge whether to continue searching for a new parking space, if yes, return to S2; if no, close the parking space search function; S5. Determine whether the parking space status is confirmed for the first time. If so, start the automatic parking and start the second parking space status confirmation process during the parking process, that is, perform radar obstacle detection and parking space fusion judgment again; if not, start the automatic parking and turn off the parking space search function after parking is completed; The steps for small obstacle recognition are: According to the coordinates of the center point of the available parking space obtained by preliminary determination, a fisheye camera in a parking space corresponding side view selection module corresponding to the position of the available parking space is selected to output a fisheye image; The fisheye image is preprocessed to obtain a corresponding side view image; the preprocessing is to calibrate the fisheye camera to obtain a corresponding calibration description file, perform distortion correction on the fisheye image according to the calibration description file, adjust the image brightness, perform median filtering, and remove abnormal noise to obtain the corresponding side view image; The depth information of each pixel in the corresponding side view image is predicted by the depth estimation algorithm to obtain the side view monocular depth map; The parking area in the side-view monocular depth map is taken as the target area and cropped to obtain a depth map of the target area; Calculate the depth difference between each pixel in the target area depth map and its surrounding pixels to obtain the maximum depth difference D of the target area depth map. m , maximum depth difference D m The corresponding target pixel depth d black and the maximum depth difference D m The pixel depth d of the neighboring points gray ; According to the pixel depth d of the adjacent points gray , maximum depth difference D m The obstacle height is calculated based on the installation height of the fisheye camera. When the obstacle meets the preset height condition, it is determined that there is a small obstacle that affects parking safety; otherwise, it does not affect parking safety.

2. The method for identifying small obstacles in parking spaces based on visual depth estimation according to claim 1, characterized in that: The method for calculating the depth difference between each pixel in the target area depth map and its surrounding pixels is the hole adjacent depth difference calculation method, and its implementation steps are: to a certain extent, expand the receptive field of pixel depth difference calculation, adjust the receptive field according to the different hole rates, select five specific points around the target pixel as adjacent points, and calculate the depth difference between the target pixel and the adjacent points respectively, and screen to obtain the maximum depth difference D of the target area depth map. m , and calculate the maximum depth difference D m The corresponding target pixel depth d black and the maximum depth difference D m The pixel depth d of the neighboring points gray ; The specific five points are selected as: the farthest points directly above, directly to the left, and directly to the right of the target pixel in the receptive field, and the two points farthest from the target pixel.

3. The method for identifying small obstacles in parking spaces based on visual depth estimation according to claim 2, characterized in that: Maximum depth difference D m and its corresponding target pixel depth d black and the maximum depth difference D m The pixel depth d of the neighboring points gray The calculation method is: The depth differences between the target pixel and the farthest points directly above, directly to the left, and directly to the right of the target pixel in the receptive field, as well as the two points farthest from the target pixel, are calculated to be Δd1, Δd2, Δd3, Δd4, and Δd5 respectively; At the edge of the image in the depth map of the target area, pixels are expanded around the edge of the pixel. The RGB-D value of the expanded pixel is the same as the target pixel value at the edge. Then there is the maximum pixel depth difference D at a target pixel point i on the depth map of the target area. i The calculation formula is: D i =Max(Δd1,Δd2,Δd3,Δd4,Δd5) The maximum depth difference D of the target area depth map m The calculation formula is: D m =Max(D1,D2,D3,…D i ……) Then the maximum depth difference D of the target area depth map is m , and the corresponding target pixel depth d black , the pixel depth of the adjacent point with the maximum depth difference d gray The relationship between: D m =|d black -d gray |。 4. The method for identifying small obstacles in parking spaces based on visual depth estimation according to claim 3, characterized in that: The specific calculation method of obstacle height is as follows: The installation height H of the fisheye camera v and obstacle height H ob The following calculation relationship exists: Then we have: When the obstacle height H ob When the following preset altitude conditions are met: It is determined that there is a small obstacle that affects parking safety; otherwise, it does not affect parking safety; where L d is the minimum height of the vehicle chassis, σ is the obstacle chassis safety factor, and σ≥1.

5. A system using the method for identifying small obstacles in parking spaces based on visual depth estimation as claimed in claim 1, characterized in that: include: A parking space corresponding side view selection module is used to select a fisheye camera in the parking space corresponding side view selection module corresponding to the position of the available parking space to output a fisheye image according to the coordinates of the center point of the available parking space obtained by preliminary determination; The corresponding side view image preprocessing module is used to preprocess the fisheye image to obtain the corresponding side view image; the preprocessing is to calibrate the fisheye camera to obtain the corresponding calibration description file, perform distortion correction on the fisheye image according to the calibration description file, adjust the image brightness, perform median filtering, and remove abnormal noise to obtain the corresponding side view image; The monocular image depth detection module is used to predict the depth information of each pixel in the corresponding side view image through the depth estimation algorithm to obtain the side view monocular depth map; A target area depth map cropping module, used to take the parking area in the side view monocular depth map as the target area and crop it to obtain a target area depth map; The target area depth difference calculation module is used to calculate the depth difference between each pixel in the target area depth map and its surrounding pixels, and obtain the maximum depth difference D of the target area depth map. m , maximum depth difference D m The corresponding target pixel depth d black and the maximum depth difference D m The pixel depth d of the neighboring points gray ; The target area obstacle judgment module is used to determine the obstacle according to the pixel depth d of the adjacent points. gray , maximum depth difference D m The obstacle height is calculated based on the installation height of the fisheye camera. When the obstacle meets the preset height condition, it is determined that there is a small obstacle that affects parking safety; otherwise, it does not affect parking safety.

6. The system according to claim 5, characterized in that The method for calculating the depth difference between each pixel in the target area depth map and its surrounding pixels is the hole adjacent depth difference calculation method, and its implementation steps are: to a certain extent, expand the receptive field of pixel depth difference calculation, adjust the receptive field according to the different hole rates, select five specific points around the target pixel as adjacent points, and calculate the depth difference between the target pixel and the adjacent points respectively, and screen to obtain the maximum depth difference D of the target area depth map. m , and calculate the maximum depth difference D m The corresponding target pixel depth d black and the maximum depth difference D m The pixel depth d of the neighboring points gray ; The specific five points are selected as: the farthest points directly above, directly to the left, and directly to the right of the target pixel in the receptive field, and the two points farthest from the target pixel.

7. The system according to claim 6, characterized in that Maximum depth difference D m and its corresponding target pixel depth d black and the maximum depth difference D m The pixel depth d of the neighboring points gray The calculation method is: The depth differences between the target pixel and the farthest points directly above, directly to the left, and directly to the right of the target pixel in the receptive field, as well as the two points farthest from the target pixel, are calculated to be Δd1, Δd2, Δd3, Δd4, and Δd5 respectively; At the edge of the image in the depth map of the target area, pixels are expanded around the edge of the pixel. The RGB-D value of the expanded pixel is the same as the target pixel value at the edge. Then there is the maximum pixel depth difference D at a target pixel point i on the depth map of the target area. i The calculation formula is: D i =Max(Δd1,Δd2,Δd3,Δd4,Δd5) The maximum depth difference D of the target area depth map m The calculation formula is: D m =Max(D1,D2,D3,…D i ……) Then the maximum depth difference D of the target area depth map is m , and the corresponding target pixel depth d black , the pixel depth of the adjacent point with the maximum depth difference d gray The relationship between: D m =|d black -d gray |。 8. The system according to claim 3, characterized in that the obstacle The specific height calculation method is: The installation height H of the fisheye camera v and obstacle height H ob The following calculation relationship exists: Then we have: When the obstacle height H ob When the following preset altitude conditions are met: It is determined that there is a small obstacle that affects parking safety; otherwise, it does not affect parking safety; where L d is the minimum height of the vehicle chassis, σ is the obstacle chassis safety factor, and σ≥1.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images