Method and device for detecting distance between vehicle and obstacle

Abstract

The invention discloses a method and device for detecting the distance between a vehicle and an obstacle, and the method comprises the steps: obtaining a target fisheye image including the obstacle, the target fisheye image is an image used for describing the surrounding environment of the vehicle, and the obstacle included in the target fisheye image is a non-ground object with continuous features; and converting the target fisheye image into a hue saturation brightness color space to obtain a saturation sub-image and a brightness sub-image, and respectively determining a first non-driving area and a second non-driving area of the vehicle in the road according to the saturation sub-image and the brightness sub-image. The first non-driving area and the second non-driving area are areas in the target fisheye image, in which the vehicle cannot drive, or areas in which obstacles in the road are located, which are respectively determined in two modes. According to the first non-driving area and the second non-driving area, the optimal fitting line segment is determined, and description of the edge of the non-road area in the road is more accurate. And determining the distance between the vehicle and the obstacle based on the optimal fitting line segment.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle control, and in particular to a method and device for detecting the distance between a vehicle and an obstacle. Background Art

[0002] With the rapid development and innovation of vehicle industry technology, vehicles have entered thousands of households. In many scenarios, if the driver is not sensitive to the size of the vehicle and the environment, it may cause accidents such as the vehicle scraping against obstacles. For example, when driving a vehicle to park on the side of the road or in a parallel parking situation, if the driver is not sensitive to the size of the vehicle and the environment, the tire may scrape against the shoulder of the road.

[0003] Taking the shoulder as an example of an obstacle, in the relevant technology, three-dimensional imaging equipment such as lidar is generally installed on the vehicle, and three-dimensional point cloud information of the vehicle's surrounding environment is generated based on the three-dimensional imaging equipment. Objects above the ground are identified based on plane fitting and clustering, and non-shoulder targets are filtered out according to specific heights. Point cloud information that meets the height conditions and morphological characteristics is retained, so as to fit the shoulder edge and realize shoulder identification and positioning.

[0004] Although this method can use spatial information to more accurately identify the road shoulder, it generally also needs to be combined with visible light visual information to perceive the road shoulder, and the cost of three-dimensional imaging equipment is relatively high, so the application scenarios are limited. Summary of the invention

[0005] In response to the above problems, the present application provides a method and device for detecting the distance between a vehicle and an obstacle, which are used to reduce the cost of vehicle identification and obstacle positioning and expand the scope of application scenarios.

[0006] Based on this, the embodiments of the present application disclose the following technical solutions:

[0007] On the one hand, an embodiment of the present application provides a method for detecting the distance between a vehicle and an obstacle, the method comprising:

[0008] Acquire a target fisheye image including an obstacle, wherein the target fisheye image is an image used to describe the surrounding environment of the vehicle, and the obstacle is a non-ground object with continuous features;

[0009] Convert the target fisheye image to a hue, saturation, and brightness color space to obtain a saturation sub-image and a brightness sub-image for the pixel points included in the target fisheye image;

[0010] determining a first non-driving area of ​​the vehicle on the road according to the saturation sub-image;

[0011] determining a second non-driving area of ​​the vehicle on the road according to the brightness sub-image;

[0012] determining a best-fit line segment according to the first non-driving area and the second non-driving area, wherein the best-fit line segment is used to describe an edge of the non-road area of ​​the road;

[0013] The distance between the vehicle and the obstacle is determined according to the best fitting line segment.

[0014] Optionally, acquiring a target fisheye image including obstacles includes:

[0015] Acquire a fisheye image set and camera internal and external parameter calibration information, wherein the fisheye image set includes a plurality of fisheye images taken from different positions of the vehicle;

[0016] Correcting the fisheye images included in the fisheye image set according to the camera internal and external parameter calibration information to obtain a distortion-free top view image set including a plurality of distortion-free top view images;

[0017] Identify the overlapping area between the plurality of the distortion-free overhead images to obtain an identification result, wherein the identification result is used to identify whether the overlapping area between two of the distortion-free overhead images in the set of distortion-free overhead images includes an object whose morphological difference is greater than a preset morphological threshold;

[0018] If it is determined according to the recognition result that the target undistorted overhead image includes an object whose morphological difference is greater than a preset morphological threshold, the target undistorted overhead image is determined as the target fisheye image, and the target undistorted overhead image is one of the two undistorted overhead images that include an object whose morphological difference is greater than the preset morphological threshold.

[0019] Optionally, the identifying an overlapping area between the plurality of the distortion-free overhead images to obtain an identification result includes:

[0020] Acquire a first binary contour image corresponding to a first distortion-free top view image and a second binary contour image corresponding to a second distortion-free top view image, wherein the first distortion-free top view image and the second distortion-free top view image are two distortion-free top view images including an overlapping area;

[0021] compressing the first binary contour image to obtain a first one-dimensional identity code, and compressing the second binary contour image to obtain a second one-dimensional identity code;

[0022] According to the first one-dimensional identity code and the second one-dimensional identity code, identification results for the first undistorted top view image and the second undistorted top view image are determined.

[0023] Optionally, determining a first non-driving area of ​​the vehicle on the road according to the saturation sub-image includes:

[0024] Determine the pixel points included in the target fisheye image as target pixel points respectively;

[0025] If the pixel value of the target pixel in the saturation sub-image is greater than or equal to a preset saturation threshold, the target pixel is added to a first non-driving area of ​​the vehicle on the road.

[0026] Optionally, the method further includes:

[0027] Acquire a first one-dimensional identity code and a second one-dimensional identity code, wherein the first one-dimensional identity code and the second one-dimensional identity code are determined based on two undistorted top-view images including an overlapping area;

[0028] comparing the first one-dimensional identity code and the second one-dimensional identity code to obtain inconsistency points;

[0029] The first non-driving area is adjusted according to the inconsistent points to obtain an updated first non-driving area.

[0030] Optionally, determining a second non-driving area of ​​the vehicle on the road according to the brightness sub-image includes:

[0031] Image equalization and threshold segmentation are performed on the brightness sub-image of the pixel points included in the target fisheye image to obtain a second non-driving area of ​​the vehicle on the road.

[0032] Optionally, determining the best fitting line segment for identifying the obstacle according to the first non-driving area and the second non-driving area includes:

[0033] Obtaining a non-road edge image according to the first non-driving area and the second non-driving area;

[0034] Traversing the non-road edge image, selecting the first m non-zero pixels in each row to obtain the obstacle contour;

[0035] Performing Hough probability straight line detection on the obstacle contour to obtain n candidate fitting line segments;

[0036] According to preset conditions, the optimal fitting line segment is determined from the n candidate fitting line segments, wherein the preset conditions are one or a combination of multiple conditions: the segment length of the candidate fitting line segment, the distance between the midpoint of the candidate fitting line segment and the vehicle, the number of fitting points in the candidate fitting line segment, and the variance distance between the segment angle of the candidate fitting line segment and the average angle of the candidate fitting line segment.

[0037] Optionally, if the distance between the vehicle and the obstacle is a measured distance between the vehicle and the obstacle, determining the distance between the vehicle and the obstacle according to the optimal fitting line segment includes:

[0038] Obtaining the gear position of the vehicle and the conversion ratio between image coordinates and world coordinates;

[0039] If the gear position of the vehicle is the forward gear, determining the distance between the vehicle and the optimal fitting line segment based on the front wheel of the vehicle corresponding to the target fisheye image;

[0040] If the gear position of the vehicle is the reverse gear, the distance between the vehicle and the optimal fitting line segment is determined based on the rear wheel of the vehicle corresponding to the target fisheye image;

[0041] The measured distance between the vehicle and the obstacle is determined according to the conversion ratio between the image coordinates and the world coordinates and the distance between the vehicle and the best fitting line segment.

[0042] Optionally, if the distance between the vehicle and the obstacle is an actual distance adjusted based on a measured distance between the vehicle and the obstacle, the actual distance at time k is determined as follows:

[0043] Get the predicted distance at time k-1, the vehicle speed at time k, and the tire steering angle at time k;

[0044] Determine the predicted distance at time k according to the actual distance at time k-1, the vehicle speed at time k and the tire steering angle at time k;

[0045] The actual distance at time k is determined according to the predicted distance at time k, the measured distance at time k and the gain at time k, wherein the gain at time k is determined according to the system prediction error at time k.

[0046] On the other hand, the present application provides a device for detecting the distance between a vehicle and an obstacle, the method comprising: an acquisition unit, a conversion unit, and a determination unit;

[0047] The acquisition unit is used to acquire a target fisheye image including an obstacle, wherein the target fisheye image is an image used to describe the surrounding environment of the vehicle, and the obstacle is a non-ground object with continuous features;

[0048] The conversion unit is used to convert the target fisheye image into a hue, saturation and brightness color space to obtain a saturation sub-image and a brightness sub-image for the pixel points included in the target fisheye image;

[0049] The determining unit is used to determine a first non-driving area of ​​the vehicle on the road according to the saturation sub-image;

[0050] The determining unit is further used to determine a second non-driving area of ​​the vehicle on the road according to the brightness sub-image;

[0051] The determining unit is further used to determine a best-fit line segment according to the first non-driving area and the second non-driving area, wherein the best-fit line segment is used to describe an edge of the non-road area of ​​the road;

[0052] The determining unit is further configured to determine the distance between the vehicle and the obstacle according to the optimal fitting line segment.

[0053] Optionally, the acquisition unit is specifically used to:

[0054] Acquire a fisheye image set and camera internal and external parameter calibration information, wherein the fisheye image set includes a plurality of fisheye images taken from different positions of the vehicle;

[0055] Correcting the fisheye images included in the fisheye image set according to the camera internal and external parameter calibration information to obtain a distortion-free top view image set including a plurality of distortion-free top view images;

[0056] Identify the overlapping area between the plurality of the distortion-free overhead images to obtain an identification result, wherein the identification result is used to identify whether the overlapping area between two of the distortion-free overhead images in the set of distortion-free overhead images includes an object whose morphological difference is greater than a preset morphological threshold;

[0057] If it is determined according to the recognition result that the target undistorted overhead image includes an object whose morphological difference is greater than a preset morphological threshold, the target undistorted overhead image is determined as the target fisheye image, and the target undistorted overhead image is one of the two undistorted overhead images that include an object whose morphological difference is greater than the preset morphological threshold.

[0058] Optionally, the acquisition unit is specifically used to:

[0059] Acquire a first binary contour image corresponding to a first distortion-free top view image and a second binary contour image corresponding to a second distortion-free top view image, wherein the first distortion-free top view image and the second distortion-free top view image are two distortion-free top view images including an overlapping area;

[0060] compressing the first binary contour image to obtain a first one-dimensional identity code, and compressing the second binary contour image to obtain a second one-dimensional identity code;

[0061] According to the first one-dimensional identity code and the second one-dimensional identity code, identification results for the first undistorted top view image and the second undistorted top view image are determined.

[0062] Optionally, the determining unit is specifically configured to:

[0063] Determine the pixel points included in the target fisheye image as target pixel points respectively;

[0064] If the pixel value of the target pixel in the saturation sub-image is greater than or equal to a preset saturation threshold, the target pixel is added to a first non-driving area of ​​the vehicle on the road.

[0065] Optionally, the device further includes an updating unit, configured to:

[0066] Acquire a first one-dimensional identity code and a second one-dimensional identity code, wherein the first one-dimensional identity code and the second one-dimensional identity code are determined based on two undistorted top-view images including an overlapping area;

[0067] comparing the first one-dimensional identity code and the second one-dimensional identity code to obtain inconsistency points;

[0068] The first non-driving area is adjusted according to the inconsistent points to obtain an updated first non-driving area.

[0069] Optionally, the determining unit is specifically configured to:

[0070] Image equalization and threshold segmentation are performed on the brightness sub-image of the pixel points included in the target fisheye image to obtain a second non-driving area of ​​the vehicle on the road.

[0071] Optionally, the determining unit is specifically configured to:

[0072] Obtaining a non-road edge image according to the first non-driving area and the second non-driving area;

[0073] Traversing the non-road edge image, selecting the first m non-zero pixels in each row to obtain the obstacle contour;

[0074] Performing Hough probability straight line detection on the obstacle contour to obtain n candidate fitting line segments;

[0075] According to preset conditions, the optimal fitting line segment is determined from the n candidate fitting line segments, wherein the preset conditions are one or a combination of multiple conditions: the segment length of the candidate fitting line segment, the distance between the midpoint of the candidate fitting line segment and the vehicle, the number of fitting points in the candidate fitting line segment, and the variance distance between the segment angle of the candidate fitting line segment and the average angle of the candidate fitting line segment.

[0076] Optionally, if the distance between the vehicle and the obstacle is a measured distance between the vehicle and the obstacle, the determining unit is specifically configured to:

[0077] Obtaining the gear position of the vehicle and the conversion ratio between image coordinates and world coordinates;

[0078] If the gear position of the vehicle is the forward gear, determining the distance between the vehicle and the optimal fitting line segment based on the front wheel of the vehicle corresponding to the target fisheye image;

[0079] If the gear position of the vehicle is the reverse gear, the distance between the vehicle and the optimal fitting line segment is determined based on the rear wheel of the vehicle corresponding to the target fisheye image;

[0080] The measured distance between the vehicle and the obstacle is determined according to the conversion ratio between the image coordinates and the world coordinates and the distance between the vehicle and the best fitting line segment.

[0081] Optionally, if the distance between the vehicle and the obstacle is an actual distance adjusted based on a measured distance between the vehicle and the obstacle, the determining unit is specifically configured to determine the actual distance at time k, and the actual distance at time k is determined as follows:

[0082] Get the predicted distance at time k-1, the vehicle speed at time k, and the tire steering angle at time k;

[0083] Determine the predicted distance at time k according to the actual distance at time k-1, the vehicle speed at time k and the tire steering angle at time k;

[0084] The actual distance at time k is determined according to the predicted distance at time k, the measured distance at time k and the gain at time k, wherein the gain at time k is determined according to the system prediction error at time k.

[0085] In another aspect, the present application provides a computer device, the device comprising a processor and a memory:

[0086] The memory is used to store program code and transmit the program code to the processor;

[0087] The processor is used to execute the method described in the above aspects according to the instructions in the program code.

[0088] On the other hand, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store a computer program, and the computer program is used to execute the method described in the above aspects.

[0089] On the other hand, an embodiment of the present application provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method described in the above aspects.

[0090] The advantages of the above technical solution of this application are:

[0091] A target fisheye image including obstacles is obtained, and the target fisheye image is an image used to describe the surrounding environment of the vehicle, and the obstacles included therein are non-ground objects with continuous features. The target fisheye image is converted into a hue, saturation, and brightness color space to obtain a saturation sub-image and a brightness sub-image, and the first non-driving area and the second non-driving area of ​​the vehicle on the road are determined respectively according to the saturation sub-image and the brightness sub-image. The first non-driving area and the second non-driving area are areas in the target fisheye image that the vehicle cannot drive, or in other words, areas where obstacles on the road are located, which are determined by two methods. The best fitting line segment determined according to the first non-driving area and the second non-driving area more accurately describes the edge of the non-road area on the road. Thus, based on the more accurate best fitting line segment, the distance between the vehicle and the obstacle is determined. As a result, it is no longer necessary to obtain point cloud data with a three-dimensional imaging device, but to obtain the target fisheye image based on a low-cost device, and determine the distance between the vehicle and the obstacle based on the target fisheye image, thereby reducing the cost of vehicle identification and obstacle positioning and expanding the scope of application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0092] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0093] Figure 1 A flow chart of a method for detecting the distance between a vehicle and an obstacle provided in an embodiment of the present application;

[0094] Figure 2 A schematic diagram of a method for detecting the distance between a vehicle and an obstacle provided in an embodiment of the present application;

[0095] Figure 3 A schematic diagram of an undistorted top view provided in an embodiment of the present application;

[0096] Figure 4A schematic diagram of an overlapping area including lane lines provided in an embodiment of the present application;

[0097] Figure 5 A schematic diagram of an overlapping area including a shoulder provided in an embodiment of the present application;

[0098] Figure 6 A schematic diagram of converting a target fisheye image into a hue, saturation, and brightness color space provided in an embodiment of the present application;

[0099] Figure 7 A schematic diagram of a first non-driving area provided in an embodiment of the present application;

[0100] Figure 8 A schematic diagram of determining the distance between a vehicle and a best-fit line segment provided in an embodiment of the present application;

[0101] Fig. 9 A schematic diagram of displaying obstacles in a panoramic 2D overhead image provided by an embodiment of the present application;

[0102] Fig.10 A schematic diagram of a distance detection device between a vehicle and an obstacle provided in an embodiment of the present application;

[0103] Fig.11 A structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0104] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0105] Combine the following Figure 1 , a method for detecting the distance between a vehicle and an obstacle provided in an embodiment of the present application is introduced. Figure 1 , which is a flow chart of a method for detecting the distance between a vehicle and an obstacle provided in an embodiment of the present application. The method may include S101-S106.

[0106] S101: Acquire a target fisheye image including obstacles.

[0107] The target fisheye image is an image used to describe the environment around the vehicle, and the target fisheye image includes obstacles. The obstacles are non-ground objects with continuous features, such as shoulders. The shoulders can be raised curbs on both sides of the road, or soil slopes on both sides of the road that are lower than the road.

[0108] As a possible implementation method, the target fisheye image may be an image obtained after an edge protection filtering operation. The edge protection filtering operation can remove image noise while retaining edge information that may be mistaken for noise, so that subsequent recognition can be more accurate. Figure 2 , which is a schematic diagram of a method for detecting the distance between a vehicle and an obstacle provided in an embodiment of the present application. Figure 2 As shown in a, the sub-image is the target fisheye image after the edge protection filtering operation, including the lane line and the shoulder.

[0109] As a possible implementation method, since there are not always obstacles around the vehicle when it is driving, in order to reduce computing power, the vehicle speed and / or steering wheel angle can be obtained.

[0110] Method 1: If the vehicle speed is less than or equal to the preset speed threshold, it means that the vehicle is in a low-speed scenario. At this time, the vehicle may need to park on the side of the road or park sideways. At this time, the probability of scratching an obstacle on the road is greatly increased, so S101 is executed to enable the distance detection between the vehicle and the obstacle in the required scenario to reduce the computing power.

[0111] Method 2: If the vehicle speed is less than or equal to the preset speed threshold, and the steering wheel angle of the vehicle is greater than the preset angle threshold, it means that the vehicle is not only in a low-speed scenario, but also has a higher probability of pulling over or parking sideways. At this time, the probability of scratching an obstacle on the road is greatly increased, then S101 is executed to enable the distance detection between the vehicle and the obstacle in the required scenario to reduce the computing power.

[0112] As a possible implementation method, with the rapid development and innovation of vehicle industry technology, the degree of vehicle intelligence is increasing, and the on-board panoramic surround view imaging system is configured in all levels of vehicle models to improve consumers' perception of the road environment and reduce the accident rate in low-speed scenes. Among them, the on-board panoramic surround view imaging system includes a fisheye camera, which can collect fisheye images and determine the target fisheye image from the collected fisheye images. For details on the determination method, see S1011-S1014.

[0113] S1011: Obtain a fisheye image set and camera internal and external parameter calibration information.

[0114] The fisheye image set includes multiple fisheye images taken from different positions of the vehicle.

[0115] In actual applications, the front bumper, rear bumper, left mirror and right mirror of the vehicle are all equipped with fisheye cameras, which can capture multiple fisheye images. For example, each fisheye camera captures the front fisheye image, the rear fisheye image, the left fisheye image and the right fisheye image, among which the front fisheye image is the fisheye image captured by the fisheye camera installed on the front bumper, which is used to capture the environment in front of the vehicle.

[0116] S1012: Correcting the fisheye images included in the fisheye image set respectively according to the camera internal and external parameter calibration information to obtain a distortion-free top view image set including a plurality of distortion-free top view images.

[0117] Since the fisheye image captured by the fisheye camera is distorted, it is necessary to correct the captured fisheye image to obtain an undistorted top view image. For example, the fisheye image can be corrected by a pre-trained fisheye camera imaging model to obtain an undistorted top view. For another example, the fisheye image can be corrected by using a fisheye camera imaging model that belongs to a physical model, and the model parameters of the fisheye camera imaging model can be pre-calibrated at a calibration site.

[0118] See also Figure 3 , which is a schematic diagram of an undistorted top view provided in an embodiment of the present application. Figure 3 In the above figure, each fisheye camera collects the front fisheye image, the rear fisheye image, the left fisheye image and the right fisheye image respectively. The front fisheye image is the fisheye image collected by the fisheye camera installed on the front bumper, which is used to collect the environment in front of the vehicle, and so on. The front fisheye image is corrected to obtain the front undistorted top view image, and so on, the rear undistorted top view image, the left undistorted top view image and the right undistorted top view image can be obtained.

[0119] S1013: Identify overlapping areas between multiple undistorted overhead images to obtain a recognition result.

[0120] The recognition result is used to identify whether the overlapping area between two undistorted overhead view images in the undistorted overhead view image set includes an object whose morphological difference is greater than a preset morphological threshold.

[0121] Since the undistorted top view image set includes multiple undistorted top view images taken from different positions of the vehicle, and since fisheye cameras are generally wide-angle cameras, there may be overlapping areas between the two undistorted top view images. Figure 3 , there is an overlapping area between the undistorted top view image on the front side and the undistorted top view image on the right side, that is, Figure 3 In the same way, Figure 3 There is an overlapping area between ④ and ⑤. Figure 3 There is an overlapping area between ⑥ and ⑦. Figure 3 There is an overlapping area between ⑧ and ①.

[0122] Obstacles are non-ground objects. In reality, obstacles will protrude or sink from the road. Due to the correction principle of fisheye cameras and the different installation angles and locations of different fisheye cameras, the same obstacle will be projected onto the ground in different shapes. The closer the obstacle is to the vehicle body, the greater the difference. However, there will be no projection problems for planes without height, such as roads and signs. Therefore, the recognition result can be obtained based on the overlapping areas between multiple distortion-free overhead images.

[0123] As a possible implementation, two undistorted overhead images including overlapping areas in the undistorted overhead image set are taken as an example. For the convenience of explanation, the two undistorted overhead images including overlapping areas are referred to as the first undistorted overhead image and the second undistorted overhead image. For details, see S10131-S10133:

[0124] S10131: Obtain a first binary contour map corresponding to the first undistorted top view image and a second binary contour map corresponding to the second undistorted top view image.

[0125] As a possible implementation manner, edge detection may be performed on the first distortion-free top view image to obtain a first binary contour map, and edge detection may be performed on the second distortion-free top view image to obtain a second binary contour map.

[0126] See also Figure 4 , which is a schematic diagram of an overlapping area including lane lines provided in an embodiment of the present application. Figure 4 In the figure, the first image in the first row is an overlapping area included in the undistorted top view image on the right (the first undistorted top view image), such as Figure 3 ③ in the figure; the first image in the second row is an overlapping area included in the front undistorted top view image (the second undistorted top view image), such as Figure 3 ② in the figure. After edge detection, the two images were respectively given the first binary contour map ( Figure 4 The second image in the first row) and the second binary contour map ( Figure 4 (second image in the second row).

[0127] Similarly, see Figure 5 , which is a schematic diagram of an overlapping area including a shoulder provided in an embodiment of the present application. Figure 5 In the figure, the first image in the first row is an overlapping area included in the undistorted top view image on the right (the first undistorted top view image), such as Figure 3 ③ in the figure; the first image in the second row is an overlapping area included in the front undistorted top view image (the second undistorted top view image), such as Figure 3② in the figure. After edge detection, the two images were respectively given the first binary contour map ( Figure 5 The second image in the first row) and the second binary contour map ( Figure 5 (second image in the second row).

[0128] S10132: compressing the first binary contour image to obtain a first one-dimensional identity code, and compressing the second binary contour image to obtain a second one-dimensional identity code.

[0129] As a possible implementation method, the compression process can refer to formula (1) to formula (3):

[0130]

[0131]

[0132]

[0133] Among them, K m×1 is a compressed convolution kernel with m rows and 1 column; C 1×n A is a 1-row n-column convolution graph; m×n is a binary contour image with m rows and n columns; dst is the identity code to be calculated; dst (1,j) is the identity value of the j-th column position in the one-dimensional identity code.

[0134] Since the one-dimensional identity code is difficult to observe with the naked eye, Figure 4 and Figure 5 The one-dimensional identity code corresponding to the binary contour image in the figure is visualized as a two-dimensional identity code, which is the third image in each row of the figure, such as Figure 4 The third image in the first row is the two-dimensional identity code corresponding to the first binary contour image. It can be understood that in practical applications, it is not necessary to visualize the one-dimensional identity code as a two-dimensional identity code.

[0135] S10133: Determine recognition results for the first undistorted top view image and the second undistorted top view image according to the first one-dimensional identity code and the second one-dimensional identity code.

[0136] To determine whether there is an obstacle in the overlapping area, the two one-dimensional identity codes can be compared to see if they are similar. If they are similar, it is determined that there is no obstacle; if they are not similar, it is determined that there is an obstacle.

[0137] As a possible implementation method, the recognition result can be obtained based on the Hamming distance value obtained by the fixed window, as shown in formula (4):

[0138]

[0139] Where H is the Hamming distance value; n is the length of the one-dimensional identity code, i is the position of the point whose value is greater than 0 in the first one-dimensional identity code; the [·] symbol indicates a binary judgment, and the result of the bracket operation is 1 when the condition in the bracket is true, and the result of the bracket operation is 0 when the condition in the bracket is false; is the identity value of position j in the second one-dimensional identity code corresponding to position i in the first one-dimensional identity code; k is the fixed window size.

[0140] Therefore, the recognition result obtained based on the Hamming distance value obtained by the fixed window increases the tolerance range, so that the obtained recognition result is more accurate and will not obtain an erroneous recognition result due to a slight deviation.

[0141] Continue to see Figure 4 , the Hamming distance value in box 1 and box 2 is 0, that is, the one-dimensional identity codes corresponding to the two-dimensional identity codes are similar, and there are no objects (obstacles) whose morphological differences are greater than the preset morphological threshold. Figure 5 If the Hamming distance value of box 1 is 0 and the Hamming distance value of box 2 is 3, it is considered that the one-dimensional identity code corresponding to the two-dimensional identity code in box 1 is similar, and there is no object (obstacle) whose morphological difference is greater than the preset morphological threshold, and the one-dimensional identity code corresponding to the two-dimensional identity code in box 2 is not similar, and there is an object (obstacle) whose morphological difference is greater than the preset morphological threshold. As a possible implementation method, a preset Hamming threshold can be set, such as 2.

[0142] S1014: If it is determined according to the recognition result that the target undistorted top view image includes an object whose morphological difference is greater than a preset morphological threshold, the target undistorted top view image is determined as a target fisheye image.

[0143] The target undistorted top view image is one of the two undistorted top view images including an object whose morphological difference is greater than a preset morphological threshold. As a possible implementation, the right undistorted top view image and the left undistorted top view image may be preferentially used as the target fisheye image because the vehicle is generally parked by the side of the road.

[0144] For example, if the overlapping area of ​​the front undistorted top view image and the overlapping area of ​​the right undistorted top view image include an object whose morphological difference is greater than a preset morphological threshold, the right undistorted top view image is determined as the target fisheye image;

[0145] If the overlapping area of ​​the front undistorted top view image and the overlapping area of ​​the left undistorted top view image include an object with a morphological difference greater than a preset morphological threshold, the left undistorted top view image is determined as the target fisheye image;

[0146] If the overlapping area of ​​the rear undistorted top view image and the overlapping area of ​​the right undistorted top view image include an object having a morphological difference greater than a preset morphological threshold, the right undistorted top view image is determined as the target fisheye image;

[0147] If the overlapping area of ​​the rear undistorted top view image and the overlapping area of ​​the left undistorted top view image include an object with a morphological difference greater than a preset morphological threshold, the left undistorted top view image is determined as the target fisheye image.

[0148] S102: Convert the target fisheye image to a hue, saturation, and brightness color space to obtain a saturation sub-image and a brightness sub-image for pixels included in the target fisheye image.

[0149] See also Figure 6 , which is a schematic diagram of converting a target fisheye image into a hue, saturation, and brightness color space provided by an embodiment of the present application. Figure 6 In the present invention, the target fisheye image is converted into the hue-saturation-value (HSV) color space, and three sub-images are obtained, namely, the hue (H) sub-image, the saturation (S) sub-image, and the value (V) sub-image.

[0150] S103: Determine a first non-driving area of ​​the vehicle on the road according to the saturation sub-image.

[0151] The first non-driving area is determined based on the saturation sub-image and is an area on the road where the vehicle cannot drive, such as Figure 2 The d sub-image in the figure shows the saturation sub-image. Figure 2 The white area shown in the sub-image e in FIG is the first non-driving area. The saturation sub-image can be used to segment road areas such as cement and asphalt with low saturation.

[0152] As a possible implementation, the pixels included in the target fisheye image can be respectively determined as target pixels; if the pixel value of the target pixel in the saturation sub-image is greater than or equal to the preset saturation threshold, the target pixel is added to the first non-driving area of ​​the vehicle on the road. If the pixel value of the target pixel in the saturation sub-image is less than the preset saturation threshold, the target pixel is added to the drivable area of ​​the vehicle on the road. See formula (5):

[0153]

[0154] Among them, S (i,j) is the pixel value of the pixel at position (i, j) in the target fisheye image in the saturation sub-image; P (i,j) is the pixel value of the road segmentation mask P at the pixel point at position (i, j) in the target fisheye image.

[0155] See also Figure 7, which is a schematic diagram of a first non-driving area provided in an embodiment of the present application. Figure 7 In the figure, the first picture on the left is the target fisheye image obtained by the edge protection filtering technology, and the second picture on the left is the road segmentation map obtained based on formula (5), where the black area is the drivable area and the white area is the first non-drivable area.

[0156] As a possible implementation method, due to the complex composition of the road, some shoulders may be made of cement, so the accuracy of the first non-driving area obtained by segmentation based only on formula (5) is low, such as the cement shoulder may be segmented out at the same time as other resolution areas. Based on this, the first one-dimensional identity code and the second one-dimensional identity code can be obtained, and the first one-dimensional identity code and the second one-dimensional identity code can be compared to obtain the inconsistency point, and the first non-driving area can be adjusted according to the inconsistency point to obtain the updated first non-driving area.

[0157] For example, based on formula (4), the first inconsistent point is obtained, and a vertical line is drawn from the first inconsistent point or its previous consistent point to obtain a dividing line, thereby obtaining an updated first non-driving area. Figure 7 , the third picture on the left is the updated first non-driving area obtained based on this method. For another example, all inconsistent points are found, and these inconsistent points are fitted to obtain the segmentation line, thereby obtaining the updated first non-driving area.

[0158] S104: Determine a second non-driving area of ​​the vehicle on the road according to the brightness sub-image.

[0159] The second non-driving area is determined based on the brightness sub-image, and is an area on the road where the vehicle cannot drive, such as Figure 2 The b sub-image in the figure shows the brightness sub-image. Figure 2 The white area shown in the c sub-image in FIG is the second non-driving area. The brightness sub-image is a lighting robust extractor that is less affected by lighting interference, so the outline of the obstacle can be identified.

[0160] As a possible implementation manner, image equalization and threshold segmentation are performed on the brightness sub-image of the pixels included in the target fisheye image to obtain a second non-driving area of ​​the vehicle on the road.

[0161] S105: Determine an optimal fitting line segment according to the first non-driving area and the second non-driving area.

[0162] The first non-driving area and the second non-driving area are non-driving areas for roads obtained in two different ways. Based on the intersection of the two, an edge image of an area without a road, i.e., a non-road edge image, can be determined, and then the best fitting line segment can be obtained. The best fitting line segment is used to describe the edge of the non-road area for the road.

[0163] Continue to see Figure 2 , sub-image f includes the intersection of the first non-driving area and the second non-driving area, wherein the white area is an edge image without a road area, i.e., a non-road edge image. The white line shown in sub-image g is the optimal fitting line segment.

[0164] As a possible implementation method, the optimal fitting line segment can be obtained by using the Hough probability line detection technology. For details, see S1051-S1054:

[0165] S1051: Obtain a non-road edge image according to the first non-driving area and the second non-driving area.

[0166] S1052: Traverse the non-road edge image from left to right, select the first n non-zero pixels in each row, and obtain the obstacle contour.

[0167] As a possible implementation, since the non-road edge image contains a lot of edge information, in order to eliminate edge interference, if the target fisheye image is the right fisheye image, the non-road edge image is traversed from left to right; if the target fisheye image is the left fisheye image, the non-road edge image is traversed from right to left. As a possible implementation, it can also be traversed from top to bottom or from bottom to top. For example, if the target fisheye image is the right fisheye image, the non-road edge image can be traversed from top to bottom and from left to right; if the target fisheye image is the left fisheye image, the non-road edge image can be traversed from top to bottom and from right to left. Select the first m non-zero pixels in each row, where m is an integer greater than 1, to obtain the obstacle contour.

[0168] S1053: Perform Hough probability line detection on the obstacle contour to obtain n candidate fitting line segments.

[0169] S1054: Determine the best fitting line segment from the n candidate fitting line segments according to preset conditions.

[0170] Among them, the preset conditions are one or a combination of multiple conditions: the segment length of the candidate fitting segment, the distance between the midpoint of the candidate fitting segment and the vehicle, the number of fitting points in the candidate fitting segment, and the variance distance between the segment angle of the candidate fitting segment and the average angle of the candidate fitting segment.

[0171] The four conditions can be used as influencing factors for determining the best fitting line segment at the same time, or one or more of them can be selected as influencing factors for determining the best fitting line segment. Among them, the longer the line segment length, the higher the probability that the line segment is the contour line of the obstacle, because the obstacle is a non-ground object with continuous features such as the shoulder. The smaller the distance between the midpoint of the candidate fitting line segment and the vehicle, the higher the probability that the line segment is the contour line of the obstacle. The more fitting points there are in the candidate fitting line segment, the higher the probability that the line segment is the contour line of the obstacle. The larger the variance distance between the line segment angle of the candidate fitting line segment and the average angle of the candidate fitting line segment, the higher the probability that the line segment is the contour line of the obstacle.

[0172] Taking the four conditions as the influencing factors for determining the optimal fitting line segment as an example, it can be expressed as formula (6):

[0173]

[0174] Where n is the number of candidate fitting line segments; i is the i-th candidate fitting line segment; f(i) is the score of the i-th candidate fitting line segment; l i is the length of the i-th candidate fitting line segment; p i is the distance between the midpoint of the i-th candidate fitting line segment and the vehicle; k i is the number of fitting points in the i-th candidate fitting line segment; d i is the line segment angle of the i-th candidate fitting line segment. The candidate fitting line segment with the highest score can be selected as the optimal fitting line segment.

[0175] S106: Determine the distance between the vehicle and the obstacle according to the optimal fitting line segment.

[0176] According to the best fitting line segment, the distance between the vehicle and the obstacle is determined, and the obstacle is located. Figure 2 , sub-figure h shows a schematic diagram of shoulder detection and positioning.

[0177] If the distance between the vehicle and the obstacle is the measured distance between the vehicle and the obstacle, that is, the distance value measured by S101-S106 is directly used as the distance between the vehicle and the obstacle, and the distance between the vehicle and the obstacle is obtained based on method 1 (A1-A4). If the distance between the vehicle and the obstacle is the actual distance adjusted based on the measured distance between the vehicle and the obstacle, the distance between the vehicle and the obstacle is obtained based on method 2 (B1-B3). The following are respectively explained.

[0178] Method 1:

[0179] A1: Get the vehicle's gear position and the conversion ratio between image coordinates and world coordinates.

[0180] A2: If the vehicle is in forward gear, the distance between the vehicle and the optimal fitting line segment is determined based on the front wheel of the vehicle corresponding to the target fisheye image.

[0181] A3: If the vehicle is in reverse gear, the distance between the vehicle and the optimal fitting line segment is determined based on the rear wheel of the vehicle corresponding to the target fisheye image.

[0182] If the target fisheye image is a right fisheye image, the distance between the vehicle and the best fitting line segment is determined based on the right front wheel of the vehicle. If the target fisheye image is a left fisheye image, the distance between the vehicle and the best fitting line segment is determined based on the left front wheel of the vehicle.

[0183] See also Figure 8 , which is a schematic diagram of determining the distance between a vehicle and an optimal fitting line segment provided by an embodiment of the present application. Figure 8 In the figure, a plane rectangular coordinate system is established with the center of the vehicle as the origin. D1 is the distance between the vehicle and the optimal fitting line segment based on the right front wheel of the vehicle, and D2 is the distance between the vehicle and the optimal fitting line segment based on the rear wheel of the rear side of the vehicle.

[0184] A4: Determine the measured distance between the vehicle and the obstacle based on the conversion ratio between the image coordinates and the world coordinates and the distance between the vehicle and the best fitting line segment.

[0185] As a possible implementation method, according to the conversion ratio between the image coordinates and the world coordinates and the distance between the vehicle and the optimal fitting line segment, the measured distance between the vehicle and the obstacle can be expressed as formula (7):

[0186] distance=D×ratio (7)

[0187] Wherein, distance represents the measured distance between the vehicle and the obstacle, D is D1 or D2, and ratio is the conversion ratio between image coordinates and world coordinates.

[0188] Method 2:

[0189] B1: Get the predicted distance at time k-1, the vehicle speed at time k, and the tire steering angle at time k.

[0190] The predicted distance at time k-1 is the predicted distance at time k-1, the vehicle speed at time k is the vehicle speed at time k, and the tire steering angle at time k is the tire steering angle at time k.

[0191] B2: Determine the predicted distance at time k based on the actual distance at time k-1, the vehicle speed at time k, and the tire steering angle at time k.

[0192] As a possible implementation method, according to the actual distance at time k-1, the vehicle speed at time k and the tire steering angle at time k, the predicted distance at time k can be expressed as formula (8):

[0193]

[0194] in, is the predicted distance at time k, that is, the predicted distance at time k; X k-1 is the actual distance at time k-1; α is the tire steering angle at time k; V k is the vehicle speed at time k; dt is the time interval for state update.

[0195] B3: Determine the actual distance at time k based on the predicted distance at time k, the measured distance at time k, and the gain at time k.

[0196] As a possible implementation method, according to the predicted distance at time k, the measured distance at time k and the gain at time k, the actual distance at time k can be expressed as formula (9):

[0197]

[0198] Among them, X k is the actual distance at time k, that is, the actual distance at time k; is the predicted distance at time k; H k is the gain at time k; Z k is the distance measured at time k, that is, the distance value obtained based on the above formula (7).

[0199] As a possible implementation method, the k-time gain is the k-time Kalman filter gain, which can be expressed as formula (10)-formula (12):

[0200]

[0201]

[0202]

[0203] Among them, H k is the gain at time k; is the system prediction error at time k; R is the distance detection noise between the vehicle and the obstacle; P k-1 is the error state at time k-1; Q is the error noise; P k is the error state at time k.

[0204] As a possible implementation method, after the obstacle is detected, the obstacle can be drawn in the panoramic 2D bird's-eye view image, such as Fig. 9 shown.

[0205] It can be seen from the above technical solution that a target fisheye image including obstacles is obtained, and the target fisheye image is an image used to describe the surrounding environment of the vehicle, and the obstacles included therein are non-ground objects with continuous features. The target fisheye image is converted into a hue, saturation, and brightness color space to obtain a saturation sub-image and a brightness sub-image, and the first non-driving area and the second non-driving area of ​​the vehicle on the road are determined respectively according to the saturation sub-image and the brightness sub-image. The first non-driving area and the second non-driving area are areas in the target fisheye image that the vehicle cannot drive in two ways, or in other words, areas where obstacles on the road are located. According to the first non-driving area and the second non-driving area, the best fitting line segment determined is more accurate in describing the edge of the non-road area on the road. Therefore, based on the more accurate best fitting line segment, the distance between the vehicle and the obstacle is determined. As a result, it is no longer necessary to obtain point cloud data with a three-dimensional imaging device, but to obtain the target fisheye image based on a low-cost device, and determine the distance between the vehicle and the obstacle based on the target fisheye image, thereby reducing the cost of vehicle identification and obstacle positioning and expanding the scope of application scenarios.

[0206] In addition to the method for detecting the distance between a vehicle and an obstacle provided in the embodiment of the present application, a device for detecting the distance between a vehicle and an obstacle is also provided. Fig.10 As shown, it includes: an acquisition unit 1001, a conversion unit 1002 and a determination unit 1003;

[0207] The acquisition unit 1001 is used to acquire a target fisheye image including an obstacle, wherein the target fisheye image is an image used to describe the surrounding environment of the vehicle, and the obstacle is a non-ground object with continuous features;

[0208] The conversion unit 1002 is used to convert the target fisheye image into a hue, saturation and brightness color space to obtain a saturation sub-image and a brightness sub-image for the pixel points included in the target fisheye image;

[0209] The determining unit 1003 is used to determine a first non-driving area of ​​the vehicle on the road according to the saturation sub-image;

[0210] The determining unit 1003 is further configured to determine a second non-driving area of ​​the vehicle on the road according to the brightness sub-image;

[0211] The determining unit 1003 is further configured to determine a best-fit line segment according to the first non-driving area and the second non-driving area, wherein the best-fit line segment is used to describe an edge of the non-road area of ​​the road;

[0212] The determining unit 1003 is further configured to determine the distance between the vehicle and the obstacle according to the optimal fitting line segment.

[0213] As a possible implementation manner, the obtaining unit 1001 is specifically configured to:

[0214] Acquire a fisheye image set and camera internal and external parameter calibration information, wherein the fisheye image set includes a plurality of fisheye images taken from different positions of the vehicle;

[0215] Correcting the fisheye images included in the fisheye image set according to the camera internal and external parameter calibration information to obtain a distortion-free top view image set including a plurality of distortion-free top view images;

[0216] Identify the overlapping area between the plurality of the distortion-free overhead images to obtain an identification result, wherein the identification result is used to identify whether the overlapping area between two of the distortion-free overhead images in the set of distortion-free overhead images includes an object whose morphological difference is greater than a preset morphological threshold;

[0217] If it is determined according to the recognition result that the target undistorted overhead image includes an object whose morphological difference is greater than a preset morphological threshold, the target undistorted overhead image is determined as the target fisheye image, and the target undistorted overhead image is one of the two undistorted overhead images that include an object whose morphological difference is greater than the preset morphological threshold.

[0218] As a possible implementation manner, the obtaining unit 1001 is specifically configured to:

[0219] Acquire a first binary contour image corresponding to a first distortion-free top view image and a second binary contour image corresponding to a second distortion-free top view image, wherein the first distortion-free top view image and the second distortion-free top view image are two distortion-free top view images including an overlapping area;

[0220] compressing the first binary contour image to obtain a first one-dimensional identity code, and compressing the second binary contour image to obtain a second one-dimensional identity code;

[0221] According to the first one-dimensional identity code and the second one-dimensional identity code, identification results for the first undistorted top view image and the second undistorted top view image are determined.

[0222] As a possible implementation manner, the determining unit 1003 is specifically configured to:

[0223] Determine the pixel points included in the target fisheye image as target pixel points respectively;

[0224] If the pixel value of the target pixel in the saturation sub-image is greater than or equal to a preset saturation threshold, the target pixel is added to a first non-driving area of ​​the vehicle on the road.

[0225] As a possible implementation manner, the device further includes an updating unit, configured to:

[0226] Acquire a first one-dimensional identity code and a second one-dimensional identity code, wherein the first one-dimensional identity code and the second one-dimensional identity code are determined based on two undistorted top-view images including an overlapping area;

[0227] comparing the first one-dimensional identity code and the second one-dimensional identity code to obtain inconsistency points;

[0228] The first non-driving area is adjusted according to the inconsistent points to obtain an updated first non-driving area.

[0229] As a possible implementation manner, the determining unit 1003 is specifically configured to:

[0230] Image equalization and threshold segmentation are performed on the brightness sub-image of the pixel points included in the target fisheye image to obtain a second non-driving area of ​​the vehicle on the road.

[0231] As a possible implementation manner, the determining unit 1003 is specifically configured to:

[0232] Obtaining a non-road edge image according to the first non-driving area and the second non-driving area;

[0233] Traversing the non-road edge image, selecting the first m non-zero pixels in each row to obtain the obstacle contour;

[0234] Performing Hough probability straight line detection on the obstacle contour to obtain n candidate fitting line segments;

[0235] The optimal fitting line segment is determined from the n candidate fitting line segments according to the segment length of the candidate fitting line segment, the distance between the midpoint of the candidate fitting line segment and the vehicle, the number of fitting points in the candidate fitting line segment and / or the variance distance between the segment angle of the candidate fitting line segment and the average angle of the candidate fitting line segment.

[0236] As a possible implementation manner, if the distance between the vehicle and the obstacle is a measured distance between the vehicle and the obstacle, the determining unit 1003 is specifically configured to:

[0237] Obtaining the gear position of the vehicle and the conversion ratio between image coordinates and world coordinates;

[0238] If the gear position of the vehicle is the forward gear, determining the distance between the vehicle and the optimal fitting line segment based on the front wheel of the vehicle corresponding to the target fisheye image;

[0239] If the gear position of the vehicle is the reverse gear, the distance between the vehicle and the optimal fitting line segment is determined based on the rear wheel of the vehicle corresponding to the target fisheye image;

[0240] The measured distance between the vehicle and the obstacle is determined according to the conversion ratio between the image coordinates and the world coordinates and the distance between the vehicle and the best fitting line segment.

[0241] As a possible implementation manner, if the distance between the vehicle and the obstacle is an actual distance adjusted based on the measured distance between the vehicle and the obstacle, the determining unit 1003 is specifically used to determine the actual distance at time k, and the actual distance at time k is determined as follows:

[0242] Get the predicted distance at time k-1, the vehicle speed at time k, and the tire steering angle at time k;

[0243] Determine the predicted distance at time k according to the actual distance at time k-1, the vehicle speed at time k and the tire steering angle at time k;

[0244] The actual distance at time k is determined according to the predicted distance at time k, the measured distance at time k and the gain at time k, wherein the gain at time k is determined according to the system prediction error at time k.

[0245] It can be seen from the above technical solution that a target fisheye image including obstacles is obtained, and the target fisheye image is an image used to describe the surrounding environment of the vehicle, and the obstacles included therein are non-ground objects with continuous features. The target fisheye image is converted into a hue, saturation, and brightness color space to obtain a saturation sub-image and a brightness sub-image, and the first non-driving area and the second non-driving area of ​​the vehicle on the road are determined respectively according to the saturation sub-image and the brightness sub-image. The first non-driving area and the second non-driving area are areas in the target fisheye image that the vehicle cannot drive in two ways, or in other words, areas where obstacles on the road are located. According to the first non-driving area and the second non-driving area, the best fitting line segment determined is more accurate in describing the edge of the non-road area on the road. Therefore, based on the more accurate best fitting line segment, the distance between the vehicle and the obstacle is determined. As a result, it is no longer necessary to obtain point cloud data with a three-dimensional imaging device, but to obtain the target fisheye image based on a low-cost device, and determine the distance between the vehicle and the obstacle based on the target fisheye image, thereby reducing the cost of vehicle identification and obstacle positioning and expanding the scope of application scenarios.

[0246] The present application also provides a computer device, see Fig.11 , which shows a structural diagram of a computer device provided in an embodiment of the present application, such as Fig.11 As shown, the device includes a memory 1110 and a processor 1120:

[0247] The memory 1110 is used to store program codes and transmit the program codes to the processor;

[0248] The processor 1120 is used to execute any one of the distance detection methods between a vehicle and an obstacle provided in the above embodiments according to the instructions in the program code.

[0249] An embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store a computer program, wherein the computer program is used to execute any one of the methods for detecting the distance between a vehicle and an obstacle provided in the above embodiments.

[0250] The embodiment of the present application also provides a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the distance detection method between the vehicle and the obstacle provided in various optional implementations of the above aspects.

[0251] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the various embodiments can be referred to each other. For the system or device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0252] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0253] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0254] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0255] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for detecting the distance between a vehicle and an obstacle, characterized in that: The method comprises: Acquire a target fisheye image including an obstacle, wherein the target fisheye image is an image used to describe the surrounding environment of the vehicle, and the obstacle is a non-ground object with continuous features; Convert the target fisheye image to a hue, saturation, and brightness color space to obtain a saturation sub-image and a brightness sub-image for the pixel points included in the target fisheye image; determining a first non-driving area of ​​the vehicle on the road according to the saturation sub-image; determining a second non-driving area of ​​the vehicle on the road according to the brightness sub-image; determining a best-fit line segment according to the first non-driving area and the second non-driving area, wherein the best-fit line segment is used to describe an edge of the non-road area of ​​the road; The distance between the vehicle and the obstacle is determined according to the best fitting line segment.

2. The method according to claim 1, characterized in that The step of acquiring a target fisheye image including obstacles comprises: Acquire a fisheye image set and camera internal and external parameter calibration information, wherein the fisheye image set includes a plurality of fisheye images taken from different positions of the vehicle; Correcting the fisheye images included in the fisheye image set according to the camera internal and external parameter calibration information to obtain a distortion-free top view image set including a plurality of distortion-free top view images; Identify the overlapping area between the plurality of the distortion-free overhead images to obtain an identification result, wherein the identification result is used to identify whether the overlapping area between two of the distortion-free overhead images in the set of distortion-free overhead images includes an object whose morphological difference is greater than a preset morphological threshold; If it is determined according to the recognition result that the target undistorted overhead image includes an object whose morphological difference is greater than a preset morphological threshold, the target undistorted overhead image is determined as the target fisheye image, and the target undistorted overhead image is one of the two undistorted overhead images that include an object whose morphological difference is greater than the preset morphological threshold.

3. The method according to claim 2, characterized in that The step of identifying the overlapping area between the plurality of the distortion-free overhead images to obtain the identification result includes: Acquire a first binary contour image corresponding to a first distortion-free top view image and a second binary contour image corresponding to a second distortion-free top view image, wherein the first distortion-free top view image and the second distortion-free top view image are two distortion-free top view images including an overlapping area; compressing the first binary contour image to obtain a first one-dimensional identity code, and compressing the second binary contour image to obtain a second one-dimensional identity code; According to the first one-dimensional identity code and the second one-dimensional identity code, identification results for the first undistorted top view image and the second undistorted top view image are determined.

4. The method according to claim 1, characterized in that: The determining, according to the saturation sub-image, a first non-driving area of ​​the vehicle on the road comprises: Determine the pixel points included in the target fisheye image as target pixel points respectively; If the pixel value of the target pixel in the saturation sub-image is greater than or equal to a preset saturation threshold, the target pixel is added to a first non-driving area of ​​the vehicle on the road.

5. The method according to claim 4, characterized in that The method further comprises: Acquire a first one-dimensional identity code and a second one-dimensional identity code, wherein the first one-dimensional identity code and the second one-dimensional identity code are determined based on two undistorted top-view images including an overlapping area; comparing the first one-dimensional identity code and the second one-dimensional identity code to obtain inconsistencies; The first non-driving area is adjusted according to the inconsistent points to obtain an updated first non-driving area.

6. The method according to claim 1, characterized in that The determining, according to the brightness sub-image, a second non-driving area of ​​the vehicle on the road comprises: Image equalization and threshold segmentation are performed on the brightness sub-image of the pixel points included in the target fisheye image to obtain a second non-driving area of ​​the vehicle on the road.

7. The method according to claim 1, characterized in that The determining, according to the first non-driving area and the second non-driving area, the best fitting line segment for identifying the obstacle comprises: Obtaining a non-road edge image according to the first non-driving area and the second non-driving area; Traversing the non-road edge image, selecting the first m non-zero pixels in each row to obtain the obstacle contour; Performing Hough probability straight line detection on the obstacle contour to obtain n candidate fitting line segments; According to preset conditions, the optimal fitting line segment is determined from the n candidate fitting line segments, wherein the preset conditions are one or a combination of multiple conditions: the segment length of the candidate fitting line segment, the distance between the midpoint of the candidate fitting line segment and the vehicle, the number of fitting points in the candidate fitting line segment, and the variance distance between the segment angle of the candidate fitting line segment and the average angle of the candidate fitting line segment.

8. The method according to claim 1, characterized in that If the distance between the vehicle and the obstacle is a measured distance between the vehicle and the obstacle, determining the distance between the vehicle and the obstacle according to the best fitting line segment includes: Obtaining the gear position of the vehicle and the conversion ratio between image coordinates and world coordinates; If the gear position of the vehicle is the forward gear, determining the distance between the vehicle and the optimal fitting line segment based on the front wheel of the vehicle corresponding to the target fisheye image; If the gear position of the vehicle is the reverse gear, the distance between the vehicle and the optimal fitting line segment is determined based on the rear wheel of the vehicle corresponding to the target fisheye image; The measured distance between the vehicle and the obstacle is determined according to the conversion ratio between the image coordinates and the world coordinates and the distance between the vehicle and the best fitting line segment.

9. The method according to claim 8, characterized in that If the distance between the vehicle and the obstacle is an actual distance adjusted based on the measured distance between the vehicle and the obstacle, the actual distance at time k is determined as follows: Get the predicted distance at time k-1, the vehicle speed at time k, and the tire steering angle at time k; Determine the predicted distance at time k according to the actual distance at time k-1, the vehicle speed at time k and the tire steering angle at time k; The actual distance at time k is determined according to the predicted distance at time k, the measured distance at time k and the gain at time k, wherein the gain at time k is determined according to the system prediction error at time k.

10. A device for detecting the distance between a vehicle and an obstacle, characterized in that: The method comprises: an acquisition unit, a conversion unit and a determination unit; The acquisition unit is used to acquire a target fisheye image including an obstacle, wherein the target fisheye image is an image used to describe the surrounding environment of the vehicle, and the obstacle is a non-ground object with continuous features; The conversion unit is used to convert the target fisheye image into a hue, saturation and brightness color space to obtain a saturation sub-image and a brightness sub-image for the pixel points included in the target fisheye image; The determining unit is used to determine a first non-driving area of ​​the vehicle on the road according to the saturation sub-image; The determining unit is further used to determine a second non-driving area of ​​the vehicle on the road according to the brightness sub-image; The determining unit is further used to determine a best-fit line segment according to the first non-driving area and the second non-driving area, wherein the best-fit line segment is used to describe an edge of the non-road area of ​​the road; The determining unit is further configured to determine the distance between the vehicle and the obstacle according to the optimal fitting line segment.

Images