Obstacle detection method of vehicle, vehicle-mounted system and readable storage medium

Abstract

The invention discloses a vehicle obstacle detection method, a vehicle-mounted system and a readable storage medium. The method comprises the following steps: acquiring obstacle features in a first image and a second image, and converting the obstacle features in the second image into the first image to obtain a first obstacle feature set; obtaining a plurality of Bev lane lines and a plurality of 2D lane lines, and matching a corresponding Bev lane line for each 2D lane line; performing inverse perspective transformation on the first obstacle features in the first obstacle feature set to obtain second obstacle features under the vehicle body coordinate system; determining corresponding 2D lane line key points according to the plurality of 2D lane lines; determining corresponding Bev lane line key points according to the plurality of Bev lane lines; according to the 2D lane line key point, the Bev lane line key point, the first obstacle feature and the second obstacle feature, the actual position of the obstacle in the vehicle body coordinate system is obtained. Through the above mode, the positioning accuracy of the obstacle is improved.

Description

Technical Field

[0001] The present application relates to the technical field of obstacle detection, and in particular to a vehicle obstacle detection method, a vehicle-mounted system, and a readable storage medium. Background Art

[0002] Positioning static obstacles in the vehicle body coordinate system is a necessary module for high-level intelligent driving assistance systems. The positioning results of static obstacles and their relative position to the lane line will directly affect the system's decision-making and deployment of the vehicle, and are crucial to the navigation assistance system and lane centering control functions. Incorrect positioning results of static obstacles will increase the risk of misbraking and collision of the vehicle.

[0003] How to accurately perform 3D positioning is the key to obstacle ranging technology. Summary of the invention

[0004] The present application provides a vehicle obstacle detection method, a vehicle-mounted system, and a readable storage medium, which can improve the accuracy of locating obstacles and thereby improve the accuracy of distance measurement between the vehicle and the obstacle.

[0005] In a first aspect, the present application provides a vehicle obstacle detection method, the method comprising: acquiring obstacle features in a first image and a second image, and converting the obstacle features in the second image into the first image to obtain a first obstacle feature set; wherein the first image and the second image are images acquired at different focal lengths at the same time and the same viewing angle; the focal length corresponding to the second image is greater than the focal length corresponding to the first image; acquiring a plurality of Bev lane lines and a plurality of 2D lane lines, and matching a corresponding Bev lane line for each 2D lane line to obtain a matching relationship; performing an inverse perspective transformation on a first obstacle feature in the first obstacle feature set to obtain a second obstacle feature in a vehicle body coordinate system, and obtaining a second obstacle feature set composed of the second obstacle features; determining a 2D lane line key point corresponding to each first obstacle feature according to the plurality of 2D lane lines; and determining a Bev lane line key point corresponding to each second obstacle feature according to the plurality of Bev lane lines; obtaining an actual position of an obstacle corresponding to the first obstacle feature in the vehicle body coordinate system according to the matching relationship, the 2D lane line key point, the Bev lane line key point, the first obstacle feature, and the second obstacle feature.

[0006] Among them, obtaining a number of Bev lane lines and a number of 2D lane lines includes: inputting the first image into a 2D segmentation model to obtain a number of 2D lane lines; inputting the first image, the second image and the remaining perspective images into a Bev detection model to obtain a number of Bev lane lines.

[0007] Among them, determining the 2D lane line key point corresponding to each first obstacle feature based on a plurality of 2D lane lines includes: determining two target 2D lane lines corresponding to each first obstacle feature based on a plurality of 2D lane lines; and finding the 2D lane line key point corresponding to the corresponding first obstacle feature in the two target 2D lane lines.

[0008] Among them, determining the Bev lane line key point corresponding to each second obstacle feature based on a number of Bev lane lines includes: determining two target Bev lane lines corresponding to each second obstacle feature based on a number of Bev lane lines; and finding the Bev lane line key point corresponding to the corresponding second obstacle feature in the two target Bev lane lines.

[0009] After obtaining the second obstacle feature set composed of the second obstacle features, the method further includes: using a plurality of Bev lane lines to constrain the second obstacle features in the second obstacle feature set; and removing the constrained second obstacle features from the second obstacle feature set.

[0010] The step of converting obstacle features in the second image into the first image to obtain a first obstacle feature set includes: converting obstacle features in the second image into the first image using a homography matrix to obtain the first obstacle feature set.

[0011] Among them, each 2D lane line is matched with the corresponding Bev lane line to obtain a matching relationship, including: obtaining the initial point in each 2D lane line; performing an inverse perspective transformation on the initial point to obtain a target point in the vehicle body coordinate system; finding a target Bev lane line corresponding to the target point from a number of Bev lane lines, and establishing a matching relationship between the 2D lane line corresponding to the target point and the target Bev lane line.

[0012] Among them, according to the matching relationship, the 2D lane line key point, the Bev lane line key point, the first obstacle feature and the second obstacle feature, the actual position of the obstacle corresponding to the first obstacle feature in the vehicle body coordinate system is obtained, including: taking the ordinate of the Bev lane line key point as the ordinate of the obstacle; calculating the ratio between the abscissa difference of the Bev lane line key point and the abscissa difference of the 2D lane line key point, and the difference between the abscissa of the 2D lane line key point and the abscissa of the first obstacle feature; and obtaining the abscissa and ordinate of the obstacle according to the ratio, the difference, the Bev lane line key point and the 2D lane line key point.

[0013] In a second aspect, the present application provides a vehicle-mounted system, which includes a memory and a processor coupled to the memory, the memory storing at least one computer program, and when the at least one computer program is loaded and executed by the processor, it is used to implement the method provided in the first aspect.

[0014] In a third aspect, the present application provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are executed by a processor, they are used to implement the method provided in the first aspect.

[0015] The beneficial effects of the present application are as follows: different from the prior art, the vehicle obstacle detection method, vehicle-mounted system, and readable storage medium provided by the present application obtain obstacle features in a first image and a second image, and convert the obstacle features in the second image into the first image to obtain a first obstacle feature set; wherein the first image and the second image are images acquired at different focal lengths at the same time and the same viewing angle; the focal length corresponding to the second image is greater than the focal length corresponding to the first image; a plurality of Bev lane lines and a plurality of 2D lane lines are obtained, and each 2D lane line is matched with a corresponding Bev lane line to obtain a matching relationship; for the first The first obstacle feature in the obstacle feature set is subjected to inverse perspective transformation to obtain the second obstacle feature in the vehicle body coordinate system, and the second obstacle feature set composed of the second obstacle features is obtained; the 2D lane line key points corresponding to each first obstacle feature are determined according to a number of 2D lane lines; and the Bev lane line key points corresponding to each second obstacle feature are determined according to a number of Bev lane lines; the actual position of the obstacle corresponding to the first obstacle feature in the vehicle body coordinate system is obtained according to the matching relationship, the 2D lane line key points, the Bev lane line key points, the first obstacle feature and the second obstacle feature. By using the stable 2D lane line and the Bev lane line in the vehicle body coordinate system, according to the relative position relationship between the lane line and the obstacle, the actual position of the obstacle in the vehicle body coordinate system is stably output, the positioning accuracy of the obstacle is improved, and then the accuracy of the distance measurement between the vehicle and the obstacle is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:

[0017] Figure 1 It is a flow chart of an embodiment of a vehicle obstacle detection method provided by the present application;

[0018] Figure 2 It is a flowchart of an embodiment of establishing a matching relationship between Bev lane lines and 2D lane lines provided by the present application;

[0019] Figure 3 is a flow chart of another embodiment of the vehicle obstacle detection method provided by the present application;

[0020] Figure 4 This is the Bev lane line diagram provided by this application;

[0021] Figure 5 It is a 2D lane line schematic diagram provided by this application;

[0022] Figure 6 is a flow chart of another embodiment of the vehicle obstacle detection method provided by the present application;

[0023] Figure 7 is a flow chart of another embodiment of the vehicle obstacle detection method provided by the present application;

[0024] Figure 8 is a flow chart of another embodiment of the vehicle obstacle detection method provided by the present application;

[0025] Fig. 9 It is a structural schematic diagram of an embodiment of a vehicle-mounted system provided by the present application;

[0026] Fig.10 It is a structural schematic diagram of an embodiment of a computer-readable storage medium provided by the present application. DETAILED DESCRIPTION

[0027] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It will be understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for ease of description, only some but not all structures related to the present application are shown in the drawings. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0028] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0029] Positioning static obstacles in the vehicle body coordinate system is a necessary module for high-level intelligent driving assistance systems. The positioning results of static obstacles and their relative position relationship with the lane line will directly affect the system's decision-making and deployment of the vehicle, and are crucial to the navigation assistance system and lane centering control functions. Incorrect positioning results of static obstacles will increase the risk of misbraking and collision of the vehicle. Among them, the vehicle coordinate system is a dynamic coordinate system used to describe the movement of the vehicle. The center of mass of the vehicle is often selected as the origin (or the midpoint of the rear axle of the vehicle), the X-axis is parallel to the ground and points to the front of the driver's field of view, the Z-axis points upward, and the Y-axis points to the driver's left hand side.

[0030] How to accurately perform 3D positioning is the key to obstacle ranging technology.

[0031] The obstacle distance measurement methods can be roughly divided into monocular vision-based methods and binocular vision-based methods. Among them, the monocular vision-based methods can be further divided into: imaging size-based methods, regression model-based methods and geometric relationship-based methods.

[0032] The binocular vision-based method combines the internal and external parameters of the binocular camera, uses binocular parallax, combines the principle of similar triangles and the imaging model to locate and measure obstacles.

[0033] Based on the imaging size method, an a priori assumption is made about the actual size of a part of the obstacle, and then the mapping relationship between the pixel size and the actual size is established in combination with the internal and external parameters of the camera to locate the 3D coordinates of the obstacle.

[0034] Based on the regression model method, a deep neural network model for obstacle ranging is trained according to a large amount of data.

[0035] Method based on geometric relationship derivation. The imaging model of the camera is used as the theoretical basis, combined with the principle of similar triangles and inverse perspective transformation, to derive the mapping relationship between the image coordinate system of the obstacle feature points and the vehicle body coordinate system. The method based on binocular vision requires binocular cameras to capture the target image, and the two cameras need to have exactly the same internal and external parameters, which is easily affected by the road environment and camera posture during vehicle driving.

[0036] The imaging size-based method assumes that the sizes of obstacles are the same. This method is not universal because crash barriers and cones, which are both static obstacles, are different. Even if they are the same cones, there are differences in size. Therefore, this method is not universal.

[0037] The regression model-based method requires additional computing power to train the regression model and needs to collect a large amount of training data. From the perspective of actual engineering applications, this method is currently not very realistic.

[0038] The method based on geometric relationship deduction requires accurate estimation of the road environment and the camera's posture angle. For example, when going up and down hills, on curves, or when the camera shakes, the lateral and longitudinal distance measurement of obstacles will become unstable.

[0039] Based on this, the present application proposes to obtain obstacle features in a first image and a second image, and convert the obstacle features in the second image into the first image to obtain a first obstacle feature set; wherein the first image and the second image are images acquired at different focal lengths at the same time and the same viewing angle; the focal length corresponding to the second image is greater than the focal length corresponding to the first image; obtain a plurality of Bev lane lines and a plurality of 2D lane lines, and match the corresponding Bev lane line for each 2D lane line to obtain a matching relationship; perform an inverse perspective transformation on the first obstacle feature in the first obstacle feature set to obtain a second obstacle feature in a vehicle body coordinate system, and obtain a second obstacle feature set composed of the second obstacle features; determine the 2D lane line key points corresponding to each first obstacle feature according to the plurality of 2D lane lines; and determine the Bev lane line key points corresponding to each second obstacle feature according to the plurality of Bev lane lines; obtain the actual position of the obstacle corresponding to the first obstacle feature in the vehicle body coordinate system according to the matching relationship, the 2D lane line key points, the Bev lane line key points, the first obstacle feature, and the second obstacle feature. By using the stable 2D lane lines and the Bev lane lines in the vehicle body coordinate system, the actual position of the obstacle in the vehicle body coordinate system is stably output according to the relative position relationship between the lane lines and the obstacles, thereby improving the accuracy of locating the obstacles and thus improving the accuracy of the distance measurement between the vehicle and the obstacles. For details, please refer to the following embodiments.

[0040] See also Figure 1 , Figure 1 1 is a flow chart of an embodiment of a vehicle obstacle detection method provided by the present application. The method comprises:

[0041] Step 11: Obtain obstacle features in the first image and the second image, and convert the obstacle features in the second image into the first image to obtain a first obstacle feature set; wherein the first image and the second image are images acquired at different focal lengths at the same time and the same viewing angle; and the focal length corresponding to the second image is greater than the focal length corresponding to the first image.

[0042] In some embodiments, during road driving, in order to accurately perceive the environment around the vehicle body, 7 on-board cameras installed on the vehicle are used to obtain perspective images from 7 surrounding perspectives, namely, front short focus img0, right front img1, right rear img2, rear img3, left rear img4, left front img5, and front long focus img6. Among them, the first image is the front short focus img0, and the second image is the front long focus img6.

[0043] Then, the trained obstacle detection model is used to detect the obstacles ps in img0 and pl in img6. Since the short-focus image is a wide-angle scene and can only capture nearby targets, the long-focus image brings distant targets closer and is suitable for narrow-angle scenes, which can detect distant targets more accurately. The bounding box of the i-th obstacle detected in the img0 and img6 images is represented by (u i ′,v i ′,w i ,h i ), in the image coordinate system, the upper left corner of the image is the origin, the right is the horizontal u direction, and the downward is the vertical v direction, where (u i ′,v i ′) represents the coordinates of the upper left corner of the bounding box, w i Indicates the width of the bounding box, h i Indicates the height of the bounding box.

[0044] Find the feature points of the obstacle, usually represented by the ground point of the obstacle: the center point of the lower edge of the detection box, and the coordinates of the feature point of the i-th obstacle are p i (u i ,v i ), where: i =u i ′+w i / 2;v i =v i '+h. That is, each obstacle feature has corresponding coordinates.

[0045] In some embodiments, the obstacles may be static obstacles and / or dynamic obstacles. For example, the static obstacles may be cones, water barriers, crash columns, crash barrels, triangular warning signs, etc. in road scenes.

[0046] In some embodiments, the obstacle features in the second image may be converted into the first image using a homography matrix to obtain a first obstacle feature set.

[0047] In some embodiments, since the input image involved in the subsequent 2D lane line segmentation is img0, in order to facilitate the subsequent calculation of the obstacle position using the lane line constraint, the obstacle p detected in the images img0 and img6 is i Perform unified processing: replace p in img6 il Get p in img0 through homography transformation is , p il (u il ,v il ) in which l means telephoto, p is (u is ,v is) where s represents short focal length. The mathematical formula of homography transformation is:

[0048] p is =H·p il .

[0049] Where H is the homography matrix. In this application, it is assumed that this parameter is known. After this step, obstacles are uniformly represented by p i It means that if the number of obstacles within the viewing angle is set to S, then all obstacles can be expressed as

[0050] Step 12: Obtain a number of Bev lane lines and a number of 2D lane lines, and match each 2D lane line with a corresponding Bev lane line to obtain a matching relationship.

[0051] In some embodiments, the perspective image can be detected using corresponding models to obtain corresponding Bev lane lines and 2D lane lines. Please refer to the following embodiments for details, which will not be described here.

[0052] See also Figure 2 , the matching relationship can be obtained by:

[0053] Step 21: Get the initial point in each 2D lane line.

[0054] Step 22: Perform inverse perspective transformation on the initial point to obtain the target point in the vehicle body coordinate system.

[0055] Step 23: Find the target Bev lane line corresponding to the target point from among the multiple Bev lane lines, and establish a matching relationship between the 2D lane line corresponding to the target point and the target Bev lane line.

[0056] In this application, the lateral distance is used to determine the relative position of the lane line, and then the Bev lane line and the 2D lane line are matched. Since the lateral distance between the lane lines of adjacent lanes is generally about 3m (i.e., the width of one lane), the lateral position of the initial point of the lane line is used in this application to match the Bev lane line and the 2D lane line:

[0057] For the nth lane line l in the perspective image img0 n , the inverse perspective transformation parameters of the nearby feature points are less likely to be affected by vehicle bumps and road environment, so the initial point sp n (u sn ,v sn ) is transformed into the vehicle body coordinate system through the inverse perspective transformation matrix as SP n ′(x sn ,y sn ,0), the formula for inverse perspective transformation is: SP n ′=K·(R·spn +t).

[0058] Where R is the rotation matrix, which is related to the external parameters, t is the translation vector, and both represent the transformation of the camera coordinate system relative to the body coordinate system. K represents the camera intrinsic parameter matrix. In the present invention, R, t and K are all known. Traverse all Bev lane lines in the body coordinate system and find the ones that are the same as SP at the same longitudinal coordinate. n ′The mth Bev lane line, that is, l n The matching Bev lane line is L m , the minimum distance is denoted as d n , expressed as:

[0059]

[0060] If d n <1, then the set of points matching the subscripts of the 2D lane line and the Bev lane line is denoted as match:

[0061] Use the above method to match each 2D lane line in img0 to the corresponding Bev lane line.

[0062] Step 13: Perform an inverse perspective transformation on the first obstacle features in the first obstacle feature set to obtain second obstacle features in the vehicle body coordinate system, and obtain a second obstacle feature set consisting of the second obstacle features.

[0063] From a functional perspective, if there is a lateral error in the obstacle ranging, it may cause the obstacle to mistakenly invade the lane where the vehicle is traveling, causing the vehicle to slow down and other related functions to brake. This application uses lane lines to constrain the lateral distance of the obstacle (the y direction in the vehicle body coordinate system), and the longitudinal (forward) distance (the x direction in the vehicle body coordinate system) uses the longitudinal distance after inverse perspective transformation.

[0064] In this step, p in the i-th obstacle image coordinate system is i (u i ,v i ) is transformed to the vehicle body coordinates using the same inverse perspective transformation i (x i ,y i ,0):P i =K·(R·p i +t). Where R is the rotation matrix, which is related to the external parameters, t is the translation vector, and both represent the transformation of the camera coordinate system relative to the vehicle body coordinate system, and K represents the camera intrinsic parameter matrix. That is, the first obstacle feature in the first obstacle feature set is subjected to an inverse perspective transformation to obtain the second obstacle feature in the vehicle body coordinate system.

[0065] Step 14: Determine a 2D lane line key point corresponding to each first obstacle feature based on a plurality of 2D lane lines; and determine a Bev lane line key point corresponding to each second obstacle feature based on a plurality of Bev lane lines.

[0066] In some embodiments, there is a relative relationship between the obstacle feature and the lane line, and the key points corresponding to the obstacle feature and the lane line can be found according to the relative relationship. Please refer to the following embodiments for details, which will not be described here.

[0067] Step 15: According to the matching relationship, the 2D lane line key points, the Bev lane line key points, the first obstacle feature and the second obstacle feature, the actual position of the obstacle corresponding to the first obstacle feature in the vehicle body coordinate system is obtained.

[0068] In some embodiments, after the actual position of the obstacle in the vehicle body coordinate system is obtained, a distance measurement calculation may be performed in combination with the actual position of the vehicle to obtain the distance between the obstacle and the vehicle.

[0069] In some embodiments, the distance between all obstacles in the image and the vehicle can be calculated using the method of the present application.

[0070] In this embodiment, obstacle features in a first image and a second image are obtained, and the obstacle features in the second image are converted into the first image to obtain a first obstacle feature set; wherein the first image and the second image are images acquired at different focal lengths at the same time and the same viewing angle; the focal length corresponding to the second image is greater than the focal length corresponding to the first image; a plurality of Bev lane lines and a plurality of 2D lane lines are obtained, and a corresponding Bev lane line is matched for each 2D lane line to obtain a matching relationship; an inverse perspective transformation is performed on the first obstacle feature in the first obstacle feature set to obtain a second obstacle feature in a vehicle body coordinate system, and a second obstacle feature set composed of the second obstacle features is obtained; a 2D lane line key point corresponding to each first obstacle feature is determined according to the plurality of 2D lane lines; and a Bev lane line key point corresponding to each second obstacle feature is determined according to the plurality of Bev lane lines; and an actual position of the obstacle corresponding to the first obstacle feature in the vehicle body coordinate system is obtained according to the matching relationship, the 2D lane line key point, the Bev lane line key point, the first obstacle feature, and the second obstacle feature. By utilizing the stable 2D lane lines and the Bev lane lines in the vehicle body coordinate system, the actual position of the obstacle in the vehicle body coordinate system is stably output according to the relative position relationship between the lane lines and the obstacles, thereby improving the accuracy of obstacle positioning and thus improving the accuracy of distance measurement between the vehicle and the obstacle.

[0071] See also Figure 3 , Figure 3FIG. 1 is a flow chart of another embodiment of a vehicle obstacle detection method provided by the present application. The method comprises:

[0072] Step 31: Obtain obstacle features in the first image and the second image, and convert the obstacle features in the second image into the first image to obtain a first obstacle feature set; wherein the first image and the second image are images acquired at different focal lengths at the same time and the same viewing angle; and the focal length corresponding to the second image is greater than the focal length corresponding to the first image.

[0073] Step 32: Input the first image into a 2D segmentation model to obtain a number of 2D lane lines; input the first image, the second image and the remaining view images into a Bev detection model to obtain a number of Bev lane lines.

[0074] In this step, the output results of the 2D segmentation model and the Bev detection model can be post-processed accordingly, such as point extraction, fitting, and filtering to obtain the corresponding lane line cubic curve analytical expression. The Bev lane line is the 3D lane line in the vehicle body coordinate system, while the perspective view (first image) obtains the 2D lane line, and the coordinates of the starting point and the ending point of the lane line are calculated. The starting point is the point closest to the vehicle, and the ending point is the point farthest from the vehicle.

[0075] Using the Bev lane line detection model and images from 7 perspectives, the mth Bev lane line L is detected m (x) The analytical formula is: L m (x) = C m0 +c m1 x+c m2 x 2 +c m3 x 3 .

[0076] L m (x) is denoted as Lm, where c m0 -c m3 are the parameters of the analytical expression of Bev lane lines respectively. The number of bev lane lines within the viewing angle is set to M, and the Bev lane lines within the viewing angle are expressed as:

[0077] L m is a curve in the vehicle body coordinate system, on the plane of z = 0, and in this application, it is assumed that the height of the obstacle is 0. Figure 4 As shown. m The starting point coordinates are marked as SP m (x Sm ,y Sm ,0), the starting point of the Bev lane line within the viewing angle is expressed as:

[0078] The end point coordinates are EP m (x Em ,y Em ,0), the end point of the Bev lane line within the viewing angle is expressed as:

[0079] Use the 2D segmentation model to segment the nth 2D lane line l in img0 n (v):l n (v) = k n0 +k n1 v+k n2 v 2 +k n3 v 3 .

[0080] Will l n (v) is denoted as l n , is the curve in the image coordinate system. k n0 -k n3 are the coefficients of the 2D lane line analytical expression respectively. The number of 2D lane lines within the viewing angle is set to N, and the 2D lane lines within the viewing angle are expressed as:

[0081] ln is the curve in the image coordinate system, such as Figure 5 As shown. The starting point of ln is marked as sp n (u sn ,v sn ), the starting point of the 2D lane line within the viewing angle is expressed as:

[0082] The end point coordinates are marked as ep n (u en ,v en ), the end point of the 2D lane line within the viewing angle is expressed as:

[0083]

[0084] In some embodiments, L m The visible lane lines have a wider range, but only have corresponding results in world coordinates. If only the perspective transformation matrix is ​​used to convert Lm to img0, when encountering uphill and downhill, forks, small bends and other scenes or when the vehicle shakes, resulting in inaccurate camera extrinsics, the lane lines in the image coordinate system will have a large error with the actual situation and are prone to shaking, making it impossible to accurately judge the relative position with obstacles on img0.

[0085] The ln segmented from img0 can accurately segment the lane lines in the perspective view. The segmentation result can be used to determine the relative position to the obstacles, but the visible distance is limited. If the inverse perspective transformation is used to convert it into a 3D lane line in the vehicle body coordinate system, it will also be affected by vehicle bumps and road conditions in the distance, causing the lane lines to appear inward and outward.

[0086] This application combines the advantages and disadvantages of Bev lane lines and 2D lane lines segmented on perspective views, and uses two types of lane lines to constrain obstacle ranging. The stability of the obstacle's 3D coordinates in the vehicle body coordinate system is not affected by bumps and road environment.

[0087] Step 33: Match each 2D lane line with the corresponding Bev lane line to obtain a matching relationship.

[0088] Step 34: performing an inverse perspective transformation on the first obstacle features in the first obstacle feature set to obtain second obstacle features in the vehicle body coordinate system, and to obtain a second obstacle feature set consisting of the second obstacle features.

[0089] Step 35: Determine a 2D lane line key point corresponding to each first obstacle feature based on a plurality of 2D lane lines; and determine a Bev lane line key point corresponding to each second obstacle feature based on a plurality of Bev lane lines.

[0090] Step 36: According to the matching relationship, the 2D lane line key points, the Bev lane line key points, the first obstacle feature and the second obstacle feature, the actual position of the obstacle corresponding to the first obstacle feature in the vehicle body coordinate system is obtained.

[0091] In this embodiment, obstacle features in a first image and a second image are obtained, and the obstacle features in the second image are converted into the first image to obtain a first obstacle feature set; wherein the first image and the second image are images acquired at different focal lengths at the same time and the same viewing angle; the focal length corresponding to the second image is greater than the focal length corresponding to the first image; a plurality of Bev lane lines and a plurality of 2D lane lines are obtained, and a corresponding Bev lane line is matched for each 2D lane line to obtain a matching relationship; an inverse perspective transformation is performed on the first obstacle feature in the first obstacle feature set to obtain a second obstacle feature in a vehicle body coordinate system, and a second obstacle feature set composed of the second obstacle features is obtained; a 2D lane line key point corresponding to each first obstacle feature is determined according to the plurality of 2D lane lines; and a Bev lane line key point corresponding to each second obstacle feature is determined according to the plurality of Bev lane lines; and an actual position of the obstacle corresponding to the first obstacle feature in the vehicle body coordinate system is obtained according to the matching relationship, the 2D lane line key point, the Bev lane line key point, the first obstacle feature, and the second obstacle feature. By utilizing the stable 2D lane lines and the Bev lane lines in the vehicle body coordinate system, the actual position of the obstacle in the vehicle body coordinate system is stably output according to the relative position relationship between the lane lines and the obstacles, thereby improving the accuracy of obstacle positioning and thus improving the accuracy of distance measurement between the vehicle and the obstacle.

[0092] See also Figure 6 , Figure 6 FIG. 1 is a flow chart of another embodiment of a vehicle obstacle detection method provided by the present application. The method comprises:

[0093] Step 61: Obtain obstacle features in the first image and the second image, and convert the obstacle features in the second image into the first image to obtain a first obstacle feature set; wherein the first image and the second image are images acquired at different focal lengths at the same time and the same viewing angle; and the focal length corresponding to the second image is greater than the focal length corresponding to the first image.

[0094] Step 62: Obtain a number of Bev lane lines and a number of 2D lane lines, and match each 2D lane line with a corresponding Bev lane line to obtain a matching relationship.

[0095] Step 63: performing an inverse perspective transformation on the first obstacle features in the first obstacle feature set to obtain second obstacle features in the vehicle body coordinate system, and to obtain a second obstacle feature set consisting of the second obstacle features.

[0096] Step 64: Determine two target 2D lane lines corresponding to each first obstacle feature according to the plurality of 2D lane lines.

[0097] In some embodiments, the two target 2D lane lines are 2D lane lines that are closest to the first obstacle feature.

[0098] In some embodiments, in the short-focus perspective image (first image), the image coordinates p for the i-th obstacle are i (u i ,v i ), traverse the 2D lane line l on the perspective image img0, and find the distance p according to the relative position relationship between the obstacle and the lane line i The two nearest lane lines l n1 and l n2 . First find l n1 :

[0099]

[0100] Assume that l n1 When the equation holds true, n=n1.

[0101]

[0102] Assume that l n2 When the equation holds true, n=n2.

[0103] Step 65: Find the 2D lane line key point corresponding to the corresponding first obstacle feature in the two target 2D lane lines.

[0104] Combine each 2D lane line with the corresponding Bev lane line to obtain the matching relationship and two target 2D lane lines, and find the i-th obstacle p i (u i ,v i ) The two nearest lane lines l n1 and l n2 Two 2D key points a on i1 (u i1 ,v i1 ) and a i2 (u i2 ,v i2 ), the v direction coordinates of the key points and the v direction coordinates of the obstacles i Same, where:

[0105] u i1 = l n1 (v i ), v i1 =v i .

[0106] u i2 = l n2 (v i ), v i2 =v i .

[0107] Step 66: Determine two target Bev lane lines corresponding to each second obstacle feature according to a plurality of Bev lane lines.

[0108] According to the matching relationship, after two target 2D lane lines are determined, two target Bev lane lines corresponding to each second obstacle feature can be determined from a plurality of Bev lane lines.

[0109] Step 67: Find the Bev lane line key point corresponding to the corresponding second obstacle feature in the two target Bev lane lines.

[0110] According to each 2D lane line matching the corresponding Bev lane line, a matching relationship is obtained. For example, the point set match(n,m) of the 2D lane line and the Bev lane line matching, given the 2D lane lines with subscripts n1 and n2, the corresponding Bev lane lines L with subscripts m1 and m2 can be obtained. m1 and L m2 Combined with the 3D vehicle coordinates P of the obstacle i (x i ,y i ,0), find the corresponding 3D key point A in the vehicle body coordinate system i1 (x i1 ,y i1 ,0) and A i2 (x i2 ,y i2 ,0), where:

[0111] x i1 =x i ,y i1 =L m1 (x i ).

[0112] x i2 =x i ,y i2 =L m2 (x i ).

[0113] The 2D coordinates a of the key points obtained above i1 and a i2 On the segmented 2D lane line, the stability is not affected by the environment such as road bumps, and the 3D coordinates of the key points A i1 and A i2 On the Bev lane line, stability is not affected by road bumps and other environmental factors. Because the 2D lane line and the Bev lane line match, it can be considered that a i1 and A i1 Yes, corresponding to i2 and A i2Are corresponding.

[0114] Step 68: According to the matching relationship, the 2D lane line key points, the Bev lane line key points, the first obstacle feature and the second obstacle feature, the actual position of the obstacle corresponding to the first obstacle feature in the vehicle body coordinate system is obtained.

[0115] In this embodiment, obstacle features in a first image and a second image are obtained, and the obstacle features in the second image are converted into the first image to obtain a first obstacle feature set; wherein the first image and the second image are images acquired at different focal lengths at the same time and the same viewing angle; the focal length corresponding to the second image is greater than the focal length corresponding to the first image; a plurality of Bev lane lines and a plurality of 2D lane lines are obtained, and a corresponding Bev lane line is matched for each 2D lane line to obtain a matching relationship; an inverse perspective transformation is performed on the first obstacle feature in the first obstacle feature set to obtain a second obstacle feature in a vehicle body coordinate system, and a second obstacle feature set composed of the second obstacle features is obtained; a 2D lane line key point corresponding to each first obstacle feature is determined according to the plurality of 2D lane lines; and a Bev lane line key point corresponding to each second obstacle feature is determined according to the plurality of Bev lane lines; and an actual position of the obstacle corresponding to the first obstacle feature in the vehicle body coordinate system is obtained according to the matching relationship, the 2D lane line key point, the Bev lane line key point, the first obstacle feature, and the second obstacle feature. By utilizing the stable 2D lane lines and the Bev lane lines in the vehicle body coordinate system, the actual position of the obstacle in the vehicle body coordinate system is stably output according to the relative position relationship between the lane lines and the obstacles, thereby improving the accuracy of obstacle positioning and thus improving the accuracy of distance measurement between the vehicle and the obstacle.

[0116] See also Figure 7 , Figure 7 FIG. 1 is a flow chart of another embodiment of a vehicle obstacle detection method provided by the present application. The method comprises:

[0117] Step 71: Obtain obstacle features in the first image and the second image, and convert the obstacle features in the second image into the first image to obtain a first obstacle feature set; wherein the first image and the second image are images acquired at different focal lengths at the same time and the same viewing angle; and the focal length corresponding to the second image is greater than the focal length corresponding to the first image.

[0118] Step 72: Obtain a number of Bev lane lines and a number of 2D lane lines, and match each 2D lane line with a corresponding Bev lane line to obtain a matching relationship.

[0119] Step 73: performing an inverse perspective transformation on the first obstacle features in the first obstacle feature set to obtain second obstacle features in the vehicle body coordinate system, and to obtain a second obstacle feature set consisting of the second obstacle features.

[0120] Step 74: using a plurality of Bev lane lines, constraining the second obstacle features in the second obstacle feature set; and removing the constrained second obstacle features from the second obstacle feature set.

[0121] From a functional perspective, because the system uses lane lines to control the vehicle, if the farthest point of the lane line does not reach the longitudinal distance of the obstacle, then the obstacle has little effect on the vehicle control function at the current position. Therefore, this step uses the end point of the lane line to constrain the obstacle longitudinally. If the obstacle exceeds the longitudinal coordinates of the end points of all lane lines, the obstacle is hidden. For the i-th obstacle, the xi value of Pi should be less than the x coordinate of the end point EP of all Bev lane lines, that is, it should satisfy:

[0122]

[0123] Step 75: Determine a 2D lane line key point corresponding to each first obstacle feature based on a plurality of 2D lane lines; and determine a Bev lane line key point corresponding to each second obstacle feature based on a plurality of Bev lane lines.

[0124] Step 76: According to the matching relationship, the 2D lane line key points, the Bev lane line key points, the first obstacle feature and the second obstacle feature, the actual position of the obstacle corresponding to the first obstacle feature in the vehicle body coordinate system is obtained.

[0125] In this embodiment, obstacle features in a first image and a second image are obtained, and the obstacle features in the second image are converted into the first image to obtain a first obstacle feature set; wherein the first image and the second image are images acquired at different focal lengths at the same time and the same viewing angle; the focal length corresponding to the second image is greater than the focal length corresponding to the first image; a plurality of Bev lane lines and a plurality of 2D lane lines are obtained, and a corresponding Bev lane line is matched for each 2D lane line to obtain a matching relationship; an inverse perspective transformation is performed on the first obstacle feature in the first obstacle feature set to obtain a second obstacle feature in a vehicle body coordinate system, and a second obstacle feature set composed of the second obstacle features is obtained; a 2D lane line key point corresponding to each first obstacle feature is determined according to the plurality of 2D lane lines; and a Bev lane line key point corresponding to each second obstacle feature is determined according to the plurality of Bev lane lines; and an actual position of the obstacle corresponding to the first obstacle feature in the vehicle body coordinate system is obtained according to the matching relationship, the 2D lane line key point, the Bev lane line key point, the first obstacle feature, and the second obstacle feature. By utilizing the stable 2D lane lines and the Bev lane lines in the vehicle body coordinate system, the actual position of the obstacle in the vehicle body coordinate system is stably output according to the relative position relationship between the lane lines and the obstacles, thereby improving the accuracy of obstacle positioning and thus improving the accuracy of distance measurement between the vehicle and the obstacle.

[0126] See also Figure 8 , Figure 8 FIG. 1 is a flow chart of another embodiment of a vehicle obstacle detection method provided by the present application. The method comprises:

[0127] Step 81: Obtain obstacle features in the first image and the second image, and convert the obstacle features in the second image into the first image to obtain a first obstacle feature set; wherein the first image and the second image are images acquired at different focal lengths at the same time and the same viewing angle; and the focal length corresponding to the second image is greater than the focal length corresponding to the first image.

[0128] Step 82: Obtain a number of Bev lane lines and a number of 2D lane lines, and match each 2D lane line with a corresponding Bev lane line to obtain a matching relationship.

[0129] Step 83: performing an inverse perspective transformation on the first obstacle features in the first obstacle feature set to obtain second obstacle features in the vehicle body coordinate system, and to obtain a second obstacle feature set consisting of the second obstacle features.

[0130] Step 84: Determine a 2D lane line key point corresponding to each first obstacle feature based on a plurality of 2D lane lines; and determine a Bev lane line key point corresponding to each second obstacle feature based on a plurality of Bev lane lines.

[0131] Step 85: Use the ordinate of the Bev lane line key point as the ordinate of the obstacle; calculate the ratio between the abscissa difference of the Bev lane line key point and the abscissa difference of the 2D lane line key point, and the difference between the abscissa of the 2D lane line key point and the abscissa of the first obstacle feature; obtain the abscissa and ordinate of the obstacle based on the ratio, difference, Bev lane line key point and 2D lane line key point.

[0132] For the i-th obstacle, use the 2D key point a obtained in the above step i1 and a i2 and 3D key point A i1 and A i2 , assuming that the actual distance represented by the pixels with the same longitudinal distance between two key points on the 2D image is the same, that is, the 3D distance is the same. Based on this assumption, this application calculates the 3D and 2D ratios between the two key points, combining the obstacle on the 2D image with the 2D key point a i1 , find the pixel distance between the obstacle and the 3D key point A in the vehicle body coordinates i1 Then, the actual coordinates ρ(x′, y′) of the obstacle in the vehicle body coordinate system after passing through the lane line constraint are calculated.

[0133] Before calculating, you need to prepare the p obtained in the above steps. i (u i ,v i ), P i (x i ,y i ,0),a i1 (u i1 ,v i1 ), a i2 (u i2 ,v i2 ), A i1 (x i1 ,y i1 ,0),A i2 (x i2 ,y i2 ,0). The actual coordinates ρ(x′, y′, 0) of the obstacle in the vehicle body coordinate system are calculated according to the following formula:

[0134]

[0135] In this embodiment, obstacle features in a first image and a second image are obtained, and the obstacle features in the second image are converted into the first image to obtain a first obstacle feature set; wherein the first image and the second image are images acquired at different focal lengths at the same time and the same viewing angle; the focal length corresponding to the second image is greater than the focal length corresponding to the first image; a plurality of Bev lane lines and a plurality of 2D lane lines are obtained, and a corresponding Bev lane line is matched for each 2D lane line to obtain a matching relationship; an inverse perspective transformation is performed on the first obstacle feature in the first obstacle feature set to obtain a second obstacle feature in a vehicle body coordinate system, and a second obstacle feature set composed of the second obstacle features is obtained; a 2D lane line key point corresponding to each first obstacle feature is determined according to the plurality of 2D lane lines; and a Bev lane line key point corresponding to each second obstacle feature is determined according to the plurality of Bev lane lines; and an actual position of the obstacle corresponding to the first obstacle feature in the vehicle body coordinate system is obtained according to the matching relationship, the 2D lane line key point, the Bev lane line key point, the first obstacle feature, and the second obstacle feature. By utilizing the stable 2D lane lines and the Bev lane lines in the vehicle body coordinate system, the actual position of the obstacle in the vehicle body coordinate system is stably output according to the relative position relationship between the lane lines and the obstacles, thereby improving the accuracy of obstacle positioning and thus improving the accuracy of distance measurement between the vehicle and the obstacle.

[0136] In an application scenario, it can be divided into the following steps:

[0137] Step 1: While the vehicle is driving, use the seven on-board cameras around the vehicle body to obtain perspective images of the vehicle body's surrounding environment from seven viewing angles.

[0138] Step 2: Detect obstacles in the forward-looking long-focus and short-focus images using a detection or segmentation model.

[0139] Step 3: Find the feature points that can represent obstacles in the obstacle output results of the model.

[0140] Step 4: Use homography to transform the obstacles detected on the long-focus image to the short-focus image, and unify the long and short focal lengths of the obstacles.

[0141] Step 5: Use the lane line Bev detection model and the 2D segmentation model to detect the Bev lane line results and the 2D segmentation results on the perspective image respectively, and obtain the equations and related properties of the two lane lines.

[0142] Step 6: Match the Bev lane line and the 2d lane line.

[0143] Step 7: Perform inverse perspective transformation on the feature points of the obstacle.

[0144] Step 8: Use the end point of the lane line to constrain the obstacle longitudinally.

[0145] Step 9: Using the lane line 2D segmentation result on the perspective view, find the two lane lines closest to the obstacle according to the nearest principle.

[0146] Step 10: Combine the results in step 6 and step 9 to find the 2D and 3D coordinates of the two key points on the two nearest lane lines.

[0147] Step 11: Use 2D key points and Bev key points to measure and locate obstacles.

[0148] Step 12: Repeat steps 7-11 for the other obstacles.

[0149] See also Fig. 9 , Fig. 9 1 is a schematic diagram of the structure of an embodiment of the vehicle-mounted system provided by the present application. The vehicle-mounted system 90 includes a memory 91 and a processor 92 coupled to the memory 91. The memory 91 stores at least one computer program. When the at least one computer program is loaded and executed by the processor 92, it is used to implement the following method:

[0150] Obtain obstacle features in the first image and the second image, and convert the obstacle features in the second image into the first image to obtain a first obstacle feature set; wherein the first image and the second image are images acquired at different focal lengths at the same time and the same viewing angle; the focal length corresponding to the second image is greater than the focal length corresponding to the first image; obtain a plurality of Bev lane lines and a plurality of 2D lane lines, and match the corresponding Bev lane line for each 2D lane line to obtain a matching relationship; perform an inverse perspective transformation on the first obstacle feature in the first obstacle feature set to obtain a second obstacle feature in a vehicle body coordinate system, and obtain a second obstacle feature set composed of the second obstacle features; determine a 2D lane line key point corresponding to each first obstacle feature according to the plurality of 2D lane lines; and determine a Bev lane line key point corresponding to each second obstacle feature according to the plurality of Bev lane lines; obtain the actual position of the obstacle corresponding to the first obstacle feature in the vehicle body coordinate system according to the matching relationship, the 2D lane line key point, the Bev lane line key point, the first obstacle feature, and the second obstacle feature.

[0151] In some embodiments, when at least one computer program is loaded and executed by the processor 92, it is also used to implement the following method: input the first image into a 2D segmentation model to obtain a number of 2D lane lines; input the first image, the second image, and the remaining perspective images into a Bev detection model to obtain a number of Bev lane lines.

[0152] In some embodiments, when at least one computer program is loaded and executed by the processor 92, it is also used to implement the following method: determining two target 2D lane lines corresponding to each first obstacle feature based on a plurality of 2D lane lines; and finding a 2D lane line key point corresponding to the corresponding first obstacle feature in the two target 2D lane lines.

[0153] In some embodiments, when at least one computer program is loaded and executed by the processor 92, it is also used to implement the following method: determine two target Bev lane lines corresponding to each second obstacle feature based on a number of Bev lane lines; find the Bev lane line key point corresponding to the corresponding second obstacle feature in the two target Bev lane lines.

[0154] In some embodiments, when at least one computer program is loaded and executed by the processor 92, it is also used to implement the following method: using a plurality of Bev lane lines to constrain the second obstacle features in the second obstacle feature set; and removing the constrained second obstacle features from the second obstacle feature set.

[0155] In some embodiments, when at least one computer program is loaded and executed by the processor 92 , it is also used to implement the following method: using a homography matrix to transform obstacle features in the second image into the first image to obtain a first obstacle feature set.

[0156] In some embodiments, when at least one computer program is loaded and executed by the processor 92, it is also used to implement the following method: obtaining an initial point in each 2D lane line; performing an inverse perspective transformation on the initial point to obtain a target point in the vehicle body coordinate system; finding a target Bev lane line corresponding to the target point from a number of Bev lane lines, and establishing a matching relationship between the 2D lane line corresponding to the target point and the target Bev lane line.

[0157] In some embodiments, when at least one computer program is loaded and executed by the processor 92, it is also used to implement the following method: using the longitudinal coordinate of the Bev lane line key point as the longitudinal coordinate of the obstacle; calculating the ratio between the transverse coordinate difference of the Bev lane line key point and the transverse coordinate difference of the 2D lane line key point, and the difference between the transverse coordinate of the 2D lane line key point and the transverse coordinate of the first obstacle feature; obtaining the transverse coordinate and longitudinal coordinate of the obstacle based on the ratio, difference, Bev lane line key point and 2D lane line key point.

[0158] In some embodiments, when at least one computer program is loaded and executed by the processor 92, it is also used to implement the method of any of the above embodiments.

[0159] See also Fig.10 , Fig.101 is a schematic diagram of a computer-readable storage medium according to an embodiment of the present application. The computer-readable storage medium 100 stores computer-executable instructions 101, which are used to implement the following method when executed by a processor:

[0160] Obtain obstacle features in the first image and the second image, and convert the obstacle features in the second image into the first image to obtain a first obstacle feature set; wherein the first image and the second image are images acquired at different focal lengths at the same time and the same viewing angle; the focal length corresponding to the second image is greater than the focal length corresponding to the first image; obtain a plurality of Bev lane lines and a plurality of 2D lane lines, and match the corresponding Bev lane line for each 2D lane line to obtain a matching relationship; perform an inverse perspective transformation on the first obstacle feature in the first obstacle feature set to obtain a second obstacle feature in a vehicle body coordinate system, and obtain a second obstacle feature set composed of the second obstacle features; determine a 2D lane line key point corresponding to each first obstacle feature according to the plurality of 2D lane lines; and determine a Bev lane line key point corresponding to each second obstacle feature according to the plurality of Bev lane lines; obtain the actual position of the obstacle corresponding to the first obstacle feature in the vehicle body coordinate system according to the matching relationship, the 2D lane line key point, the Bev lane line key point, the first obstacle feature, and the second obstacle feature.

[0161] In some embodiments, when the computer execution instruction 101 is executed by the processor, it is also used to implement the following method: input the first image into a 2D segmentation model to obtain a number of 2D lane lines; input the first image, the second image and the remaining perspective images into a Bev detection model to obtain a number of Bev lane lines.

[0162] In some embodiments, when the computer execution instruction 101 is executed by the processor, it is also used to implement the following method: determine two target 2D lane lines corresponding to each first obstacle feature based on a plurality of 2D lane lines; find the 2D lane line key point corresponding to the corresponding first obstacle feature in the two target 2D lane lines.

[0163] In some embodiments, when the computer execution instruction 101 is executed by the processor, it is also used to implement the following method: determine two target Bev lane lines corresponding to each second obstacle feature based on a number of Bev lane lines; find the Bev lane line key point corresponding to the corresponding second obstacle feature in the two target Bev lane lines.

[0164] In some embodiments, when the computer execution instruction 101 is executed by the processor, it is also used to implement the following method: using a plurality of Bev lane lines to constrain the second obstacle features in the second obstacle feature set; and removing the constrained second obstacle features from the second obstacle feature set.

[0165] In some embodiments, when the computer-executable instruction 101 is executed by the processor, it is also used to implement the following method: using the homography matrix to transform the obstacle features in the second image into the first image to obtain a first obstacle feature set.

[0166] In some embodiments, when the computer execution instruction 101 is executed by the processor, it is also used to implement the following method: obtain the initial point in each 2D lane line; perform an inverse perspective transformation on the initial point to obtain the target point in the vehicle body coordinate system; find the target Bev lane line corresponding to the target point from a number of Bev lane lines, and establish a matching relationship between the 2D lane line corresponding to the target point and the target Bev lane line.

[0167] In some embodiments, when the computer execution instruction 101 is executed by the processor, it is also used to implement the following method: using the ordinate of the Bev lane line key point as the ordinate of the obstacle; calculating the ratio between the horizontal coordinate difference of the Bev lane line key point and the horizontal coordinate difference of the 2D lane line key point, and the difference between the horizontal coordinate of the 2D lane line key point and the horizontal coordinate of the first obstacle feature; obtaining the horizontal coordinate and vertical coordinate of the obstacle according to the ratio, difference, Bev lane line key point and 2D lane line key point.

[0168] In some embodiments, the computer-executable instruction 101 is also used to implement the method of any of the above embodiments when executed by the processor.

[0169] In summary, the obstacle detection method, vehicle-mounted system, and readable storage medium provided by the present application obtain obstacle features in a first image and a second image, and convert the obstacle features in the second image into the first image to obtain a first obstacle feature set; wherein the first image and the second image are images acquired at different focal lengths at the same time and the same viewing angle; the focal length corresponding to the second image is greater than the focal length corresponding to the first image; a plurality of Bev lane lines and a plurality of 2D lane lines are obtained, and a corresponding Bev lane line is matched for each 2D lane line to obtain a matching relationship; a first obstacle feature in the first obstacle feature set is subjected to an inverse perspective transformation to obtain a second obstacle feature in a vehicle body coordinate system, and a second obstacle feature set composed of the second obstacle features is obtained; a 2D lane line key point corresponding to each first obstacle feature is determined according to the plurality of 2D lane lines; and a Bev lane line key point corresponding to each second obstacle feature is determined according to the plurality of Bev lane lines; and the actual position of the obstacle corresponding to the first obstacle feature in the vehicle body coordinate system is obtained according to the matching relationship, the 2D lane line key point, the Bev lane line key point, the first obstacle feature, and the second obstacle feature. By utilizing the stable 2D lane lines and the Bev lane lines in the vehicle body coordinate system, the actual position of the obstacle in the vehicle body coordinate system is stably output according to the relative position relationship between the lane lines and the obstacles, thereby improving the accuracy of obstacle positioning and thus improving the accuracy of distance measurement between the vehicle and the obstacle.

[0170] Furthermore, the present application will process a characteristic point integrated into a static obstacle, so in principle it is universal for static obstacles, including cones, water barriers, crash barriers, crash barrels and triangular warning signs.

[0171] Furthermore, the present application utilizes the 2D lane lines segmented in the image coordinate system and the 3D Bev lane lines in the vehicle body coordinate system to constrain the position of obstacles, so that the stability of the 3D coordinates of the obstacles in the vehicle body coordinate system is not affected by the camera attitude angle and the road environment.

[0172] Furthermore, the present application utilizes the relative position relationship between obstacles and lane lines, combined with the distance between two 3D key points and the proportional relationship between two 2D key points, to locate obstacles, thereby improving positioning accuracy.

[0173] In the several embodiments provided in this application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only illustrative, for example, the division of the modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0174] If the integrated units in the above other embodiments are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor 10 (processor) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0175] The above description is only an implementation method of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A vehicle obstacle detection method, characterized in that: The method comprises: Obtain obstacle features in a first image and a second image, and convert the obstacle features in the second image into the first image to obtain a first obstacle feature set; wherein the first image and the second image are images acquired at the same time and the same viewing angle with different focal lengths; and the focal length corresponding to the second image is greater than the focal length corresponding to the first image; Acquire a plurality of Bev lane lines and a plurality of 2D lane lines, and match each of the 2D lane lines with a corresponding Bev lane line to obtain a matching relationship; Performing an inverse perspective transformation on a first obstacle feature in the first obstacle feature set to obtain a second obstacle feature in a vehicle body coordinate system, and obtaining a second obstacle feature set consisting of the second obstacle features; Determine a 2D lane line key point corresponding to each of the first obstacle features according to the plurality of 2D lane lines; and determine a Bev lane line key point corresponding to each of the second obstacle features according to the plurality of Bev lane lines; According to the matching relationship, the 2D lane line key points, the Bev lane line key points, the first obstacle feature and the second obstacle feature, the actual position of the obstacle corresponding to the first obstacle feature in the vehicle body coordinate system is obtained.

2. The obstacle detection method according to claim 1, characterized in that: The obtaining of a plurality of Bev lane lines and a plurality of 2D lane lines includes: Inputting the first image into a 2D segmentation model to obtain the plurality of 2D lane lines; The first image, the second image and the remaining perspective images are input into the Bev detection model to obtain the plurality of Bev lane lines.

3. The obstacle detection method according to claim 1, characterized in that: The determining, according to the plurality of 2D lane lines, a 2D lane line key point corresponding to each of the first obstacle features comprises: Determine two target 2D lane lines corresponding to each of the first obstacle features according to the plurality of 2D lane lines; Find a 2D lane line key point corresponding to the first obstacle feature in the two target 2D lane lines.

4. The obstacle detection method according to claim 1, characterized in that: The determining, according to the plurality of Bev lane lines, a Bev lane line key point corresponding to each second obstacle feature comprises: Determine two target Bev lane lines corresponding to each of the second obstacle features according to the plurality of Bev lane lines; Find the Bev lane line key point corresponding to the corresponding second obstacle feature among the two target Bev lane lines.

5. The obstacle detection method according to claim 1, characterized in that: After obtaining the second obstacle feature set consisting of the second obstacle features, the method further includes: Using the plurality of Bev lane lines, constraining the second obstacle feature in the second obstacle feature set; And the constrained second obstacle features are removed from the second obstacle feature set.

6. The obstacle detection method according to claim 1, characterized in that: The step of converting the obstacle features in the second image into the first image to obtain a first obstacle feature set includes: The obstacle features in the second image are converted into the first image using a homography matrix to obtain the first obstacle feature set.

7. The obstacle detection method according to claim 1, characterized in that: The matching of each of the 2D lane lines to the corresponding Bev lane line to obtain a matching relationship includes: Obtaining an initial point in each of the 2D lane lines; Performing an inverse perspective transformation on the initial point to obtain a target point in the vehicle body coordinate system; A target Bev lane line corresponding to the target point is found from the plurality of Bev lane lines, and a matching relationship is established between the 2D lane line corresponding to the target point and the target Bev lane line.

8. The obstacle detection method according to claim 1, characterized in that: The obtaining, according to the matching relationship, the 2D lane line key points, the Bev lane line key points, the first obstacle feature and the second obstacle feature, an actual position of the obstacle corresponding to the first obstacle feature in the vehicle body coordinate system includes: The ordinate of the Bev lane line key point is used as the ordinate of the obstacle; Calculate the ratio between the horizontal coordinate difference of the Bev lane line key point and the horizontal coordinate difference of the 2D lane line key point, and the difference between the horizontal coordinate of the 2D lane line key point and the horizontal coordinate of the first obstacle feature; The horizontal coordinate and the vertical coordinate of the obstacle are obtained according to the ratio, the difference, the Bev lane line key point and the 2D lane line key point.

9. A vehicle-mounted system, characterized in that: The vehicle-mounted system includes a memory and a processor coupled to the memory, the memory stores at least one computer program, and when the at least one computer program is loaded and executed by the processor, it is used to implement the method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the method according to any one of claims 1 to 8.

Images