Vehicle local map generation method and device, electronic equipment and vehicle

Abstract

The invention provides a vehicle local map generation method and apparatus, an electronic device and a vehicle. The method comprises the steps of collecting an image in front of a driving vehicle in real time; for each image of each frame, performing identification based on the image to obtain road information; constructing a plurality of sampling points of the current frame based on the road information; performing point chain construction based on the road information and the plurality of sampling points of the current frame to obtain a point chain of the current frame; and generating a local map based on the point chain corresponding to each current frame. According to the method, the sensing range, lane line parameters and the like of the lane line in a camera coordinate system are identified; sampling points are extracted according to lane line parameters; then, performing point chain construction by using the road information and the plurality of sampling points of the current frame to obtain a point chain of the current frame; forming a local map by point chains; compared with single-frame sensing, the mode of forming the local map through the point chain can express the position and the shape of the lane line more accurately, so that the precision of constructing the local map is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of vehicle map, and particularly relates to a vehicle local map generation method and device, an electronic device and a vehicle. BACKGROUND

[0002] Autonomous driving refers to the ability of a vehicle to drive autonomously without human intervention through artificial intelligence, sensors and other technologies. With the development of autonomous driving technology, the autonomous driving scheme under the light map mode is gradually reducing its dependence on high-precision maps, changing the way of loading offline high-precision maps to generate local maps in real time through perception results, and constructing local maps online; so as to understand the position and lane information of the vehicle in autonomous driving through the local map to perform path planning and navigation tasks.

[0003] At present, in autonomous driving, a single frame of perception result is often used for local semantic vector mapping and road topology reasoning. For the above method, in some complex road surface scenarios, such as lane separation and merging and curved road sections, the accuracy of perception will decrease, which will directly affect the accuracy of local map construction. SUMMARY

[0004] Therefore, the embodiments of the present application provide a vehicle local map generation method and device, an electronic device and a vehicle to solve the problem of low accuracy of local map construction in the prior art.

[0005] To achieve the above object, the embodiments of the present application provide the following technical solutions:

[0006] The first aspect of the embodiments of the present application shows a vehicle local map generation method, which comprises:

[0007] Real-time acquisition of images in front of the vehicle;

[0008] For each frame of image, the road information is obtained by identifying based on the image;

[0009] A plurality of sampling points of the current frame are constructed based on the road information, and the sampling points have a preset distance;

[0010] Point chain construction is performed based on the road information and the plurality of sampling points of the current frame to obtain the point chain of the current frame;

[0011] A local map is generated based on the point chain corresponding to each frame.

[0012] Optionally, constructing a plurality of sampling points of the current frame based on the road information comprises:

[0013] For each image of each frame, it is judged whether there is a confidence greater than a preset threshold in the road information corresponding to each image;

[0014] If there is, the initial sampling points of the current frame are generated based on the road information with the confidence greater than the preset threshold;

[0015] The initial sampling points are processed according to a preset distance to obtain a plurality of sampling points of the current frame constructed by the preset distance.

[0016] Optionally, point chain construction is performed based on the road information and the plurality of sampling points of the current frame to obtain the point chain of the current frame, including:

[0017] If it is determined that the current frame is the first frame, interval points are obtained based on the lane line parameters and the perception information in the road information and a preset range;

[0018] Based on each sampling point under the current frame and the interval points, the average point of the sampling points in each two adjacent interval points is calculated and taken as the center point of the point chain;

[0019] The point chain of the current frame is constructed based on the center point.

[0020] Optionally, point chain construction is performed based on the road information and the plurality of sampling points of the current frame to obtain the point chain of the current frame, including:

[0021] If it is determined that the current frame is not the first frame, a historical point chain and a corresponding historical interval point are obtained;

[0022] If it is determined that the sampling points of the current frame and the historical point chain have an overlapping area and are in the same lane line, the sampling points of the current frame and the historical point chain are fused to obtain the point chain of the current frame.

[0023] Optionally, the sampling points of the current frame and the historical point chain are fused to obtain the point chain of the current frame, including:

[0024] The sampling points of the current frame in the interval corresponding to the historical interval point are determined by traversing the interval corresponding to the historical interval point;

[0025] The historical point chain is updated based on the sampling points of the current frame in the interval corresponding to the historical interval point and the center point of the interval corresponding to the historical interval point;

[0026] The new interval points corresponding to each sampling point of the current frame are determined based on the road information corresponding to each image of the current frame with the confidence greater than the preset threshold and a preset range.

[0027] calculating an average point of the sampling points within each two new interval points as a new center point based on each sampling point and the new interval points remaining in the current frame;

[0028] constructing a point chain of the current frame based on the updated history point chain and the new center point.

[0029] Optionally, generating a local map based on the point chain corresponding to each current frame, comprising:

[0030] if it is determined that the current frame is the first frame, obtaining a current position of the vehicle;

[0031] calculating a distance between each center point in the point chain of the current frame and the current position of the vehicle;

[0032] calculating a weight of the center point based on the distance;

[0033] deleting the center point with a weight less than a preset weight threshold, and constructing a local map based on the remaining center points of the current frame.

[0034] Optionally, generating a local map based on the point chain corresponding to each current frame, comprising:

[0035] if it is determined that the current frame is not the first frame, obtaining a center point of the current frame and a center point of the history point chain in an overlapping area based on the point chain of the current frame and the history point chain, the overlapping area being an overlapping part of the sampling points of the current frame and the history point chain;

[0036] updating the weight of the center point of the current frame in the overlapping area based on the weight of the center point of the current frame and the weight of the center point of the history point chain;

[0037] calculating a weight of the remaining center points of the current frame based on the distance between the remaining center points and the current position of the vehicle;

[0038] deleting the center point with a weight less than a preset weight threshold based on the updated weight of the center point of the current frame in the overlapping area and the weight of the remaining center points, and constructing a local map based on the remaining center points of the current frame.

[0039] The second aspect of the embodiment of the present application shows a vehicle local map generation device, the device comprising:

[0040] a collection unit configured to collect images in front of the vehicle in real time;

[0041] a recognition unit configured to recognize the images based on each image of each frame to obtain road information;

[0042] A construction unit is configured to construct a plurality of sampling points of a current frame based on the road information, and a preset distance exists between the sampling points; point chain construction is performed based on the road information and the plurality of sampling points of the current frame to obtain a point chain of the current frame;

[0043] A generation unit is configured to generate a local map based on the point chain corresponding to each current frame.

[0044] A third aspect of the embodiment of the present application shows an electronic device, which comprises a processor and a memory, the memory is configured to store data generation program code and data, and the processor is configured to call the program instructions in the memory to execute the vehicle local map generation method as described in the first aspect of the embodiment of the present application.

[0045] A fourth aspect of the embodiment of the present application shows a vehicle, which comprises a storage program, wherein when the program runs, the device in the vehicle is controlled to execute the vehicle local map generation method as shown in the first aspect of the embodiment of the present application.

[0046] Based on the above-mentioned vehicle local map generation method, device, electronic device and vehicle provided by the embodiment of the present application, the method comprises: collecting images in front of a vehicle driving in real time; for each image of each frame, road information is obtained based on the image; a plurality of sampling points of a current frame are constructed based on the road information, and a preset distance exists between the sampling points; point chain construction is performed based on the road information and the plurality of sampling points of the current frame to obtain a point chain of the current frame; and a local map is generated based on the point chain corresponding to each current frame. In the embodiment of the present application, the sensing range and lane line parameters of the lane line in the camera coordinate system are identified; the sampling points are extracted according to the lane line parameters; then, the point chain construction is performed based on the road information and the plurality of sampling points of the current frame to obtain the point chain of the current frame; and the local map is constituted by the point chain; compared with single-frame sensing, the local map constituted by the point chain can more accurately express the position and shape of the lane line, thereby improving the accuracy of local map construction. BRIEF DESCRIPTION OF DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of the provided drawings.

[0048] Figure 1 A structural schematic diagram of the map point chain system shown in the embodiment of the present application;

[0049] Figure 2A flowchart of a vehicle local map generation method shown in an embodiment of the present application is shown in the figure;

[0050] Figure 3 A flowchart of point chain construction shown in an embodiment of the present application is shown in the figure;

[0051] Figure 4 An overlapping area of a point chain of a current frame and a sampling point of a historical point chain shown in an embodiment of the present application is shown in the figure;

[0052] Figure 5 A schematic diagram of point chain construction local map shown in an embodiment of the present application is shown in the figure;

[0053] Figure 6 A flowchart of vehicle local map generation shown in an embodiment of the present application is shown in the figure;

[0054] Figure 7 A structure schematic diagram of a vehicle local map generation device shown in an embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0055] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0056] The terms "first", "second", "third", "fourth" and the like (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or a chronological sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0057] It should be noted that the description of "first", "second" and the like in the present application is only for the purpose of description and cannot be understood as indicating or implying the relative importance of the technical features indicated or implying the number of technical features indicated. Therefore, the features defined as "first", "second" can be explicitly or implicitly included at least one of the features. In addition, the technical solutions of various embodiments can be combined with each other, but it must be based on the realization of ordinary skilled in the art, when the combination of technical solutions appears contradictory or cannot be realized, it should be considered that the combination of technical solutions does not exist, nor in the protection scope required by the present application.

[0058] In the present application, the terms "include", "contain" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or includes the elements inherent to such process, method, article or equipment. Without more limitation, the element defined by the sentence "including a…" does not exclude the presence of another identical element in the process, method, article or equipment including the element.

[0059] As known from the background art, in addition to constructing a local semantic vector map for the perception result of a single frame, there is also a technology of directly generating a local map end to end, inputting visual perception information into a model to obtain a constructed local map. For the technology of directly generating a local map end to end, since the behavior of the current model does not have complete explainability, when the map generated end to end has a problem, a lot of time needs to be spent to adjust the model, thereby affecting the accuracy of local map construction.

[0060] Based on this, the present application collects road information around the vehicle through a visual sensor, specifically including identifying the perception range of lane lines in the camera coordinate system and lane line parameters, etc.; fusing the current position of the vehicle through the Global Positioning System (GPS), Inertial Measurement Unit (IMU) and wheel speed sensor on the autonomous vehicle; extracting sampling points according to the lane line parameters; if the current frame is not the first frame, using the current position of the vehicle, tracking the sampling points of the current frame and the historical point chain in the ego coordinate system, fusing the sampling points that are successfully tracked with the point chain, and generating a new point chain for the sampling points that are not successfully tracked. The present solution can more accurately express the position and shape of the lane line by generating a point chain for the recognized road information, is less affected by abnormal perception, and can also deal with the scene of lane line occlusion disappearance because the results of historical frames are fused.

[0061] Referring to Figure 1This is a schematic diagram of the structure of the map point chain system shown in an embodiment of the present invention;

[0062] The map dot chain system includes an on-board position sensor (GPS / IMU / wheel speed sensor) 10, a vision sensor 20, a lane line recognition module 30, a dot chain construction module 40, a local map generator 50, and a vehicle controller 60 (not shown in the figure).

[0063] The vehicle controller 60 is connected to the onboard position sensor (GPS / IMU / wheel speed sensor) 10, the vision sensor 20, the lane line recognition module 30, the point chain construction module 40, and the local map generator 50.

[0064] based on Figure 1 The architecture shown specifically implements the process of generating a local map of the vehicle, such as... Figure 2 As shown, it includes the following steps:

[0065] Step S201: Acquire images of the area in front of the vehicle in real time;

[0066] In the specific implementation of step S201, during the vehicle's operation, the vehicle controller controls the vehicle's vision sensors to collect images containing lane lines around the vehicle in real time. When the vision sensors collect images containing lane lines around the vehicle, they send the images to the lane line recognition module.

[0067] It should be noted that the visual sensor captures multiple images in each frame.

[0068] Step S202: For each frame of the image, perform recognition based on the image to obtain road information;

[0069] It should be noted that road information includes lane line parameters, perception range, and confidence level.

[0070] In the specific implementation step S22, for each image in each frame, the vehicle controller controls the lane line recognition module to recognize the lane lines in the image captured by the vision sensor, obtain lane line parameters, namely polynomial coefficients c0, c1, c2, c3, perception range (r1, r2), and road information such as the confidence level of the lane line parameters, and sends the road information corresponding to each image in each frame to the point chain construction module.

[0071] Where r1 is the starting position of lane line perception, the x-coordinate in the vehicle coordinate system; and r2 is the ending position of lane line perception, the x-coordinate in the vehicle coordinate system.

[0072] Furthermore, the vehicle coordinate system uses the vehicle's direction of travel as the x-axis and the direction perpendicular to the direction of travel to the left as the y-axis.

[0073] The confidence degree refers to a clear degree of the lane line.

[0074] It should be noted that the lane line recognition model arranged in the lane line recognition module is obtained by training based on different images containing lane lines and different third-degree polynomial coefficients of the lane lines, perception ranges, and confidence degrees.

[0075] Step S203: constructing a plurality of sampling points of the current frame based on the road information.

[0076] The plurality of sampling points have a preset distance.

[0077] It should be noted that in the process of specifically implementing step S203, the following steps are included.

[0078] Step S11: for each image of each frame, determining whether there is a confidence degree greater than a preset threshold in the road information corresponding to each image, if yes, executing step S12, and if no, deleting the lane line parameter corresponding to the confidence degree.

[0079] In the process of specifically implementing step S11, the vehicle controller control point chain construction module executes real-time determination of the confidence degree of each image for each frame of a plurality of images, compares whether the confidence degree is greater than a preset threshold, if yes, retains the lane line parameter and the corresponding road information, and then executes step S12, and if no, deletes the lane line parameter corresponding to the confidence degree.

[0080] It should be noted that the preset threshold is set according to multiple experiments or experience, for example, can be set to 95%.

[0081] Step S12: generating initial sampling points of the current frame based on the road information with the confidence degree greater than the preset threshold.

[0082] It should be noted that in the process of specifically implementing step S12 by the vehicle controller control point chain construction module, the following steps are included.

[0083] Step S21: determining a horizontal coordinate value range of the sampling points based on the perception range in the road information with the confidence degree greater than the preset threshold.

[0084] In the process of specifically implementing step S21, first, the road information with the confidence degree greater than the preset threshold in any image of the frame is selected; the horizontal coordinate needs to be greater than or equal to the perception start position of the lane line and less than or equal to the perception end position of the lane line according to the perception range under the road information, and is taken as the horizontal coordinate value range of the sampling points; that is, the horizontal coordinate of the sampling point can be any coordinate from the perception start position of the lane line to the perception end position of the lane line.

[0085] Step S22: calculating the initial sampling points by using the lane line parameters in the road information and the horizontal coordinate value range.

[0086] In the process of implementing step S22, first, in the perception range, the lane line parameters are substituted into formula (1) for processing to obtain a plurality of sampling points (x pt , y pt ).

[0087] Formula (1):

[0088]

[0089] wherein the polynomial coefficients c0, c1, c2, c3 are lane line parameters, (r1, r2) is the perception range, x pt is the horizontal coordinate of the sampling point, and y pt is the vertical coordinate of the sampling point.

[0090] It should be noted that one horizontal coordinate is one decimal point after the decimal point, for example, 5.1 meters is one horizontal coordinate, and 5.2 meters is one horizontal coordinate. After step S22, the sampling points corresponding to each horizontal coordinate in the perception range can be obtained, that is, the number of each horizontal coordinate in the perception range is equal to the number of sampling points.

[0091] Then, the plurality of sampling points (x pt , y pt ) obtained by formula (1) are screened based on the horizontal coordinate value range, and the remaining sampling points after screening are taken as the initial sampling points.

[0092] Specifically, for each sampling point (x pt , y pt ) obtained by formula (1), it is judged whether the horizontal coordinate x pt satisfies the horizontal coordinate value range. If it satisfies, it is taken as the initial sampling point, and if it does not satisfy, it is removed.

[0093] Step S13: processing the initial sampling points according to the preset distance to obtain a plurality of sampling points constructed by the preset distance for the current frame.

[0094] In the process of implementing step S13, the vehicle controller control point chain construction module executes the multiple initial sampling points obtained in step S12 as a set; the initial sampling points in the set are deleted according to a preset distance, specifically, the initial sampling point closest to the vehicle position is selected from the set as a starting point; then, the distance of the next initial sampling point (assuming it is sampling point 1) closest to the starting point in the set is calculated; if the distance is less than the preset distance, the initial sampling point 1 in the set is deleted, and the distance of the next initial sampling point (assuming it is sampling point 2) closest to it in the set is recalculated; if the distance is equal to the preset distance, it is retained, and the initial sampling point 2 is the starting point, the distance of the next initial sampling point closest to the starting point in the set is calculated, and this process is repeated until all initial sampling points in the set are processed, so as to construct a sampling point every preset distance in the vehicle driving direction.

[0095] Step S204: constructing a point chain based on the road information and the multiple sampling points of the current frame to obtain a point chain of the current frame.

[0096] It should be noted that the process of implementing step S204 by the vehicle controller control point chain construction module can be as shown in the following table. Figure 3 The method comprises the following steps:

[0097] Step S301: determining whether the current frame is the first frame, if yes, executing step S302, and if no, executing step S305.

[0098] In the process of implementing step S301, the vehicle controller determines whether the current frame image is the first frame image sent by the vision sensor, if yes, executing step S302, and if no, executing step S305.

[0099] Step S302: processing based on the lane line parameters in the road information, the perception information, and a preset range to obtain interval points;

[0100] In the process of implementing step S302, for each image under the current frame whose confidence is greater than a preset threshold, the perception starting position in the perception range corresponding to the road information of the image is selected as an initial interval point (x0, y0); the intersection of the initial interval point (x0, y0) as the center and a preset range r as the radius of a circle with the lane line parameters is taken as a new interval point (x, y), that is, the initial interval point (x0, y0) as the center, the preset range r and the lane line parameters are substituted into the equation set shown in formula (2) to obtain a new interval point (x, y), and the initial interval point (x0, y0) and the new interval point (x, y) are recorded.

[0101] Based on formula (2), a new interval point (x, y) corresponding to each image with a confidence greater than a preset threshold in the current frame is calculated, so as to record each new interval point (x, y) and the initial interval point (x0, y0).

[0102] Formula (2):

[0103]

[0104] wherein c0, c1, c2, c3 are lane line parameters, x0 is the horizontal coordinate of the initial interval point, y0 is the vertical coordinate of the initial interval point, x is the horizontal coordinate of the new interval point, y is the vertical coordinate of the new interval point, and r is a preset range.

[0105] It should be noted that the preset range is set in advance according to multiple experiments.

[0106] Optionally, in addition to the method described in step S302, a bisection method can be used to calculate the interval point based on the road information.

[0107] Specifically, for each image with a confidence greater than a preset threshold in the current frame, the perception start position and the perception end position in the perception range corresponding to the perception information in the road information are taken as a distance interval; then the bisection method is used to divide the distance interval, and the left side of the distance interval divided by the bisection method is taken as the initial interval point (x0, y0), and the right side is taken as the new interval point (x, y).

[0108] Step S303: calculating the average point of the sampling points in each two adjacent interval points based on each sampling point in the current frame and the interval point, and taking the average point as the center point of the point chain;

[0109] In the process of implementing step S303, first, for each interval point (including the initial interval point and the new interval point), the adjacent interval points are determined according to the coordinates of the interval point; the tangent line of the line connecting the two adjacent interval points is taken as the interval line; then the region between the two interval points, i.e. an interval, is taken as an interval.

[0110] Then, based on the coordinates of each sampling point in the current frame, the sampling points falling in each interval are determined; then for each interval, the average point of all sampling points falling in the interval, i.e. the point corresponding to the average value of the coordinates, is calculated; finally, the current position of the vehicle is fused by the GPS, IMU and wheel speed sensor on the autonomous vehicle, and the average point in each interval is converted to the global coordinate system according to the current position of the vehicle, as the center point of the point chain.

[0111] Step S304: constructing the point chain of the current frame based on the center point.

[0112] In the process of implementing step S304, when a new interval point is obtained by executing step S302, it is also necessary to determine in real time whether the coordinate of the new interval point obtained by step S302 is greater than or equal to the perception end position in the perception range of the corresponding image. When the coordinate of the interval point is greater than or equal to the perception end position, the generation of interval and center points is stopped, that is, the execution of steps S302 and S303 is stopped at this time, the point chain of the current frame is constructed by using all the obtained center points, and is converted to the global coordinate system for storage. Otherwise, when the coordinate of the interval point is less than the perception end position, step S302 is returned to construct a new interval point for the next image of the current frame whose confidence is greater than the preset threshold.

[0113] It should be noted that the point chain of the current frame and the current position of the vehicle corresponding to the current frame are stored.

[0114] Step S305: Obtain a historical point chain and a historical interval point corresponding thereto.

[0115] In the process of implementing step S305, the current position of the vehicle collected by the vehicle-mounted position sensor is collected; and the center point, interval and sampling point of the historical point chain are converted to the ego coordinate system by using the current position of the vehicle.

[0116] It should be noted that the historical point chain is a point chain iterated from all frames before the current frame (i.e., before the current frame) in the current driving of the vehicle.

[0117] Step S306: Determine whether the sampling point of the current frame and the historical point chain have an overlapping area and are in the same lane line. If it is determined that the sampling point of the current frame and the historical point chain have an overlapping area and are in the same lane line, step S307 is executed. If the sampling point of the current frame and the historical point chain do not have an overlapping area or are not in the same lane line, the point chain of the current frame is generated according to the process of steps S302 to S304.

[0118] It should be noted that the process of implementing step S306 includes the following steps:

[0119] Step S41: Determine whether the sampling point of the current frame and the historical point chain have an overlapping area. If yes, step S42 is executed. If no, the point chain of the current frame is generated according to the process of steps S302 to S304.

[0120] In the implementation of step S41, in the case that the lane line currently perceived by the vehicle is in the same direction as the driving direction, by comparing the abscissa x of the sampling points of the current frame with the abscissa x of the center points of the historical point chain, if there is at least one or more sampling points of the current frame whose abscissa x is within the abscissa x of any two center points, i.e., the abscissa of at least one or more sampling points is greater than the abscissa x1 of one center point and less than the abscissa x2 of another center point, it is indicated that the sampling points and the historical point chain have an overlapping area in the driving direction of the vehicle, as shown in FIG. 8, and step S42 is performed, otherwise, the point chain of the current frame is generated according to the process of steps S302 to S304. Figure 4

[0121] It should be noted that the sampling points whose abscissa is greater than the abscissa x1 of one center point and less than the abscissa x2 of another center point are taken as the sampling points of the overlapping area.

[0122] Step S42: determining whether the sampling points of the current frame and the historical point chain are in the same lane line, if yes, step S307 is performed, if not, the point chain of the current frame is generated according to the process of steps S302 to S304.

[0123] In the implementation of step S42, the distance of each sampling point in the overlapping area to the line connecting the nearest two center points is calculated, the average distance is obtained by averaging all distances, each average distance is compared, the smallest average distance is selected, and it is determined whether the smallest average distance is greater than a first threshold value, if yes, it is indicated that the sampling points of the current frame and the historical point chain come from the same lane line, at this time, it is indicated that the lane line tracking is successful, and step S307 is performed, otherwise, the point chain of the current frame is generated according to the process of steps S302 to S304.

[0124] Optionally, before step S42 is performed, the following step is further included:

[0125] It is determined whether the overlapping area satisfies a preset length, generally, when the number of sampling points of the overlapping area is greater than a second threshold value, it is indicated that the overlapping area satisfies the preset length, at this time, step S42 is performed, so that the lane line tracking is more accurate.

[0126] It should be noted that the first threshold value, the preset length, and the second threshold value are all set by the technicians according to multiple experiments.

[0127] Step S307: fusing the sampling points of the current frame and the historical point chain to obtain the point chain of the current frame.

[0128] It should be noted that the process of implementing step S307 includes the following steps:

[0129] ​Step S51: traversing the intervals corresponding to the historical interval points, to determine the sampling points in the current frame that are in the intervals corresponding to the historical interval points.

[0130] In the process of implementing step S51, since there is an overlapping area, a certain sampling point in the current frame, i.e., a sampling point in the overlapping area, is in the interval corresponding to the historical interval point. Therefore, the sampling points in the current frame that are in each interval corresponding to the historical interval points are determined by traversing the intervals corresponding to the historical interval points.

[0131] Step S52: updating the historical point chain based on the sampling points in the current frame that are in the intervals corresponding to the historical interval points and the center points in the intervals corresponding to the historical interval points.

[0132] In the process of implementing step S52, first, for each interval in the historical point chain in which there is a sampling point in the current frame, the average value of each sampling point in the current frame that falls in the interval is calculated based on the coordinates of each sampling point in the current frame and the coordinates of the center point in the interval corresponding to the historical interval point. Then, the average value of the average value and the average value of the center point in the interval corresponding to the historical interval point is calculated to obtain the first center point (x s , y s ).

[0133] In addition to the above method, the sampling points falling in the interval can also be determined based on the coordinates of each sampling point in the current frame and the coordinates of each sampling point in the historical point chain.

[0134] Then, the average point of all the sampling points falling in the interval is calculated and taken as the first center point (x s , y s ).

[0135] Then, the weight of the first center point (x s , y s ) and the weight of the center point (x m , y m ) corresponding to the same interval in the historical point chain are calculated.

[0136] Specifically, the relative distance s of the first center point converted to the ego vehicle coordinate system from the current position of the vehicle is calculated first, and the relative distance s is substituted into formula (3) to calculate the weight w s of the first center point corresponding to the historical interval point. Then, the weight w m of the center point (x m , y m ) in the historical point chain corresponding to the same interval is calculated in the above manner.

[0137] Then, the first center point (xs , y s ) and the coordinates (x m , y m ) of the center point corresponding to the same interval in the history point chain, and their respective weights w s and w m are substituted into formula (4) to update the coordinates of the first center point;

[0138] Finally, the center point of the overlapping area in the history point chain is replaced based on the updated first center point to update the history point chain.

[0139] Formula (3):

[0140]

[0141] wherein w is the weight of each center point, c is the width coefficient of the Gaussian function, which is pre-set; s is the distance of the center point relative to the vehicle.

[0142] Formula (4):

[0143]

[0144] wherein (x s , y s ) is the coordinates of the first center point, i.e. the average point, (x m , y m ) is the coordinates of the center point in the interval, w s and w m are the weights of the average point and the center point, is the center point coordinates after weighted fusion, i.e. the coordinates of the first center point after updating.

[0145] Step S53: processing the road information corresponding to each image under the current frame and the preset range based on each image under the current frame with a confidence greater than a preset threshold, to determine the new interval point corresponding to each remaining sampling point under the current frame;

[0146] In the process of specifically implementing step S53, first, each sampling point outside the interval corresponding to the history interval point under the current frame, i.e. each remaining sampling point under the current frame; first determine each image under the current frame corresponding to each remaining sampling point, and execute step S302 based on each image, that is, for each remaining sampling point under the current frame, select one interval point closest to the sampling point from the history interval points, and take it as the initial interval point, and determine the new interval point corresponding to each remaining sampling point under the current frame through the way of step S302.

[0147] Step S54: calculating an average point of the sampling points in each two new interval points as a new center point based on each sampling point and the new interval point remaining in the current frame;

[0148] The new interval point is an interval point corresponding to each sampling point remaining in the current frame.

[0149] It should be noted that when the sampling point does not fall in the interval region of the interval corresponding to the history point chain, the closest interval point to the sampling point is selected from the history interval points as the initial interval point, and the new center point is generated by the above specific implementation step S303.

[0150] Optionally, after step S54 is performed, when a new interval point is obtained in step S53, it is further needed to determine in real time whether the coordinate of the new interval point obtained in step S53 is greater than the perception end position in the perception range of the corresponding image, and when the coordinate of the interval point is greater than the perception end position, the generation of the interval and the center point is stopped.

[0151] Step S55: constructing the point chain of the current frame based on the updated history point chain and the new center point.

[0152] In the process of implementing step S55, the new center point obtained in step S54 is constructed into a first point chain, then the updated history point chain and the first point chain are spliced to obtain the point chain of the current frame, and the point chain is stored.

[0153] Optionally, the vehicle controller controls the point chain construction module to send the processed point chain of the current frame to the local map generator.

[0154] Step S205: generating a local map based on the point chain corresponding to each current frame.

[0155] It should be noted that in the process of implementing step S205 by the vehicle controller controlling the local map generator, the following steps are included:

[0156] Step S61: determining whether the current frame is the first frame, if yes, executing step S62, and if no, executing step S66.

[0157] It should be noted that the process of implementing step S61 is the same as the process of implementing step S301, and they can be referred to each other.

[0158] Step S62: obtaining the current position of the vehicle.

[0159] In the process of implementing step S62, the vehicle controller obtains the current position of the vehicle collected by the vehicle position sensor.

[0160] Step S63: calculating the distance between the center point and the current position for each center point under the point chain of the current frame;

[0161] It should be noted that the center point is a coordinate in the global coordinate system, so the center point coordinate in the global coordinate system is converted into a coordinate in the vehicle coordinate system first, and then the distance s of the center point relative to the vehicle is calculated.

[0162] The vehicle coordinate system refers to the x-axis positive direction as the vehicle driving direction, and the y-axis is perpendicular to the x-axis positive direction to the left.

[0163] The converted center point coordinate is generated by the x-coordinate.

[0164] Step S64: calculating the weight of the center point based on the distance.

[0165] In the process of specifically implementing step S64, the weight w of each center point is calculated using a Gaussian function, i.e., formula (3), according to the distance s of the center point relative to the vehicle:

[0166] Step S65: deleting the center point with a weight less than a preset weight threshold, and constructing a local map based on the remaining center points of the current frame.

[0167] In the process of specifically implementing step S65, each center point of the current point chain is traversed, and the weight of each center point is compared to delete the center point with a weight less than a preset weight threshold; then, the remaining center points of the current frame are disconnected into two point chains according to the current position of the vehicle, and the point chain with a center point number less than 2 is deleted to obtain a local map result, as shown in Figure 5

[0168] It should be noted that the local map result contains center points capable of representing the shape of the lane line.

[0169] It should be noted that the preset weight threshold is set by the technician according to multiple experiments.

[0170] Step S66: obtaining the center points of the current frame and the center points of the historical point chain in the overlapping area based on the point chain of the current frame and the historical point chain, wherein the overlapping area refers to the overlapping part of the sampling points of the current frame and the historical point chain;

[0171] Step S67: updating the weight of the center point of the current frame based on the weight of the center point of the current frame and the center point of the historical point chain.

[0172] In the process of specifically implementing steps S66 and S67, the center points of the current frame in the overlapping area, i.e., the first center points, are obtained from the point chain of the current frame; then, the center points of the historical point chain in the overlapping area are obtained from the historical point chain; the weight w of the first center point is updated based on the weight w of the first center point and the weight w of the second center point. s ​and the center point of the historical point chain in the overlapping area, i.e. the weight w of the center point corresponding to the same interval in the historical point chain m The formula (5) is substituted and processed to update the weight of the center point of the current frame in the overlapping area .

[0173] The formula (5) is substituted and processed to update the weight of the center point of the current frame in the overlapping area

[0174]

[0175] Step S67: For the remaining center points under the current frame, the weight of the center point is calculated based on the distance between the remaining center point and the current position.

[0176] The remaining center points under the current frame refer to the center points under the current frame except for the center points corresponding to the sampling points in the overlapping area.

[0177] It should be noted that the process of implementing step S67 is the same as the process of implementing steps S63 and S64, and can be referred to each other.

[0178] Step S68: Based on the weight of the center point of the current frame in the overlapping area after updating, and the weight of the remaining center points, the center points with a weight less than a preset weight threshold are deleted, and a local map is constructed based on the remaining center points under the current frame.

[0179] In the process of implementing step S68, the weight of each center point obtained in steps S66 and S67 is compared with the weight of the center point of the current frame in the overlapping area after updating, and the weight of the remaining center points, i.e. the weight of each center point, to delete the center points with a weight less than a preset weight threshold, to obtain the final remaining center points; then, the final remaining center points are disconnected into two point chains according to the current position of the vehicle, and the point chains with a number of center points less than 2 are deleted, to obtain a local map result.

[0180] It should be noted that the local map result contains center points capable of representing the shape of the lane line.

[0181] Optionally, based on the method shown in steps S201 to S205, the flowchart of the local construction is implemented, which can be as shown in Figure 6 .

[0182] In the embodiment of the present application, the road information around the vehicle is collected by a visual sensor, specifically including identifying the perception range of the lane line in the camera coordinate system and the lane line parameters, etc.; the current position of the vehicle is fused by the GPS, IMU and wheel speed sensor on the autonomous vehicle; the sampling points are extracted according to the lane line parameters; if the current frame is not the first frame, the current frame sampling points are tracked with the historical point chain in the ego coordinate system by using the current position of the vehicle, the sampling points successfully tracked are fused with the point chain, and the sampling points unsuccessfully tracked generate new point chains. Compared with single-frame perception, the local map composed of the point chains can more accurately express the position and shape of the lane line, is less affected by abnormal perception, and can also deal with the scene of lane line occlusion disappearance because the results of historical frames are fused.

[0183] Based on the vehicle local map generation method shown in the above embodiment of the present application, correspondingly, a vehicle local map generation device is also shown in the embodiment of the present application, as shown in Figure 7 The device comprises:

[0184] The acquisition unit 701 is configured to acquire images in front of the vehicle in real time.

[0185] The identification unit 702 is configured to identify the road information based on each image of each frame.

[0186] The construction unit 703 is configured to construct a plurality of sampling points of the current frame based on the road information, and the sampling points have a preset distance; and construct a point chain of the current frame based on the road information and the plurality of sampling points of the current frame.

[0187] The generation unit 704 is configured to generate a local map based on the point chain corresponding to each current frame.

[0188] The specific principles and execution processes of each unit in the vehicle local map generation device disclosed in the above embodiment of the present application are the same as the corresponding contents in the vehicle local map generation method provided in the above embodiment of the present application, and can be referred to the corresponding parts in the vehicle local map generation method disclosed in the above embodiment of the present application, which will not be described here.

[0189] In the embodiment of the present application, the perception range of the lane line in the camera coordinate system and the lane line parameters are identified; the sampling points are extracted according to the lane line parameters; if the current frame is not the first frame, the current frame sampling points are tracked with the historical point chain in the ego coordinate system by using the current position of the vehicle, the sampling points successfully tracked are fused with the point chain, and the sampling points unsuccessfully tracked generate new point chains. Compared with single-frame perception, the local map composed of the point chains can more accurately express the position and shape of the lane line, is less affected by abnormal perception, and can also deal with the scene of lane line occlusion disappearance because the results of historical frames are fused.

[0190] Optionally, based on the vehicle local map generation device shown in the above embodiment of the application, the construction unit 703 for constructing a plurality of sampling points of the current frame based on the road information is specifically configured to:

[0191] For each frame of image, it is determined whether there is a confidence value greater than a preset threshold in the road information corresponding to each image.

[0192] If there is, the initial sampling points of the current frame are generated based on the road information with the confidence value greater than the preset threshold.

[0193] The initial sampling points are processed according to a preset distance to obtain a plurality of sampling points constructed by the preset distance for the current frame.

[0194] Optionally, based on the vehicle local map generation device shown in the above embodiment of the application, the construction unit 703 for constructing a point chain of the current frame based on the road information and the plurality of sampling points of the current frame is specifically configured to:

[0195] If it is determined that the current frame is the first frame, the lane line parameters and the perception information in the road information are processed based on a preset range to obtain interval points.

[0196] The average point of the sampling points in each two adjacent interval points is calculated based on each sampling point under the current frame and the interval points, and the average point is taken as the center point of the point chain.

[0197] The point chain of the current frame is constructed based on the center point, and the point chain is saved.

[0198] Optionally, based on the vehicle local map generation device shown in the above embodiment of the application, the construction unit 703 for constructing a point chain of the current frame based on the road information and the plurality of sampling points of the current frame is specifically configured to:

[0199] If it is determined that the current frame is not the first frame, the historical point chain and the corresponding historical interval points are obtained.

[0200] If it is determined that the sampling points of the current frame and the historical point chain have an overlapping area and are in the same lane line, the sampling points of the current frame and the historical point chain are fused to obtain the point chain of the current frame.

[0201] Optionally, based on the vehicle local map generation device shown in the above embodiment of the application, the construction unit 703 for constructing a point chain of the current frame by fusing the sampling points of the current frame and the historical point chain is specifically configured to:

[0202] The sampling points of the current frame in the interval corresponding to the historical interval points are determined by traversing the interval corresponding to the historical interval points.

[0203] updating the history point chain based on the sampling points in the interval corresponding to the history interval point under the current frame and the center point in the interval corresponding to the history interval point;

[0204] processing the road information corresponding to each image with a confidence greater than a preset threshold and a preset range under the current frame to determine a new interval point corresponding to each remaining sampling point under the current frame;

[0205] calculating the average point of the sampling points in each two interval points based on the remaining sampling points under the current frame and the new interval point, and taking the average point as a new center point;

[0206] constructing the point chain of the current frame based on the updated history point chain and the center point.

[0207] Optionally, based on the vehicle local map generation device shown in the above embodiment of the present application, the generation unit 704 is specifically configured to:

[0208] if it is determined that the current frame is the first frame, obtaining the current position of the vehicle;

[0209] calculating the distance between each center point in the point chain of the current frame and the current position;

[0210] calculating the weight of the center point based on the distance;

[0211] deleting the center point with a weight less than a preset weight threshold, and constructing a local map based on the remaining center points of the current frame.

[0212] Optionally, based on the vehicle local map generation device shown in the above embodiment of the present application, the generation unit 704 is further configured to:

[0213] if it is determined that the current frame is not the first frame, obtaining the center points of the current frame and the center points of the history point chain in the overlap region based on the point chain of the current frame and the history point chain, the overlap region being the overlapping part of the sampling points of the current frame and the history point chain;

[0214] updating the weight of the center points of the current frame in the overlap region based on the weight of the center points of the current frame and the center points of the history point chain;

[0215] calculating the weight of the remaining center points of the current frame based on the distance between the remaining center points and the current position;

[0216] deleting the center point with a weight less than a preset weight threshold based on the updated weight of the center points of the current frame in the overlap region and the weight of the remaining center points, and constructing a local map based on the remaining center points of the current frame.

[0217] The electronic device provided by the embodiment of the present application includes a processor and a memory. The memory is used to store vehicle local map generation control program codes and data. The processor is used to call program instructions in the memory to perform steps shown in the vehicle local map generation method in the above embodiments.

[0218] The vehicle provided by the embodiment of the present application includes the electronic device provided by the embodiment of the present application, which is used to perform the vehicle local map generation method disclosed by the embodiment of the present application.

[0219] Each of the embodiments in the specification is described in a progressive manner, and the same and similar parts of each embodiment can be referred to each other. Each embodiment focuses on the difference from other embodiments. In particular, the system or system embodiment is basically similar to the method embodiment, so the description is relatively simple, and the relevant part can be referred to the part of the method embodiment. The system and system embodiment described above are only illustrative, and the units described as separate components can be or can not be physically separated, and the components displayed as units can be or can not be physical units, that is, they can be located in one place or distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the embodiment scheme according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0220] The skilled person can further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized by electronic hardware, computer software or combination of the two. In order to clearly show the interchangeability of hardware and software, the components and steps of each example have been described in the above description. Whether the functions are realized by hardware or software depends on the specific application and design constraints of the technical scheme. The skilled person can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0221] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to the embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for generating a local vehicle map, characterized in that, The method includes: Real-time acquisition of images of the area in front of the vehicle; For each frame of the image, road information is obtained by performing recognition based on the image; Multiple sampling points for the current frame are constructed based on the road information, and the sampling points are spaced at a preset distance. Based on the road information and multiple sampling points of the current frame, a point chain is constructed to obtain the point chain of the current frame; A local map is generated based on the point chain corresponding to each current frame.

2. The method according to claim 1, characterized in that, Based on the road information, multiple sampling points for the current frame are constructed, including: For each frame of the image, determine whether there is a confidence level greater than a preset threshold in the road information corresponding to each image; If they exist, the initial sampling points for the current frame are generated based on the road information with a confidence level greater than a preset threshold; The initial sampling points are processed according to a preset distance to obtain multiple sampling points in the current frame constructed from the preset distance.

3. The method according to claim 1, characterized in that, Based on the road information and multiple sampling points of the current frame, a point chain is constructed to obtain the point chain of the current frame, including: If the current frame is determined to be the first frame, the interval points are obtained by processing based on the lane line parameters and perception information in the road information and a preset range; Based on each sampling point in the current frame and the interval point, calculate the average point of the sampling points within every two adjacent interval points, and use it as the center point of the point chain; Construct a point chain for the current frame based on the center point.

4. The method according to claim 1, characterized in that, Based on the road information and multiple sampling points of the current frame, a point chain is constructed to obtain the point chain of the current frame, including: If it is determined that the current frame is not the first frame, obtain the historical point chain and its corresponding historical interval point; If it is determined that the sampling points of the current frame overlap with the historical point chain and are located on the same lane line, the sampling points of the current frame are fused with the historical point chain to obtain the point chain of the current frame.

5. The method according to claim 4, characterized in that, The sampling points of the current frame are fused with the historical point chain to obtain the point chain of the current frame, including: Traverse the intervals corresponding to the historical interval points to determine the sampling points in the current frame that are within the intervals corresponding to the historical interval points; The historical point chain is updated based on the sampling points in the current frame that are within the interval corresponding to the historical interval point, and the center points within the interval corresponding to the historical interval point. Based on the road information and preset range corresponding to each image with a confidence level greater than a preset threshold in the current frame, the new interval point corresponding to each remaining sampling point in the current frame is determined. Based on each remaining sampling point in the current frame and the new interval point, calculate the average point of the sampling points within every two new interval points, and use it as the new center point; Based on the updated historical point chain and the new center point, construct the point chain for the current frame.

6. The method according to claim 1 or 3, characterized in that, Generate a local map based on the point chain corresponding to each current frame, including: If the current frame is determined to be the first frame, obtain the vehicle's current position; For each center point in the point chain of the current frame, calculate the distance between the center point and the current position; Calculate the weight of the center point based on the distance; Center points with weights less than a preset weight threshold are deleted, and a local map is constructed based on the remaining center points in the current frame.

7. The method according to claim 1 or 5, characterized in that, Generate a local map based on the point chain corresponding to each current frame, including: If it is determined that the current frame is not the first frame, the center point of the current frame and the center point of the historical point chain within the overlapping area are obtained based on the point chain of the current frame and the historical point chain. The overlapping area refers to the part where the sampling points of the current frame and the historical point chain overlap. The weight of the center point of the current frame within the overlapping region is updated based on the weights of the center point of the current frame and the center points of the historical point chain. For the remaining center points in the current frame, calculate the weight of the remaining center points based on the distance between the remaining center points and the current position; based on the weight of the center points of the current frame in the updated overlapping area and the weight of the remaining center points, delete center points with weights less than a preset weight threshold, and construct a local map based on the remaining center points of the current frame.

8. A vehicle local map generation device, characterized in that, The device includes: The acquisition unit is used to acquire images of the area in front of the vehicle in real time. The recognition unit is used to identify each image in each frame to obtain road information; The construction unit is used to construct multiple sampling points of the current frame based on the road information, wherein there is a preset distance between the sampling points; and to construct a point chain based on the road information and the multiple sampling points of the current frame to obtain the point chain of the current frame. The generation unit is used to generate a local map based on the point chain corresponding to each current frame.

9. An electronic device, characterized in that, The electronic device includes a processor and a memory, the memory being used to store program code and data for data generation, and the processor being used to call program instructions in the memory to execute the vehicle local map generation method as described in any one of claims 1-7.

10. A vehicle, characterized in that, The vehicle includes a stored program, wherein, when the program is executed, it controls devices within the vehicle to perform the vehicle local map generation method as described in any one of claims 1-7.

Images