Lane line splicing method, device, equipment, storage medium and product

Abstract

The application discloses a lane line splicing method, device, equipment, storage medium and product, relates to the automatic driving technical field, discloses a lane line splicing method, comprising: obtaining the target coordinate point set of target lane line in the current road image frame and the cumulative boundary range of the target lane line;Determine the first coordinate point in the cumulative boundary range and the second coordinate point outside the cumulative boundary range in the target coordinate point set;The lane line corresponding to the first coordinate point is removed, and the splicing of the target lane line is carried out based on the second coordinate point, and the lane line data after splicing is obtained.The application removes the repeated coordinate points of the target lane line in the new road image frame through the cumulative boundary range of the target lane line, avoids the situation that the coordinate sequence of the same lane line in different frames is inconsistent in direction, and then improves the accuracy of lane line identification.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automatic driving, in particular to a lane line splicing method, device, equipment, storage medium and product. BACKGROUND

[0002] In the field of intelligent transportation systems, accurate lane perception is the key to achieving safe and efficient navigation.

[0003] In the related art, image processing and machine learning algorithms are usually used to extract lane line features in road images to identify complete lane lines in road images. However, due to the dynamic changes of the environment in the road image, the coordinate sequences of the same lane line in different frames may be inconsistent in direction, resulting in low accuracy of lane line recognition.

[0004] The above content is only used to assist in understanding the technical solutions of the present application and does not represent the acknowledgement of the above content as prior art. SUMMARY

[0005] The main purpose of the present application is to provide a lane line splicing method, which aims to solve the technical problem that the coordinate sequences of the same lane line in different frames may be inconsistent in direction, resulting in low accuracy of lane line recognition.

[0006] To achieve the above purpose, the present application provides a lane line splicing method, which comprises:

[0007] obtaining a target coordinate point set of a target lane line in a current road image frame and a cumulative boundary range of the target lane line;

[0008] determining a first coordinate point in the target coordinate point set within the cumulative boundary range and a second coordinate point outside the cumulative boundary range;

[0009] de-duplicating the lane line corresponding to the first coordinate point, and splicing the target lane line based on the second coordinate point to obtain spliced lane line data.

[0010] Optionally, after the step of de-duplicating the lane line corresponding to the first coordinate point and splicing the target lane line based on the second coordinate point to obtain spliced lane line data, the method comprises:

[0011] determining the maximum coordinate value of each coordinate system corresponding to each second coordinate point;

[0012] updating the cumulative boundary range of the target lane line based on the maximum coordinate value of each coordinate system to obtain an updated cumulative boundary range, wherein the updated cumulative boundary range is used to implement splicing of the target lane line in the next road image frame.

[0013] Optionally, the step of obtaining the target coordinate point set of the target lane line in the current road image frame comprises:

[0014] obtaining a current road image frame;

[0015] extracting a vehicle position information and an initial coordinate point set of the target lane line from the current road image frame;

[0016] converting the initial coordinate point set of the target lane line from a global coordinate system to a local coordinate system with the vehicle as the origin based on the vehicle position information, to obtain a target coordinate point set of the target lane line.

[0017] Optionally, the step of converting the initial coordinate point set of the target lane line from the global coordinate system to the local coordinate system with the vehicle as the origin based on the vehicle position information, to obtain the target coordinate point set of the target lane line comprises:

[0018] extracting a vehicle coordinate value and a vehicle orientation angle in the vehicle position information;

[0019] subtracting the coordinate value corresponding to each initial coordinate point in the initial coordinate point set of the target lane line from the vehicle coordinate value to calculate a coordinate translation value of each initial coordinate point;

[0020] calculating a rotation angle of the target lane line based on the vehicle orientation angle;

[0021] converting the initial coordinate point set of the target lane line from the global coordinate system to the local coordinate system with the vehicle as the origin based on the coordinate translation value and the rotation angle, to obtain the target coordinate point set of the target lane line.

[0022] Optionally, after the steps of obtaining the target coordinate point set of the target lane line in the current road image frame and the cumulative boundary range of the target lane line, the method comprises:

[0023] constructing a global grid;

[0024] mapping each coordinate point in the target coordinate point set to the global grid to determine a corresponding perception map of the current road image frame;

[0025] The step of determining the first coordinate point in the coordinate point set within the cumulative boundary range and the second coordinate point outside the cumulative boundary range comprises:

[0026] determining the first coordinate point in the coordinate point set within the cumulative boundary range and the second coordinate point outside the cumulative boundary range based on the perception map.

[0027] Optionally, the step of mapping each coordinate point in the target coordinate point set into the global grid to determine the perception map corresponding to the current road image frame further comprises any one of the following:

[0028] mapping each coordinate point in the target coordinate point set into a corresponding position in the grid to obtain an initial grid, performing a first thinning operation on the initial grid to obtain a target grid, and determining the perception map corresponding to the current road image frame based on the target grid;

[0029] mapping each coordinate point in the target coordinate point set into a corresponding position in the grid to obtain an initial grid, performing a first thinning operation on the initial grid, and performing a second thinning operation on the grid after the first thinning operation to obtain a target grid, wherein the thinning operation includes a floor operation or a rounding operation.

[0030] In addition, to achieve the above object, the present application further provides a lane line splicing device, which comprises:

[0031] an acquisition module, configured to acquire a target coordinate point set of a target lane line in a current road image frame and a cumulative boundary range of the target lane line;

[0032] a determination module, configured to determine a first coordinate point in the target coordinate point set within the cumulative boundary range and a second coordinate point outside the cumulative boundary range;

[0033] a de-duplication module, configured to perform de-duplication processing on the lane line corresponding to the first coordinate point, and perform splicing of the target lane line based on the second coordinate point to obtain spliced lane line data.

[0034] In addition, to achieve the above object, the present application further provides a lane line splicing device, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the lane line splicing method as described above.

[0035] In addition, to achieve the above object, the present application further provides a storage medium, which is a computer readable storage medium, and the storage medium stores a computer program, wherein the computer program is executed by a processor to implement the steps of the lane line splicing method as described above.

[0036] In addition, to achieve the above object, the present application further provides a computer program product, which comprises a computer program, wherein the computer program is executed by a processor to implement the steps of the lane line splicing method as described above.

[0037] The one or more technical solutions provided in the application have at least the following technical effects:

[0038] In the related art, lane line features in a road image are usually extracted by using image processing and machine learning algorithms, and a complete lane line is recognized and spliced. However, due to dynamic changes in the environment during vehicle driving, the coordinate sequences of the same lane line in different frames may be inconsistent in direction, resulting in low accuracy of lane line recognition. In comparison, the embodiments of the application obtain a target coordinate point set of a target lane line in a current road image frame and a cumulative boundary range of the target lane line; determine a first coordinate point in the cumulative boundary range and a second coordinate point outside the cumulative boundary range in the target coordinate point set; perform deduplication processing on the lane line corresponding to the first coordinate point, and splice the target lane line based on the second coordinate point to obtain spliced lane line data. It can be understood that, first, the cumulative boundary range of the target lane line needs to be obtained, wherein the boundary range is determined according to the coordinate points of the target lane line in all previously accumulated image frames. Second, each time a new road image frame is received, the coincident coordinate points (i.e., the first coordinate points) and the non-coincident coordinate points (i.e., the second coordinate points) between the target coordinate point set of the target lane line in the road image frame and the cumulative boundary range are determined, wherein the coincident coordinate points will cause the coordinate sequence of the lane center line to be abnormal, so deduplication is performed, and then the target lane line is spliced according to the second coordinate points, which can avoid the situation that the coordinate sequences of the same lane line in different frames are inconsistent in direction. That is, the repeated coordinate points of the target lane line in the new road image frame are removed by the cumulative boundary range of the target lane line, ensuring that the coordinate sequence of the target lane line will not have reverse points due to repeated data, thereby improving the accuracy of lane line recognition. BRIEF DESCRIPTION OF DRAWINGS

[0039] The accompanying drawings, which are incorporated into and form part of the specification, illustrate embodiments consistent with the application and, together with the specification, serve to explain the principles of the application.

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the accompanying drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, those skilled in the art can obtain other drawings from these drawings without creative labor.

[0041] Figure 1 A flowchart is provided for the first embodiment of the lane line splicing method of the application;

[0042] Figure 2 A flowchart is provided for the second embodiment of the lane line splicing method of the application;

[0043] Figure 3 A module structure diagram of a lane line splicing device according to an embodiment of the present application is shown in FIG. 1.

[0044] Figure 4 A device structure diagram of a hardware running environment involved in a lane line splicing method according to an embodiment of the present application is shown in FIG. 2.

[0045] The object, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0046] It should be understood that the specific embodiments described herein are merely intended to explain the technical solutions of the present application, and are not intended to limit the present application.

[0047] In order to better understand the technical solutions of the present application, the specific embodiments will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0048] The main solution of the embodiment of the present application is: obtaining a target coordinate point set of a target lane line in a current road image frame and a cumulative boundary range of the target lane line; determining a first coordinate point in the target coordinate point set within the cumulative boundary range and a second coordinate point outside the cumulative boundary range; performing a deduplication processing on the lane line corresponding to the first coordinate point, and performing a splicing of the target lane line based on the second coordinate point, to obtain spliced lane line data.

[0049] In the embodiment, the lane line splicing device is taken as the execution subject, and for the convenience of description, the following is referred to as "device".

[0050] In the related art, the lane line features in the road image are usually extracted by using image processing and machine learning algorithms, and the complete lane line is recognized and spliced. However, due to the dynamic changes of the environment during vehicle driving, the coordinate sequences of the same lane line in different frames may appear inconsistent in direction, resulting in low accuracy of lane line recognition.

[0051] The present application provides a solution to realize lane line splicing and improve the accuracy of lane line recognition.

[0052] From the above embodiments, it can be seen that the present application removes the repeated coordinate points of the target lane line in the new road image frame through the cumulative boundary range of the target lane line, avoids the situation that the coordinate sequences of the same lane line in different frames may appear inconsistent in direction, and improves the accuracy of lane line recognition.

[0053] It should be noted that the execution subject of the embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone or the like, or an electronic device, a terminal system or the like capable of realizing the above functions. The lane line splicing device is taken as an example to describe the embodiment and the following embodiments.

[0054] Based on this, the application provides a lane line splicing method, which refers to Figure 1 , Figure 1 FIG. 1 is a flowchart of a first embodiment of the lane line splicing method of the application.

[0055] In the embodiment, the lane line splicing method comprises steps S100-S300:

[0056] In step S100, a target coordinate point set of a target lane line in a current road image frame and a cumulative boundary range of the target lane line are acquired.

[0057] It should be noted that the target coordinate point set refers to a coordinate point set on the lane line identified by a lane line detection algorithm in the current road image frame, wherein the target coordinate point can be an end point, an inflection point or a point sampled according to a certain rule, which collectively define the position and shape of the lane line in the image; the cumulative boundary range refers to a boundary box determined according to all the coordinate points of the target lane line in the image frames accumulated before, i.e., the horizontal and vertical range of the lane line in the image, wherein the cumulative boundary range can be understood as the minimum circumscribed rectangle or polygon of the lane line in the image, which contains all parts of the lane line and can be used to represent the global position and coverage range of the lane line.

[0058] In a specific implementation, the application proposes to define a dictionary screen_lane_center_polyline for storing different lane center line splicing information, wherein the dictionary comprises key values and value values, the key value is a lane center line index, and the value value is a coordinate point list of the lane center line appearing in a scene segment, and reverse point elimination is performed based on the cumulative boundary range. Further, the application proposes to define a dictionary lane_box_polygons for recording coordinate point spatial range boundary information of the cumulative different lane line from a starting frame to a current frame, wherein the key value is a lane center line index, and the value value is a coordinate point spatial range boundary of the lane center line accumulated from the starting frame to the current frame, which is represented by x_min, x_max, y_min and y_max.

[0059] In specific implementation, the device obtains a road image through receiving an information collection module on the vehicle, wherein the road image comprises a plurality of image frames, and in an embodiment, the device can use a deep learning model such as a convolutional neural network to identify a target lane line in the image frames and determine coordinate points and an accumulated boundary range of the target lane line.

[0060] In specific implementation, the device obtains a target coordinate point set of a target lane line in a current road image frame, comprising:

[0061] obtaining a current road image frame; extracting vehicle position information and an initial coordinate point set of a target lane line from the current road image frame; converting the initial coordinate point set of the target lane line from a global coordinate system to a local coordinate system with the vehicle as the origin based on the vehicle position information, to obtain a target coordinate point set of the target lane line.

[0062] In specific implementation, the device obtains an image of a road on which the vehicle travels at a current time from a vehicle-mounted camera, and the image contains visual information of the road, lane line, and environment around the vehicle. Further, the device can extract the accurate position and pose (including position coordinates and direction) of the vehicle on the road from the current road image frame through sensors (such as GPS, IMU, wheel speed sensor, etc.) of the vehicle or visual algorithms (such as feature matching, SLAM, etc.); the device can use a lane line detection algorithm (such as a deep learning-based model) to identify the target lane line from the image and extract a series of coordinate points on the target lane line to form an initial coordinate point set of the target lane line, which defines the position of the lane line in the image coordinate system.

[0063] In specific implementation, the device converts the initial coordinate point set of the target lane line from a global coordinate system to a local coordinate system with the vehicle as the origin. Specifically, this step involves coordinate transformation, and the purpose is to convert the points on the lane line from the global coordinate system (such as the road coordinate system) to the local coordinate system with the vehicle as the center. The purpose of this is to simplify subsequent processing, because in the local coordinate system, the position of the vehicle is fixed (origin), and the relative position relationship between the lane line and the vehicle becomes more intuitive. Further, the above conversion process needs to know the pose (position and direction) of the vehicle relative to the global coordinate system, and then apply geometric transformation (such as translation and rotation) to map the points on the lane line from the global coordinate system to the local coordinate system. This transformation can be achieved through matrix operations, including rotation matrix and translation vector, etc.

[0064] In specific implementation, after the conversion is completed, the device obtains a coordinate point set in the local coordinate system of the vehicle, which is the lane line coordinate point (i.e., the target coordinate point), and the target coordinate point describes the specific position and shape of the lane line relative to the vehicle.

[0065] In a specific implementation, the step of converting the initial coordinate point set of the target lane line from the global coordinate system to the local coordinate system with the vehicle as the origin based on the vehicle position information includes:

[0066] The vehicle coordinate value and the vehicle orientation angle in the vehicle position information are extracted, the coordinate translation value of each initial coordinate point is calculated by subtracting the vehicle coordinate value from the corresponding coordinate value of each initial coordinate point in the initial coordinate point set of the target lane line, the rotation angle of the target lane line is calculated based on the vehicle orientation angle, and the initial coordinate point set of the target lane line is converted from the global coordinate system to the local coordinate system with the vehicle as the origin based on the coordinate translation value and the rotation angle to obtain the target coordinate point set of the target lane line.

[0067] It should be noted that the vehicle coordinate is the position of the vehicle in the global coordinate system, which is usually represented as (x, y) coordinates, and the vehicle coordinate value is the value of the vehicle on the x-axis coordinate system and the value of the vehicle on the y-axis coordinate system; the vehicle orientation angle is the orientation of the vehicle relative to the global coordinate system, represented as an angle value, which is used to describe the pointing direction of the vehicle.

[0068] In a specific implementation, the device first obtains corresponding map data information (map_data_item) from the information of the road image frame. Then, the position information of the ego vehicle is extracted therefrom, including the X coordinate (ego_x), the Y coordinate (ego_y), and the orientation angle (ego_theta).

[0069] In a specific implementation, the coordinate point P on the target lane line lane_center is (global_x, global_y), and the coordinate position of any point P in the ego vehicle coordinate system is calculated as follows:

[0070] x_diff = global_x - ego_x;

[0071] y_diff = global_y - ego_y;

[0072] angle_cos = math.cos(ego_theta);

[0073] angle_sin = math.sin(ego_theta);

[0074] local_x = x_diff * angle_cos - y_diff * angle_sin;

[0075] local_y = x_diff * angle_sin + y_diff * angle_cos.

[0076] Wherein, x_diff is the translation value of point P in the x direction relative to the ego vehicle, specifically the difference between the global x coordinate of point P and the global x coordinate of the ego vehicle; y_diff is the translation value of point P in the y direction relative to the ego vehicle, specifically the difference between the global y coordinate of point P and the global y coordinate of the ego vehicle; angle_cos refers to the cosine value of the heading angle of the ego vehicle; angle_sin refers to the sine value of the heading angle of the ego vehicle; local_x refers to the x coordinate of point P in the local coordinate system, specifically the difference between the translation value in the x direction multiplied by the cosine value and the translation value in the y direction multiplied by the sine value; local_y refers to the y coordinate of point P in the local coordinate system, specifically the sum of the translation value in the x direction multiplied by the sine value and the translation value in the y direction multiplied by the cosine value.

[0077] In a specific implementation, the above conversion process is to first translate point P to the position of the ego vehicle, and then rotate according to the heading angle of the ego vehicle, and finally obtain the position of point P in the local coordinate system with the ego vehicle as the origin.

[0078] Step S200, determining a first coordinate point in the cumulative boundary range and a second coordinate point outside the cumulative boundary range in the target coordinate point set;

[0079] In a specific implementation, the first coordinate point refers to the coordinate point in the target coordinate point set that is within the cumulative boundary range, which can be understood as a coordinate point that appears in the current image frame and is repeated in the historical image frame. Since the repeated coordinate point will be repeatedly recorded in the coordinate sequence, it will cause confusion in the direction, thereby affecting the directionality of the entire coordinate sequence and generating reverse points. That is, the repeated coordinate point will cause the coordinate sequence of the lane center line to be abnormal. By removing the repeated coordinate point, it can be ensured that the coordinate sequence of the target lane line will not have reverse points due to repeated data.

[0080] Further, the second coordinate point refers to the coordinate point in the target coordinate point set that is outside the cumulative boundary range, which can be understood as the extension part of the lane line in the image.

[0081] Step S300, performing a de-duplication process on the lane line corresponding to the first coordinate point, and performing a splicing of the target lane line based on the second coordinate point to obtain the spliced lane line data.

[0082] In a specific implementation, the apparatus performs deduplication on the lane line corresponding to the first coordinate points, i.e., the apparatus removes or merges those repeated or redundant coordinate points, to ensure that the coordinate sequence of the target lane line does not have reverse points due to repeated data. The deduplication process can be achieved by checking the distance, angle or other geometric features between consecutive points to ensure the smoothness and continuity of the lane line.

[0083] Further, the apparatus stitches the target lane line based on the second coordinate points, i.e., the apparatus includes these points outside the boundary range into the representation of the lane line to expand the data of the lane line, wherein the stitching process involves coordinate transformation, interpolation or other geometric algorithms to ensure that the newly added points are geometrically consistent with the existing lane line data, thereby obtaining a continuous and complete lane line representation.

[0084] In a specific implementation, after the apparatus performs deduplication on the lane line corresponding to the first coordinate points and stitches the target lane line based on the second coordinate points to obtain the stitched lane line data, the method comprises:

[0085] determining the maximum coordinate values of each coordinate system corresponding to each second coordinate point; and updating the cumulative boundary range of the target lane line based on the maximum coordinate values of each coordinate system, to obtain an updated cumulative boundary range, wherein the updated cumulative boundary range is used to implement the stitching of the target lane line in the next road image frame.

[0086] In a specific implementation, the apparatus determines the maximum coordinate values of each coordinate system corresponding to each second coordinate point, i.e., during the lane line detection process, for the second coordinate points outside the cumulative boundary range (i.e., those points beyond the current lane line boundary), the apparatus needs to determine their maximum coordinate values in the respective coordinate systems. These coordinate values include the maximum x value, the maximum y value, etc., which represent the extension range of the lane line in each direction.

[0087] Further, the maximum coordinate values obtained in the previous step are used to update the cumulative boundary range of the target lane line, wherein the updated cumulative boundary range will be used for the stitching of the lane line in the next frame of road image.

[0088] It can be understood that the lane line stitching method is preferably used in a long straight driving scenario, and by continuously accumulating the boundary range, the vehicle can be stitched on the lane center line of the long straight line.

[0089] In the related art, lane line features in a road image are usually extracted by using image processing and machine learning algorithms, and a complete lane line is recognized and spliced. However, due to dynamic changes in the environment during vehicle driving, the coordinate sequences of the same lane line in different frames may be inconsistent in direction, resulting in low accuracy of lane line recognition. In comparison, the embodiments of the present application obtain a target coordinate point set of a target lane line in a current road image frame and a cumulative boundary range of the target lane line; determine a first coordinate point in the cumulative boundary range and a second coordinate point outside the cumulative boundary range in the target coordinate point set; perform deduplication processing on the lane line corresponding to the first coordinate point, and splice the target lane line based on the second coordinate point to obtain spliced lane line data. It can be understood that, first, the present application needs to obtain the cumulative boundary range of the target lane line, wherein the boundary range is determined according to the coordinate points of the target lane line in all previously accumulated image frames. Second, each time a new road image frame is received, the present application determines the coincident coordinate points (i.e., the first coordinate points) and the non-coincident coordinate points (i.e., the second coordinate points) between the target coordinate point set of the target lane line in the road image frame and the cumulative boundary range, wherein the coincident coordinate points will cause the coordinate sequence of the lane center line to be abnormal, so deduplication is performed, and the target lane line is spliced according to the second coordinate points, which can avoid the situation that the coordinate sequences of the same lane line in different frames are inconsistent in direction. That is, by removing the duplicate coordinate points of the target lane line in the new road image frame through the cumulative boundary range of the target lane line, it is ensured that the coordinate sequence of the target lane line will not have reverse points due to duplicate data, thereby improving the accuracy of lane line recognition.

[0090] Based on the above first embodiment, the present application further proposes another embodiment, which is described with reference to Figure 2 The lane line splicing method comprises:

[0091] Step A100, obtaining a target coordinate point set of a target lane line in a current road image frame and a cumulative boundary range of the target lane line;

[0092] Step A100 is equivalent to step S100 described above, and will not be described here.

[0093] Step A200, constructing a global grid;

[0094] In a specific implementation, constructing a global grid refers to creating a global grid model in the process of constructing a perception map, which can cover the entire road environment. The global grid, as a basic framework of the map, can be used to integrate and align data from different sensors and different time points to construct a continuous and consistent map representation.

[0095] Step A300, mapping each coordinate point in the target coordinate point set to the global grid to determine the perception map corresponding to the current road image frame;

[0096] In a specific implementation, the device maps each coordinate point in the target coordinate point set to the global grid to determine the perception map corresponding to the current road image frame, i.e., converts each coordinate point on the lane line from its local coordinate in the image to a global coordinate in the global grid. Specifically, the device converts the points in the local coordinate system to the global coordinate system, so that these points can be understood and processed in a unified reference frame.

[0097] Further, by mapping these points to the global grid, the device can construct a perception map of the current road image frame, which contains the position, shape and other related road information of the lane line.

[0098] In a specific implementation, the step of mapping each coordinate point in the target coordinate point set to the global grid to determine the perception map corresponding to the current road image frame further includes any one of the following:

[0099] mapping each coordinate point in the target coordinate point set to the corresponding position in the grid to obtain an initial grid, performing a first thinning operation on the initial grid to obtain a target grid, and determining the perception map corresponding to the current road image frame based on the target grid;

[0100] mapping each coordinate point in the target coordinate point set to the corresponding position in the grid to obtain an initial grid, performing a first thinning operation on the initial grid, and performing a second thinning operation on the grid after the first thinning operation to obtain a target grid, wherein the thinning operation includes a floor operation or a rounding operation.

[0101] In a specific implementation, the device maps each coordinate point in the target coordinate point set to the corresponding position in the grid, i.e., the device converts each coordinate point on the lane line from its local coordinate in the image to a global coordinate in the global grid, so that these points can be understood and processed in a unified reference frame to obtain an initial grid. The initial grid refers to the grid obtained after mapping all coordinate points to the global grid. This grid contains information of all coordinate points, but is very dense and needs further coordinate point thinning.

[0102] Further, the device performs a first thinning operation on the initial grid, wherein the thinning operation refers to reducing the number of points in the grid to simplify the grid and improve processing efficiency. The first thinning operation can be a floor operation or a round operation, depending on the specific application requirements and algorithm design, to obtain a target grid. The target grid refers to the grid obtained after performing the first thining operation. This grid is sparser than the initial grid, but still retains the key information of the lane line.

[0103] In a specific implementation, the device determines a perception map corresponding to the current road image frame based on the target grid, i.e., the device uses the target grid to construct the perception map of the current road image frame. The perception map contains the position, shape, and other related road information of the lane line.

[0104] Further, the device can perform a second thinning operation based on the first thinning operation to achieve higher processing efficiency or meet specific application requirements. Specifically, the thinning operation includes a floor operation or a round operation, i.e., the first thinning operation and the second thinning operation are not the same. The first thinning operation can be a floor operation and the second thinning operation can be a round operation, or the first thinning operation can be a round operation and the second thinning operation can be a floor operation.

[0105] In a specific implementation, taking the first thinning operation as a round operation and the second thinning operation as a floor operation as an example, for each lane center line information, the following processing is performed:

[0106] 1. Obtain a first distance between each coordinate point in the target coordinate point set and the vehicle, and save each coordinate point in the coordinate sequence in ascending order of the first distance to obtain a coordinate sequence of the target lane line;

[0107] 2. Map the coordinate sequence to the corresponding position in the grid to obtain an initial grid;

[0108] 3. Round the coordinate values of the coordinate sequence in the initial grid to obtain a coordinate sequence of the coordinate values after rounding;

[0109] 4. Calculate the second distance between adjacent coordinate points in the initial grid (coordinate sequence after rounding the coordinate values) in sequence;

[0110] 5. Determine whether the second distance is less than a preset distance threshold;

[0111] 6. If the second distance is less than a preset distance threshold, deleting the coordinate point with a greater first distance among the adjacent coordinate points to obtain a coordinate sequence after thinning, and returning to the step of sequentially calculating the second distance between the adjacent coordinate points of the coordinate sequence in the initial grid until the second distance between the adjacent coordinate points of the coordinate sequence in the initial grid is all less than the preset distance threshold, to obtain a coordinate sequence after one-time thinning.

[0112] 7. Rounding each coordinate value of the coordinate sequence after one-time thinning to obtain a coordinate sequence after twice rounding of coordinate values.

[0113] 8. Sequentially calculating the second distance between the adjacent coordinate points of the coordinate sequence after twice rounding of coordinate values in the initial grid.

[0114] 9. If the second distance is less than a preset distance threshold, deleting the coordinate point with a greater first distance among the adjacent coordinate points to obtain a coordinate sequence after twice thinning, and returning to the step of sequentially calculating the second distance between the adjacent coordinate points of the coordinate sequence in the initial grid until the second distance between the adjacent coordinate points of the initial grid is all less than the preset distance threshold, to obtain a coordinate sequence after twice thinning.

[0115] Step A400, determining a first coordinate point in the coordinate point set within the cumulative boundary range and a second coordinate point in the coordinate point set outside the cumulative boundary range based on the perception map.

[0116] In a specific implementation, the apparatus determines the first coordinate point in the coordinate point set within the cumulative boundary range and the second coordinate point in the coordinate point set outside the cumulative boundary range on the perception map.

[0117] It should be noted that the above examples are only used for understanding the present application and do not constitute a limitation on the lane line splicing method of the present application. More forms of simple transformation based on this technical concept are within the protection scope of the present application.

[0118] The present application also provides a lane line splicing device, please refer to Figure 3 The lane line splicing device comprises:

[0119] The acquisition module 10 is configured to acquire a target coordinate point set of a target lane line in a current road image frame and a cumulative boundary range of the target lane line.

[0120] The determination module 20 is configured to determine a first coordinate point in the target coordinate point set within the cumulative boundary range and a second coordinate point in the target coordinate point set outside the cumulative boundary range.

[0121] The de-duplication module 30 is configured to perform de-duplication processing on the lane lines corresponding to the first coordinate points, and perform splicing of the target lane line based on the second coordinate points to obtain spliced lane line data.

[0122] Optionally, the lane line splicing apparatus comprises:

[0123] The maximum coordinate value determination module is configured to determine maximum coordinate values of the coordinate systems respectively corresponding to the second coordinate points.

[0124] The updating module is configured to update the accumulated boundary range of the target lane line based on the maximum coordinate values of the coordinate systems to obtain an updated accumulated boundary range, wherein the updated accumulated boundary range is used to implement splicing of the target lane line in a next road image frame.

[0125] Optionally, the step of obtaining the target coordinate point set of the target lane line in the current road image frame comprises:

[0126] The image frame obtaining module is configured to obtain a current road image frame.

[0127] The extraction module is configured to extract vehicle position information and an initial coordinate point set of the target lane line from the current road image frame.

[0128] The conversion module is configured to convert the initial coordinate point set of the target lane line from a global coordinate system to a local coordinate system with the vehicle as an origin based on the vehicle position information to obtain the target coordinate point set of the target lane line.

[0129] Optionally, the conversion module comprises:

[0130] The information extraction module is configured to extract a vehicle coordinate value and a vehicle orientation angle in the vehicle position information.

[0131] The translation value calculation module is configured to calculate coordinate translation values of the initial coordinate points by subtracting the vehicle coordinate value from the coordinate values of the initial coordinate points in the initial coordinate point set of the target lane line.

[0132] The rotation angle calculation module is configured to calculate a rotation angle of the target lane line based on the vehicle orientation angle.

[0133] The coordinate system conversion module is configured to convert the initial coordinate point set of the target lane line from the global coordinate system to the local coordinate system with the vehicle as the origin based on the coordinate translation values and the rotation angle to obtain the target coordinate point set of the target lane line.

[0134] Optionally, the lane line splicing apparatus comprises:

[0135] a constructing module configured to construct a global grid;

[0136] a mapping module configured to map each coordinate point in the target coordinate point set to the global grid to determine a perception map corresponding to a current road image frame;

[0137] Optionally, the determining module 20 further comprises:

[0138] a coordinate point determining module configured to determine, based on the perception map, a first coordinate point in the coordinate point set that is within the cumulative boundary range and a second coordinate point in the coordinate point set that is outside the cumulative boundary range.

[0139] Optionally, the mapping module comprises:

[0140] a first thinning module configured to map each coordinate point in the target coordinate point set to a corresponding position in a grid to obtain an initial grid, perform a first thinning operation on the initial grid to obtain a target grid, and determine, based on the target grid, a perception map corresponding to a current road image frame;

[0141] a second thinning module configured to map each coordinate point in the target coordinate point set to a corresponding position in a grid to obtain an initial grid, perform a first thinning operation on the initial grid, and perform a second thinning operation on the grid after the first thinning operation to obtain a target grid, wherein the thinning operation comprises a floor operation or a rounding operation.

[0142] The lane line splicing device provided by the present application adopts the lane line splicing method in the above embodiments and can solve the technical problem of lane line splicing. Compared with the prior art, the lane line splicing device provided by the present application has the same beneficial effects as the lane line splicing method provided by the above embodiments, and other technical features in the lane line splicing device are the same as the features disclosed in the above embodiments, which will not be repeated here.

[0143] The present application provides a lane line splicing device, which comprises at least one processor and a memory in communication connection with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the lane line splicing method in the above embodiment one.

[0144] The following refers to Figure 4The diagram illustrates a structural schematic suitable for implementing lane line splicing devices according to embodiments of this application. The lane line splicing devices in these embodiments may include, but are not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 4 The lane line splicing device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0145] like Figure 4 As shown, the lane line splicing device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the lane line splicing device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touch screen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. The communication device 1009 allows the lane line stitching device to communicate wirelessly or wiredly with other devices to exchange data. Although lane line stitching devices with various systems are shown in the figures, it should be understood that implementation or possession of all the systems shown is not required. More or fewer systems may be implemented alternatively.

[0146] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product comprising a computer program carried on a computer readable medium, the computer program containing program codes for executing the method shown in the flowchart. In such embodiments, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiments disclosed in the present application are executed.

[0147] The lane line splicing device provided by the present application adopts the lane line splicing method in the above-mentioned embodiments, and can solve the technical problem of lane line splicing. Compared with the prior art, the lane line splicing device provided by the present application has the same beneficial effects as the lane line splicing method provided by the above-mentioned embodiments, and other technical features in the lane line splicing device are the same as the features disclosed in the previous embodiment method, which will not be repeated here.

[0148] It should be understood that various parts of the present application can be realized by hardware, software, firmware or a combination thereof. In the description of the above-mentioned embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0149] The above is merely specific embodiments of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

[0150] The present application provides a computer readable storage medium having stored thereon computer readable program instructions (i.e. computer program) for executing the lane line splicing method in the above-mentioned embodiments.

[0151] The computer readable storage medium provided in the present application may, for example, be a U disk, but is not limited to an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, system, or device, or any combination of the above. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more conductive wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present embodiment, the computer readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer readable storage medium can be transmitted in any suitable medium, including but not limited to electrical wires, optical cables, RF (Radio Frequency), and the like, or any suitable combination of the above.

[0152] The above computer readable storage medium can be included in the lane line splicing device, or can exist separately without being assembled into the lane line splicing device.

[0153] The above computer readable storage medium carries one or more programs, which, when executed by the lane line splicing device, cause the lane line splicing device to splice lane lines.

[0154] Computer program code for carrying out operations of the present application can be written in one or more programming languages or combinations of languages including object oriented programming languages such as Java, Smalltalk, C++ or conventional procedural programming languages such as "C" or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0155] The flow and block diagrams in the drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flow and block diagrams can represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks may be executed in the reverse order, depending on the functionality involved. It will also be noted that each block of the block diagrams and / or flowchart illustrations, and combinations thereof, can be implemented by special purpose hardware-based systems that perform the specified functions or operations, or combinations of special purpose hardware and computer instructions.

[0156] The modules involved in the embodiments of the present application can be implemented in software or in hardware. In some cases, the names of the modules do not limit the modules themselves.

[0157] The readable storage medium provided by the present application is a computer readable storage medium, which stores computer readable program instructions (i.e., a computer program) for executing the lane line splicing method described above, and can solve the technical problem of lane line splicing. Compared with the prior art, the computer readable storage medium provided by the present application has the same beneficial effects as the lane line splicing method provided by the above-mentioned embodiments, and will not be repeated here.

[0158] The present application also provides a computer program product comprising a computer program, which, when executed by a processor, implements the steps of the lane line splicing method as described above.

[0159] The computer program product provided by the present application can solve the technical problem of lane line splicing. Compared with the prior art, the computer program product provided by the present application has the same beneficial effects as the lane line splicing method provided by the above-mentioned embodiments, and will not be repeated here.

[0160] The above only describes some embodiments of the present application, and does not limit the patent scope of the present application. Any equivalent structural transformation made by using the content of the present application specification and drawings, or direct / indirect application in other related technical fields is included in the patent protection scope of the present application.

Claims

1. A lane line stitching method, characterized by, The lane line splicing method comprises: acquiring a target coordinate point set of a target lane line in a current road image frame and a cumulative boundary range of the target lane line; determining a first coordinate point in the target coordinate point set within the cumulative boundary range and a second coordinate point outside the cumulative boundary range; performing deduplication processing on a lane line corresponding to the first coordinate point and splicing the target lane line based on the second coordinate point to obtain spliced lane line data; After the step of performing deduplication processing on a lane line corresponding to the first coordinate point and splicing the target lane line based on the second coordinate point to obtain spliced lane line data, the method comprises: determining maximum coordinate values of each coordinate system corresponding to each second coordinate point; updating the cumulative boundary range of the target lane line based on the maximum coordinate values of each coordinate system to obtain an updated cumulative boundary range, wherein the updated cumulative boundary range is used to realize splicing of the target lane line in a next road image frame.

2. The lane line stitching method of claim 1, wherein, The step of acquiring a target coordinate point set of a target lane line in a current road image frame comprises: acquiring a current road image frame; extracting vehicle position information and an initial coordinate point set of a target lane line from the current road image frame; based on the vehicle position information, converting the initial coordinate point set of the target lane line from a global coordinate system to a local coordinate system with the vehicle as the origin to obtain a target coordinate point set of the target lane line.

3. The lane line stitching method of claim 2, wherein, The step of converting the initial coordinate point set of the target lane line from a global coordinate system to a local coordinate system with the vehicle as the origin based on the vehicle position information to obtain a target coordinate point set of the target lane line comprises: extracting vehicle coordinate values and a vehicle orientation angle from the vehicle position information; performing difference calculation on the coordinate values of each initial coordinate point in the initial coordinate point set of the target lane line and the vehicle coordinate values to obtain coordinate translation values of each initial coordinate point; calculating a rotation angle of the target lane line based on the vehicle orientation angle; based on the coordinate translation values and the rotation angle, converting the initial coordinate point set of the target lane line from a global coordinate system to a local coordinate system with the vehicle as the origin to obtain a target coordinate point set of the target lane line.

4. The lane line stitching method of claim 1, wherein, After the step of acquiring a target coordinate point set of a target lane line in a current road image frame and a cumulative boundary range of the target lane line, the method comprises: constructing a global grid; mapping each coordinate point in the target coordinate point set to the global grid to determine a corresponding perception map of the current road image frame; The step of determining a first coordinate point in the coordinate point set within the cumulative boundary range and a second coordinate point outside the cumulative boundary range comprises: based on the perception map, determining a first coordinate point in the coordinate point set within the cumulative boundary range and a second coordinate point outside the cumulative boundary range.

5. The lane line stitching method of claim 4, wherein, The step of mapping each coordinate point in the target coordinate point set into the global grid to determine the perception map corresponding to the current road image frame further includes any one of the following: mapping each coordinate point in the target coordinate point set to a corresponding position in a grid to obtain an initial grid, performing a first thinning operation on the initial grid to obtain a target grid, and determining the perception map corresponding to the current road image frame based on the target grid; mapping each coordinate point in the target coordinate point set to a corresponding position in a grid to obtain an initial grid, performing a first thinning operation on the initial grid, and performing a second thinning operation on the grid after the first thinning operation to obtain a target grid, wherein the thinning operation includes a floor operation or a rounding operation.

6. A lane line stitching device characterized by, The device comprises: an acquisition module configured to acquire a target coordinate point set of a target lane line in a current road image frame and a cumulative boundary range of the target lane line; a determination module configured to determine first coordinate points in the target coordinate point set that are within the cumulative boundary range and second coordinate points that are outside the cumulative boundary range; a de-duplication module configured to perform de-duplication processing on lane lines corresponding to the first coordinate points, perform splicing of the target lane line based on the second coordinate points to obtain spliced lane line data, and determine maximum coordinate values of respective coordinate systems corresponding to the respective second coordinate points after obtaining the spliced lane line data; update the cumulative boundary range of the target lane line based on the maximum coordinate values of the respective coordinate systems to obtain an updated cumulative boundary range, wherein the updated cumulative boundary range is used to implement splicing of the target lane line in a next road image frame.

7. A lane line stitching apparatus, characterized by, The device comprises a memory, a processor, and a computer program stored on the memory and executable on the processor, and the computer program is configured to implement the steps of the lane line splicing method according to any one of claims 1 to 5.

8. A storage medium, characterized by The storage medium is a computer-readable storage medium, and the storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the lane line splicing method according to any one of claims 1 to 5.

9. A computer program product, characterised in that, The computer program product comprises a computer program, and the computer program is executed by a processor to implement the steps of the lane line splicing method according to any one of claims 1 to 5.

Images