Lane line fusion method and device and storage medium

Abstract

The invention relates to the field of automatic driving, and discloses a lane line fusion method and device and a storage medium, and the method comprises the steps: determining the predicted position of a historical tracking point of a lane line in a current frame according to the fusion position of the historical tracking point of the lane line in a previous frame and the vehicle acceleration corresponding to the previous frame; in response to an observation position of a current tracking point corresponding to the lane line in a current frame, determining a fusion position of the current tracking point according to the observation position of the current tracking point and the predicted position of the historical tracking point; and determining a fitting curve of the lane line according to the fusion position of the current tracking point corresponding to the lane line. Through the technical scheme of the invention, the effect of reducing the calculated amount while the lane lines are fused to adapt to various complex lane line geometrical shapes is realized, and the stability of the output result of each lane line in the presence of shielding is improved.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving, and in particular to a lane fusion method, device, and storage medium. Background Art

[0002] Vision-based lane fusion technology stems primarily from the rapid development of autonomous driving and advanced driver assistance systems (ADAS). These systems require accurate perception of the road environment, and lane recognition is a key technology. Lane departure warning systems and lane keeping assist systems, in particular, rely heavily on lane fusion results to improve driving safety.

[0003] However, lane detection fusion algorithms based on traditional image processing methods lack the accuracy and real-time performance to meet the requirements of autonomous driving. Deep learning methods are becoming increasingly mainstream. However, deep learning methods require certain assumptions when tracking lane parameters, which can easily lead to errors. Furthermore, the large magnitude of the parameters being tracked complicates the fusion process.

[0004] In view of this, the present invention is proposed. Summary of the Invention

[0005] In order to solve the above technical problems, the present invention provides a lane line fusion method, device and storage medium, which achieves the effect of reducing the amount of calculation while fusing lane lines to adapt to various complex lane line geometries, and improves the stability of the output results of each lane line in the presence of occlusion.

[0006] An embodiment of the present invention provides a lane fusion method, the method comprising:

[0007] Determining a predicted position of the historical tracking point of the lane line in the current frame based on a fused position of the historical tracking point of the lane line in the previous frame and a vehicle acceleration corresponding to the previous frame;

[0008] In response to the presence of an observed position of the current tracking point corresponding to the lane line in the current frame, determining a fused position of the current tracking point according to the observed position of the current tracking point and the predicted positions of the historical tracking points;

[0009] A fitting curve for the lane line is determined according to a fusion position of a current tracking point corresponding to the lane line.

[0010] An embodiment of the present invention provides an electronic device, comprising:

[0011] processor and memory;

[0012] The processor is configured to execute the steps of the lane fusion method described in any embodiment by calling the program or instructions stored in the memory.

[0013] An embodiment of the present invention provides a computer-readable storage medium, which stores a program or instruction. The program or instruction enables a computer to execute the steps of the lane fusion method described in any embodiment.

[0014] The embodiments of the present invention have the following technical effects:

[0015] By determining the predicted position of the historical tracking point of the lane line in the current frame based on the fused position of the historical tracking point of the lane line in the previous frame and the vehicle acceleration corresponding to the previous frame, the historical tracking point is predicted in the current frame. Furthermore, in response to the observed position of the current tracking point corresponding to the lane line in the current frame, the fused position of the current tracking point is determined based on the observed position of the current tracking point and the predicted position of the historical tracking point, so as to fuse the predicted position and the observed position to improve the effectiveness of the position of the current tracking point. Finally, the fitting curve of the lane line is determined based on the fused position of the current tracking point corresponding to the lane line, thereby achieving the effect of reducing the amount of calculation while merging the lane lines to adapt to various complex lane line geometries, and improving the stability of the output results of each lane line in the presence of occlusion. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0017] Figure 1 This is a flow chart of a lane fusion method provided by an embodiment of the present invention;

[0018] Figure 2 is a flowchart of another lane fusion method provided by an embodiment of the present invention;

[0019] Figure 3 It is a structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0020] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention are described clearly and completely below. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without inventive effort are also within the scope of protection of the present invention.

[0021] The lane line fusion method provided in the embodiment of the present invention is mainly applicable to the case of stable lane line recognition and output. The lane line fusion method provided in the embodiment of the present invention can be executed by an electronic device.

[0022] Figure 1 This is a flow chart of a lane line fusion method provided by an embodiment of the present invention. Figure 1 , the lane line fusion method specifically includes:

[0023] S110 , determining a predicted position of the historical tracking point of the lane line in the current frame based on the fused position of the historical tracking point of the lane line in the previous frame and the vehicle acceleration corresponding to the previous frame.

[0024] The historical tracking point is the position of the lane line tracked in the previous frame. The fused position of the historical tracking point in the previous frame is the position obtained by fusing the observation and prediction of the historical tracking point in the previous frame. The predicted position is the position obtained by predicting the motion of the historical tracking point based on the vehicle's motion.

[0025] Specifically, each lane line can be processed in the same way. Taking one lane line as an example, according to the vehicle acceleration corresponding to the previous frame, the fused position of the historical tracking point of the lane line in the previous frame is predicted in the current frame, and the predicted position of the historical tracking point of the lane line in the current frame can be obtained.

[0026] For example, the predicted position of the lane line at the historical tracking point in the current frame can be determined by the following formula:

[0027]

[0028] in, is the predicted position of a historical tracking point in the current frame, is the fusion position of a historical tracking point in the previous frame, dt is the time interval between the current frame and the previous frame, and a is the vehicle acceleration corresponding to the previous frame.

[0029] As you can understand, after receiving lane lines, they must first be converted from the image perspective to the Bird's Eye View (BEV) perspective. Then, based on prior information, outlier segments are removed. Lane lines from different sources within the current frame must first be correlated within the frame to ensure they originate from the same physical world. This means that the lane markings are set to be consistent.

[0030] It's important to note that when a new lane line is detected in the physical world and enters the lane line fusion process, a new lane line is created and tracking points are sampled for that lane line (sampling direction). The sampling interval is set according to the actual situation. At the same time, the parameters of the filtering algorithm are initialized for each tracking point, and the lifecycle is initialized. For example, the filtering algorithm is a Kalman filter, where parameters include a preset state transfer matrix, a preset observation matrix, and a preset process noise covariance matrix. The lifecycle includes the number of tracking frames and the number of lost frames, which are initialized to 0.

[0031] S120 : In response to the presence of an observed position of the current tracking point corresponding to the lane line in the current frame, determining a fused position of the current tracking point according to the observed position of the current tracking point and the predicted positions of historical tracking points.

[0032] The fusion position of the current tracking point is the position obtained after the observation and prediction fusion of the historical tracking points in the current frame.

[0033] The current tracking point is a tracking point that needs to be fused in the current frame. It is understandable that the current tracking point and some of the historical tracking points have the same horizontal coordinate value.

[0034] Specifically, the system determines whether the lane marking corresponds to the current tracking point's observed position in the current frame. Specifically, it determines whether the deviation between the current tracking point's observed position and the lane marking's fused position in the previous frame is within a certain range. If so, the lane marking is within the observed range, and fusion processing can proceed. Therefore, the current tracking point's observed position and the corresponding historical tracking point's predicted position are fused to obtain the fused position of the current tracking point.

[0035] Based on the above example, the following method can be used to determine the fusion position of the current tracking point based on the observed position of the current tracking point and the predicted position of the historical tracking points:

[0036] For each current tracking point, determine whether there is a historical tracking point corresponding to the current tracking point;

[0037] In response to the existence of a historical tracking point corresponding to the current tracking point, a fused position of the current tracking point is determined according to the predicted positions of the historical tracking points corresponding to the current tracking point and the observed position of the current tracking point.

[0038] Specifically, for each current tracking point, the deviation between the observed position of the current tracking point and the fused position of the historical tracking points is calculated. If the deviation is within a preset range, it is determined that a historical tracking point corresponding to the current tracking point exists, and a correspondence between the two is established. Otherwise, no historical tracking point corresponding to the current tracking point exists. If a historical tracking point corresponding to the current tracking point exists, the predicted position of the historical tracking point corresponding to the current tracking point and the observed position of the current tracking point are fused using a filtering algorithm. The fused result is the fused position of the current tracking point.

[0039] Optionally, in response to the absence of a historical tracking point corresponding to the current tracking point, the observed position of the current tracking point is determined as the fused position of the current tracking point.

[0040] Based on the above example, the following method can be used to determine the fusion position of the current tracking point based on the predicted position of the historical tracking point corresponding to the current tracking point and the observed position of the current tracking point:

[0041] The fusion position of the current tracking point is determined by the following formula:

[0042]

[0043] in, is the fusion position of the current tracking point, is the observation position of the current tracking point, is the predicted position of the historical tracking point corresponding to the current tracking point, K is the Kalman gain of the current frame, and H is the preset observation matrix.

[0044] It can be understood that the corresponding vertical coordinate is obtained by sampling the horizontal coordinate of the current tracking point on the lane line within the length range of the observation position. Here, Kalman filtering is used as an example. After obtaining the observation position of the current tracking point, the corresponding historical tracking point can be determined for each current tracking point, and the predicted position of the historical tracking point can be obtained, which is convenient for Kalman filtering processing.

[0045] Based on the above example, if Kalman filtering is used as the filtering algorithm for fusion processing, the Kalman gain of the current frame needs to be obtained, which can be:

[0046] Determine the error covariance matrix of the current frame based on the error covariance matrix corresponding to the previous frame, the preset state transfer matrix, and the preset process noise covariance matrix;

[0047] The Kalman gain of the current frame is determined according to the error covariance matrix of the current frame, the preset observation matrix and the preset observation noise covariance matrix.

[0048] The preset state transfer matrix, preset process noise covariance matrix, preset observation matrix, and preset observation noise covariance matrix are pre-calibrated parameters of the Kalman filter used for lane fusion. The error covariance matrix is ​​initialized to zero in the first frame and is iteratively accumulated in subsequent frames.

[0049] Specifically, the preset state transfer matrix, the error covariance matrix corresponding to the previous frame, and the transposed matrix of the preset state transfer matrix are sequentially multiplied, and then the preset process noise covariance matrix is ​​added to obtain the error covariance matrix of the current frame. Furthermore, the preset observation matrix, the error covariance matrix of the current frame, and the transposed matrix of the preset observation matrix are sequentially multiplied, and then the preset observation noise covariance matrix is ​​added to obtain the inverse matrix of the obtained results, which serves as the process multiplier matrix. Then, the error covariance matrix of the current frame, the transposed matrix of the preset observation matrix, and the process multiplier matrix are sequentially multiplied to obtain the Kalman gain of the current frame.

[0050] Exemplarily, the error covariance matrix of the current frame can be determined by the following formula:

[0051] P t =F*P t-1 *F t +Q t

[0052] Among them, P t is the error covariance matrix of the current frame, F is the preset state transfer matrix, F t is the transposed matrix of the preset state transfer matrix, P t-1 is the error covariance matrix of the previous frame, Q t is the preset process noise covariance matrix.

[0053] Furthermore, the Kalman gain of the current frame can be determined by the following formula:

[0054] K=P t *H t *(H*P t *H t +R t ) -1

[0055] Among them, K is the Kalman gain of the current frame, P t is the error covariance matrix of the current frame, H is the preset observation matrix, H t is the transposed matrix of the preset observation matrix, R t is the preset observation noise covariance matrix.

[0056] Based on the above example, to ensure the stability and effectiveness of lane lines, a lifecycle can be set for lane lines. The lifecycle can include the number of lost frames, so that invalid lane lines can be deleted based on the number of lost frames. Specifically, it can be:

[0057] In response to the observation position of the current tracking point corresponding to the lane line not existing in the current frame, the number of lost frames of the lane line is determined to be increased by one;

[0058] In response to the number of lost frames of the lane line being greater than or equal to a first threshold, deleting the lane line;

[0059] In response to the number of lost frames of the lane line being less than a first threshold, the predicted position of the historical tracking point of the lane line in the current frame is used as the fused position of the current tracking point.

[0060] The number of lost frames for a lane line is initialized to 0 when the lane line is created. This number indicates the number of frames in which the current tracking point's observation position does not correspond to an existing lane line. This number represents the total number of frames in which no observation position corresponds to the current tracking point. The first threshold is a pre-set value used to determine whether the number of lost frames meets the requirement for deleting the corresponding lane line.

[0061] Specifically, if the observation position of the current tracking point corresponding to the lane line does not exist in the current frame, it means that the lane line is not observed in the current frame, so the number of lost frames of the lane line is increased by one. For lane lines without observation positions, it is necessary to compare the number of lost frames of the lane line with the first threshold. If the number of lost frames of the lane line is greater than or equal to the first threshold, it means that the lane line is extremely unstable or has ended, and the lane line needs to be deleted and the processing of the lane line is stopped. If the number of lost frames of the lane line is less than the first threshold, it means that although there is no observation position for the lane line in the current frame, it may be an accidental phenomenon. The predicted position of the historical tracking point of the lane line in the current frame can be directly used as the fusion position of the current tracking point to facilitate the processing of the next frame.

[0062] S130: Determine a fitting curve for the lane line according to the fusion position of the current tracking point corresponding to the lane line.

[0063] The fitting curve is a curve formed by fitting the fusion positions of the current tracking points, which is used to represent the position and shape of the lane line.

[0064] Specifically, according to the fusion position of each current tracking point corresponding to the lane line, a preset equation is used to perform curve fitting, and the obtained curve is the fitting curve of the lane line.

[0065] Based on the above example, to ensure the stability and effectiveness of lane lines, a lifecycle can be set for lane lines. The lifecycle can include the number of tracking frames, so that the number of tracking frames can be used to determine the timing of displaying the fitted curve of the lane line. Therefore, after determining the fusion position of the current tracking point based on the observed position of the current tracking point and the predicted position of the historical tracking points, the number of tracking frames needs to be accumulated:

[0066] Determine the number of tracking frames for the lane line plus one.

[0067] The number of lane line tracking frames is initialized to 0 when the lane line is established. The number of tracking frames is used to represent the total number of frames in the current frame where the observation position of the current tracking point corresponds to the existing lane line and is used for fusion processing.

[0068] Accordingly, after determining the fitting curve of the lane line based on the fusion position of the current tracking point corresponding to the lane line, it is also possible to combine the number of tracking frames to determine whether the lane line is reliable enough for visualization. Specifically, it can be:

[0069] In response to the number of tracking frames of the lane line being greater than or equal to a second threshold, the fitting curve of the lane line is displayed on the target display screen.

[0070] The second threshold is a pre-set value used to determine whether the number of tracking frames meets the requirement for visually displaying the fitting curve of the corresponding lane line. The target display screen is a display screen used to display the fitting curve of each lane line, which can be a central control screen, etc.

[0071] Specifically, for lane lines with fitted curves, the number of tracking frames for the lane lines needs to be compared with a second threshold. If the number of tracking frames for the lane lines is greater than or equal to the second threshold, it indicates that the lane lines are relatively stable and can be visualized to provide a reference for the user. Therefore, the lane line fitting curve is displayed on the target display screen. It is understood that if the number of tracking frames for the lane lines is less than the second threshold, it means that the lane lines have just been established and are not yet completely stable. Continued attention is required until the number of tracking frames for the lane lines is greater than or equal to the second threshold, at which time the fitting curve for the lane lines begins to be displayed.

[0072] It can be understood that the sum of the number of tracking frames and the number of lost frames in the life cycle is the number of frames that the lane line lasts after it is constructed.

[0073] The present invention has the following technical effects: by determining the predicted position of the historical tracking point of the lane line in the current frame according to the fusion position of the historical tracking point of the lane line in the previous frame and the vehicle acceleration corresponding to the previous frame, the historical tracking point is predicted in the current frame; further, in response to the observation position of the current tracking point corresponding to the lane line in the current frame, the fusion position of the current tracking point is determined according to the observation position of the current tracking point and the predicted position of the historical tracking point, so as to fuse the predicted position and the observation position, thereby improving the effectiveness of the position of the current tracking point; finally, according to the fusion position of the current tracking point corresponding to the lane line, the fitting curve of the lane line is determined, thereby achieving the effect of reducing the amount of calculation while merging the lane lines to adapt to various complex lane line geometries, and improving the stability of the output results of each lane line in the presence of occlusion.

[0074] Figure 2 This is a flow chart of another lane fusion method provided by an embodiment of the present invention. Figure 2 , the lane line fusion method specifically includes:

[0075] S210 , determining a predicted position of the historical tracking point of the lane line in the current frame based on the fused position of the historical tracking point of the lane line in the previous frame and the vehicle acceleration corresponding to the previous frame.

[0076] S220 : In response to the presence of an observed position of the current tracking point corresponding to the lane line in the current frame, determining a fused position of the current tracking point according to the observed position of the current tracking point and the predicted positions of the historical tracking points.

[0077] S230: Determine the measured length of the lane line according to the fused position of the current tracking point corresponding to the lane line.

[0078] The measured length is the sum of the calculated lengths of the fusion positions of each current tracking point connected in sequence according to the sampling order.

[0079] Specifically, the lengths of every two adjacent fused positions in the fused positions of each current tracking point corresponding to the lane line are calculated, and the sum of these lengths is determined as the measured length of the lane line.

[0080] S240: Determine a target curve fitting equation based on the measured length; and determine a fitting curve for the lane line based on the fusion position of the current tracking point corresponding to the lane line and the target curve fitting equation.

[0081] The target curve fitting equation is an equation used to perform curve fitting on the fusion position of the current tracking point.

[0082] Specifically, the measured length provides a preliminary estimate of the curve's complexity. For curves with high complexity, a complex curve fitting equation is selected as the target curve fitting equation, while for curves with low complexity, a simpler curve fitting equation is selected as the target curve fitting equation. The target curve fitting equation is then used to fit the fused positions of the current tracking points corresponding to the lane line, yielding the lane line equation, or the fitted curve for that lane line.

[0083] For example, if the measured length is greater than or equal to the length threshold, the cubic equation is used as the target curve fitting equation; if the measured length is less than the length threshold, the quadratic equation is used as the target curve fitting equation. The cubic equation is: y = C0 + C1*x + C2*x 2 +C3*x 3 ; The quadratic equation is: y=C0+C1*x+C2*x 2 Among them, x and y are the independent variable and dependent variable of the equation respectively, C0 is the constant term coefficient obtained by fitting, C1 is the linear term coefficient obtained by fitting, C2 is the quadratic term coefficient obtained by fitting, and C3 is the cubic term coefficient obtained by fitting.

[0084] Of course, other candidate curve fitting equations may also be used to match the distribution of different fusion positions of current tracking points.

[0085] S250 : Determine a coordinate transformation matrix between the previous frame and the current frame according to the vehicle motion information.

[0086] The vehicle motion information includes the vehicle's speed, acceleration, etc. The coordinate transformation matrix is ​​the transformation matrix used to convert the position points in the previous frame to the current frame. It can be understood as the coordinate transformation matrix between the vehicle coordinate system of the current frame and the vehicle coordinate system of the previous frame.

[0087] Specifically, calculations based on vehicle motion information can determine the pose change between the current frame and the previous frame, including the position change and heading angle change. Based on these position and heading angle changes, the translation and rotation of the coordinate system can be determined, and the coordinate transformation matrix between the previous and current frames can be calculated.

[0088] S260: Obtain a track map in a previous vehicle coordinate system corresponding to a previous frame, and convert the track map in the previous vehicle coordinate system to the current vehicle coordinate system according to a coordinate transformation matrix to obtain an initial map.

[0089] The track map is a lightweight map constructed by combining the lane lines in previous frames. The initial map is the result of converting the track map in the previous vehicle coordinate system to the current vehicle coordinate system.

[0090] Specifically, the track map obtained by mapping the target curves of each lane line in the previous frame can be obtained. Since the track map corresponds to the corresponding previous vehicle coordinate system, it needs to be converted to the current vehicle coordinate system. Specifically, the coordinate transformation matrix can be used to transform the track map to the current vehicle coordinate system to obtain the initial map.

[0091] S270: Perform joint optimization based on the initial map and the fusion curves of each lane line in the current frame to determine a track map in the current vehicle coordinate system corresponding to the current frame.

[0092] Specifically, the fusion curves of each lane line in the current frame are spliced ​​with the initial map, and the splicing result is optimized, such as smoothing and prediction processing, so that it can refer to the results of the initial map. The map obtained by joint optimization is used as the track map in the current vehicle coordinate system corresponding to the current frame.

[0093] S280: Truncate the track map in the current vehicle coordinate system according to a preset truncation range, update the track map in the current vehicle coordinate system, and determine the target curve of each lane line.

[0094] The preset cutoff range is a pre-set range threshold to prevent the track map from expanding unnecessarily. The target curve is the lane curve within the vehicle's field of view in the track map in the current vehicle coordinate system.

[0095] Specifically, the track map in the current vehicle coordinate system is truncated according to a preset cutoff range, and the truncated map becomes the new track map in the current vehicle coordinate system. Furthermore, the field of view is determined based on the vehicle's current position, and the target curves for each lane are extracted from the portion of the new track map in the current vehicle coordinate system within the field of view. Furthermore, the track map in the current vehicle coordinate system serves as a known condition for solving the target curves in the next frame.

[0096] It is understandable that, compared to other existing fusion algorithms, the lane fusion method proposed in this embodiment abandons the traditional lane parameter tracking during the tracking phase and instead uses a specific point set for tracking. This improves the following technical issues faced by parameter tracking: The lane curvature must be within a certain range, satisfying the assumption that the heading angle is equivalent to the curve curvature; lane parameter tracking treats the entire lane as a whole. Inaccurate detection values ​​outside a certain range will affect the fitting accuracy of the entire lane; and because the magnitudes of each parameter in parameter tracking are non-uniform and vary significantly, inaccurate hyperparameter settings during the tracking filtering phase can easily lead to non-convergence in the tracking process. The lane fusion method proposed in this embodiment also incorporates mapping principles, incorporating a fusion algorithm with mapping principles. This has the following advantages: The forward and rearward view information of the lane is unified in the temporal dimension. After a period of time, the forward view point set moves into the rearward view, helping to improve the stability of the rearward view lane regression; and the mapping compensates for lane line omissions caused by pure detection, further improving the stability of the final lane output.

[0097] The present invention has the following technical effects: determining the measured length of the lane line based on the fused position of the current tracking point corresponding to the lane line, and determining a target curve fitting equation based on the measured length; determining a fitting curve for the lane line based on the fused position of the current tracking point corresponding to the lane line and the target curve fitting equation, thereby estimating the complexity of the lane line according to different measured lengths and matching an appropriate target curve fitting equation for the lane line, thereby improving the fitting effect; further, determining a coordinate transformation matrix between the previous frame and the current frame based on vehicle motion information, obtaining a track map in the previous vehicle coordinate system corresponding to the previous frame; converting the track map in the previous vehicle coordinate system to the current vehicle coordinate system based on the coordinate transformation matrix to obtain an initial map; performing joint optimization based on the initial map and the fused curves of each lane line in the current frame to determine a track map in the current vehicle coordinate system corresponding to the current frame; truncating the track map in the current vehicle coordinate system according to a preset truncation range, updating the track map in the current vehicle coordinate system, and determining a target curve for each lane line. This realizes the integration of mapping ideas, unifies forward and backward information in the time dimension, and improves the stability of lane line output.

[0098] Figure 3 Schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. Figure 3 As shown, the electronic device 300 includes one or more processors 301 and a memory 302 .

[0099] The processor 301 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 300 to perform desired functions.

[0100] The memory 302 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), a hard disk, a flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 301 may execute the program instructions to implement the lane fusion method of any embodiment of the present invention described above and / or other desired functions. Various contents such as initial external parameters, thresholds, etc. may also be stored in the computer-readable storage medium.

[0101] In one example, the electronic device 300 may further include an input device 303 and an output device 304, which are interconnected via a bus system and / or other connection mechanisms (not shown). The input device 303 may include, for example, a keyboard, a mouse, etc. The output device 304 may output various information to the outside, including warning information, braking force, etc. The output device 304 may include, for example, a display, a speaker, a printer, a communication network, and remote output devices connected thereto.

[0102] Of course, to simplify, Figure 3 Only some of the components related to the present invention in the electronic device 300 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, the electronic device 300 may further include any other appropriate components according to specific application scenarios.

[0103] In addition to the above methods and devices, an embodiment of the present invention may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to perform the steps of the lane fusion method provided by any embodiment of the present invention.

[0104] The computer program product may be written in any combination of one or more programming languages ​​to implement the operations of embodiments of the present invention, including object-oriented programming languages ​​such as Java, C++, and conventional procedural programming languages ​​such as C or similar programming languages. The program code may be executed entirely on the user's computing device, partially on the user's computing device, as a stand-alone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0105] In addition, an embodiment of the present invention may also be a computer-readable storage medium having computer program instructions stored thereon. When the computer program instructions are executed by a processor, the processor executes the steps of the lane fusion method provided by any embodiment of the present invention.

[0106] The computer-readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can, for example, include but is not limited to a system, device or component of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0107] It should be noted that the terms used in the present invention are only for describing specific embodiments and are not intended to limit the scope of this application. As shown in the present specification, unless the context clearly indicates an exception, the words "one", "an", "a kind of" and / or "the" do not specifically refer to the singular and may also include the plural. The terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method or device comprising a series of elements includes not only those elements, but also other elements that are not explicitly listed, or also include elements inherent to such process, method or device. In the absence of further restrictions, the elements defined by the sentence "comprise a..." do not exclude the presence of other identical elements in the process, method or device comprising the elements.

[0108] It should also be noted that the terms "center", "up", "down", "left", "right", "vertical", "horizontal", "inside", "outside", etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, they cannot be understood as limitations on the present invention. Unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", etc. should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or an indirect connection through an intermediate medium, or it can be a communication between the internal parts of two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0109] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the technical solutions of the embodiments of the present invention.

Claims

1. A lane fusion method, characterized in that: include: Determining a predicted position of the historical tracking point of the lane line in the current frame based on a fused position of the historical tracking point of the lane line in the previous frame and a vehicle acceleration corresponding to the previous frame; In response to the presence of an observed position of the current tracking point corresponding to the lane line in the current frame, determining a fused position of the current tracking point according to the observed position of the current tracking point and the predicted positions of the historical tracking points; A fitting curve for the lane line is determined according to a fusion position of a current tracking point corresponding to the lane line.

2. The method according to claim 1, characterized in that The determining the fusion position of the current tracking point according to the observed position of the current tracking point and the predicted position of the historical tracking point includes: For each current tracking point, determining whether there is a historical tracking point corresponding to the current tracking point; In response to the existence of a historical tracking point corresponding to the current tracking point, a fused position of the current tracking point is determined according to the predicted positions of the historical tracking points corresponding to the current tracking point and the observed position of the current tracking point.

3. The method according to claim 2, characterized in that The determining the fusion position of the current tracking point according to the predicted position of the historical tracking point corresponding to the current tracking point and the observed position of the current tracking point includes: The fusion position of the current tracking point is determined by the following formula: in, is the fusion position of the current tracking point, is the observed position of the current tracking point, is the predicted position of the historical tracking point corresponding to the current tracking point, K is the Kalman gain of the current frame, and H is the preset observation matrix.

4. The method according to claim 3, characterized in that Also includes: Determining the error covariance matrix of the current frame according to the error covariance matrix corresponding to the previous frame, a preset state transfer matrix, and a preset process noise covariance matrix; The Kalman gain of the current frame is determined according to the error covariance matrix of the current frame, the preset observation matrix and the preset observation noise covariance matrix.

5. The method according to claim 1, characterized in that After determining the predicted position of the historical tracking point of the lane line in the current frame, the method further includes: In response to the absence of an observation position of the current tracking point corresponding to the lane line in the current frame, determining a number of lost frames for the lane line plus one; wherein the number of lost frames for the lane line is initialized when the lane line is established; In response to the number of lost frames of the lane line being greater than or equal to a first threshold, deleting the lane line; In response to the number of lost frames of the lane line being less than the first threshold, the predicted position of the historical tracking point of the lane line in the current frame is used as the fused position of the current tracking point.

6. The method according to claim 1, characterized in that Determining the fitting curve of the lane line according to the fusion position of the current tracking point corresponding to the lane line includes: Determining a measured length of the lane line according to a fused position of a current tracking point corresponding to the lane line; Determining a target curve fitting equation according to the measured length; A fitting curve for the lane line is determined according to a fusion position of a current tracking point corresponding to the lane line and the target curve fitting equation.

7. The method according to claim 1, characterized in that After determining the fitting curve of the lane line according to the fusion position of the current tracking point corresponding to the lane line, the method further includes: Determining a coordinate transformation matrix between the previous frame and the current frame according to the vehicle motion information; Obtaining a track map in a previous vehicle coordinate system corresponding to the previous frame, and converting the track map in the previous vehicle coordinate system to the current vehicle coordinate system according to the coordinate transformation matrix to obtain an initial map; Performing joint optimization based on the initial map and the fusion curves of each lane line in the current frame to determine a track map in the current vehicle coordinate system corresponding to the current frame; The track map in the current vehicle coordinate system is truncated according to a preset truncation range, the track map in the current vehicle coordinate system is updated, and a target curve of each lane line is determined.

8. The method according to claim 1, characterized in that After determining the fusion position of the current tracking point according to the observed position of the current tracking point and the predicted positions of the historical tracking points, the method further includes: Determine the number of tracking frames for the lane line plus one; wherein the number of tracking frames for the lane line is initialized when the lane line is established; Accordingly, after determining the fitting curve of the lane line according to the fusion position of the current tracking point corresponding to the lane line, the method further includes: In response to the number of tracking frames of the lane line being greater than or equal to a second threshold, the fitting curve of the lane line is displayed on the target display screen.

9. An electronic device, characterized in that: The electronic device comprises: processor and memory; The processor is configured to execute the steps of the lane fusion method according to any one of claims 1 to 8 by calling the program or instructions stored in the memory.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a program or instruction, which enables a computer to execute the steps of the lane fusion method according to any one of claims 1 to 8.

Images