Lane line splicing method and device, electronic equipment and storage medium

Abstract

The embodiment of the present disclosure discloses a lane line splicing method and device, electronic equipment and a storage medium, wherein the method comprises: determining a first lane line curve and a second lane line curve to be spliced in a first coordinate system; determining a splicing constraint point based on the first lane line curve and the second lane line curve; adjusting a first curve parameter corresponding to the first lane line curve based on the splicing constraint point to obtain an adjusted first target curve parameter corresponding to the first curve parameter; adjusting a second curve parameter corresponding to the second lane line curve based on the splicing constraint point to obtain an adjusted second target curve parameter corresponding to the second curve parameter; and determining a spliced lane line based on the first target curve parameter and the second target curve parameter. The embodiment of the present disclosure adjusts the curve parameters of the two lane line curves based on the splicing constraint point, so that the two adjusted lane lines can be spliced smoothly and continuously, effectively improving the effect of the fused lane line.

Description

Technical Field

[0001] This disclosure relates to driver assistance technology, and in particular to a method, apparatus, electronic device, and storage medium for splicing lane lines. Background Technology

[0002] In assisted driving scenarios, environmental images are acquired through cameras from multiple perspectives, and lane lines are fitted based on the environmental images from each perspective. Then, the lane lines corresponding to each environmental image are transformed to a unified bird's-eye view (BEV) coordinate system to obtain the lane lines in this bird's-eye view coordinate system. The lane lines in the bird's-eye view coordinate system corresponding to each perspective are then merged to form a surround-view lane line bird's-eye view for downstream use. In related technologies, merging the lane lines in the bird's-eye view coordinate system corresponding to each perspective into a surround-view lane line bird's-eye view can easily lead to problems such as discontinuity and unevenness between two lane lines belonging to the same lane line in the merging part, resulting in poor lane line effect after merging. Summary of the Invention

[0003] To address the aforementioned technical problems, such as the poor performance of the lane markings, this disclosure is proposed. Embodiments of this disclosure provide a lane marking splicing method, apparatus, electronic device, and storage medium.

[0004] According to one aspect of the present disclosure, a method for splicing lane lines is provided, comprising: determining a first lane line curve and a second lane line curve to be spliced ​​in a first coordinate system; determining splicing constraint points based on the first lane line curve and the second lane line curve; adjusting a first curve parameter corresponding to the first lane line curve based on the splicing constraint points to obtain an adjusted first target curve parameter corresponding to the first curve parameter; adjusting a second curve parameter corresponding to the second lane line curve based on the splicing constraint points to obtain an adjusted second target curve parameter corresponding to the second curve parameter; and determining the spliced ​​lane lines based on the first target curve parameter and the second target curve parameter.

[0005] According to another aspect of the present disclosure, a lane line splicing device is provided, comprising: a first determining module, configured to determine a first lane line curve and a second lane line curve to be spliced ​​in a first coordinate system; a first processing module, configured to determine splicing constraint points based on the first lane line curve and the second lane line curve; a second processing module, configured to adjust a first curve parameter corresponding to the first lane line curve based on the splicing constraint points to obtain an adjusted first target curve parameter corresponding to the first curve parameter; a third processing module, configured to adjust a second curve parameter corresponding to the second lane line curve based on the splicing constraint points to obtain an adjusted second target curve parameter corresponding to the second curve parameter; and a fourth processing module, configured to determine the spliced ​​lane line based on the first target curve parameter and the second target curve parameter.

[0006] According to another aspect of the present disclosure, a computer-readable storage medium is provided, the storage medium storing a computer program for performing the lane line splicing method described in any of the above embodiments of the present disclosure.

[0007] According to another aspect of the present disclosure, an electronic device is provided, the electronic device comprising: a processor; a memory for storing executable instructions of the processor; the processor being configured to read the executable instructions from the memory and execute the instructions to implement the lane line splicing method described in any of the above embodiments of the present disclosure.

[0008] Based on the lane line splicing method, apparatus, electronic device and storage medium provided in the above embodiments of this disclosure, splicing constraint points are determined based on the curves of the two lane lines to be spliced, and the curve parameters of the two lane line curves are adjusted based on the splicing constraint points, so that the adjusted two lane lines can be spliced ​​smoothly and continuously, effectively improving the effect of the merged lane lines.

[0009] The technical solutions of this disclosure will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0010] The above and other objects, features, and advantages of this disclosure will become more apparent from the more detailed description of the embodiments thereof in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this disclosure and form part of the specification. They are used together with the embodiments of this disclosure to explain the disclosure and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0011] Figure 1 This is an exemplary application scenario of the lane line splicing method provided in this disclosure;

[0012] Figure 2 This is a flowchart illustrating a lane line splicing method provided in an exemplary embodiment of this disclosure;

[0013] Figure 3 This is a flowchart illustrating a lane line splicing method provided in another exemplary embodiment of this disclosure;

[0014] Figure 4 This is a flowchart illustrating step 2031 provided in an exemplary embodiment of this disclosure;

[0015] Figure 5 This is a flowchart illustrating step 2041 provided in an exemplary embodiment of this disclosure;

[0016] Figure 6 This is a schematic diagram of the state quantity update and iteration process provided in an exemplary embodiment of this disclosure;

[0017] Figure 7 This is a schematic diagram of the process for determining constraint quantities provided in an exemplary embodiment of this disclosure;

[0018] Figure 8 This is a schematic diagram illustrating the longitudinal overlap of the first lane line curve and the second lane line curve provided in an exemplary embodiment of this disclosure;

[0019] Figure 9 This is a schematic diagram of the longitudinal gap between the first lane line curve and the second lane line curve provided in an exemplary embodiment of this disclosure;

[0020] Figure 10 This is a flowchart illustrating step 2021 provided in an exemplary embodiment of this disclosure;

[0021] Figure 11 This is a schematic diagram of the structure of a lane line splicing device provided in an exemplary embodiment of this disclosure;

[0022] Figure 12 This is a schematic diagram of the structure of a lane line splicing device provided in another exemplary embodiment of this disclosure;

[0023] Figure 13 This is a schematic diagram of the structure of the first processing unit 5031 provided in an exemplary embodiment of the present disclosure;

[0024] Figure 14 This is a schematic diagram of the structure of one application embodiment of the electronic device disclosed herein. Detailed Implementation

[0025] Hereinafter, exemplary embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present disclosure, and not all embodiments of the present disclosure, and it should be understood that the present disclosure is not limited to the exemplary embodiments described herein.

[0026] It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of this disclosure.

[0027] Those skilled in the art will understand that the terms "first," "second," etc., in the embodiments of this disclosure are only used to distinguish different steps, devices, or modules, and do not represent any specific technical meaning, nor do they indicate a necessary logical order between them.

[0028] It should also be understood that in the embodiments disclosed herein, "a plurality of" may refer to two or more, and "at least one" may refer to one, two or more.

[0029] It should also be understood that any component, data or structure mentioned in the embodiments of this disclosure can generally be understood as one or more unless expressly defined or given to the contrary in the context.

[0030] Furthermore, the term "and / or" in this disclosure is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this disclosure generally indicates that the preceding and following related objects have an "or" relationship.

[0031] It should also be understood that the description of the various embodiments in this disclosure emphasizes the differences between the various embodiments, and the similarities or similarities can be referred to each other. For the sake of brevity, they will not be described in detail.

[0032] At the same time, it should be understood that, for ease of description, the dimensions of the various parts shown in the accompanying drawings are not drawn according to actual scale.

[0033] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit this disclosure or its application or use.

[0034] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0035] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0036] The embodiments disclosed herein can be applied to electronic devices such as terminal devices, computer systems, and servers, and can operate together with a wide range of other general-purpose or special-purpose computing system environments or configurations. Examples of well-known terminal devices, computing systems, environments, and / or configurations suitable for use with electronic devices such as terminal devices, computer systems, and servers include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, and distributed cloud computing environments including any of the above systems, etc.

[0037] Electronic devices such as terminal devices, computer systems, and servers can be described in the general context of computer system executable instructions (such as program modules) executed by a computer system. Typically, program modules can include routines, programs, object programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. Computer systems / servers can be implemented in distributed cloud computing environments, where tasks are executed by remote processing devices linked through communication networks. In distributed cloud computing environments, program modules can reside on local or remote computing system storage media, including storage devices.

[0038] This disclosure outlines

[0039] In the process of realizing this disclosure, the inventors discovered that in assisted driving scenarios, environmental images are acquired by cameras from multiple perspectives, and lane lines are fitted based on the environmental images from each perspective. Then, the lane lines corresponding to each environmental image are transformed to a unified bird's-eye view (BEV) coordinate system to obtain the lane lines in the bird's-eye view coordinate system. Subsequently, the lane lines in the bird's-eye view coordinate system corresponding to each perspective are merged to form a surround-view lane line bird's-eye view for downstream use. In related technologies, merging the lane lines in the bird's-eye view coordinate system corresponding to each perspective into a surround-view lane line bird's-eye view can easily lead to problems such as discontinuity and unevenness in the merging part of two lane lines belonging to the same lane line, resulting in poor lane line effect after merging.

[0040] Exemplary Overview

[0041] Figure 1 This is an exemplary application scenario of the lane line splicing method provided in this disclosure.

[0042] In assisted driving scenarios, environmental images are acquired using cameras from multiple perspectives. Lane lines are fitted based on these images from each perspective, and then the lane lines corresponding to each environmental image are transformed to a unified bird's-eye view (BEV) coordinate system. This yields lane lines in the bird's-eye view coordinate system for each perspective. These lane lines are then merged to form a surround-view lane line bird's-eye view. Using the lane line stitching method disclosed herein, when merging lane lines in the bird's-eye view coordinate system for each perspective, a first lane line curve and a second lane line curve to be stitched can be determined based on certain matching rules. These first and second lane line curves are different lane segments of the same lane line corresponding to two different perspectives. For example, the first lane segment of the leftmost lane line from the perspective of the left front camera and the second lane segment of the leftmost lane line from the perspective of the left rear camera. After the first and second lane segments are acquired by the left front camera and the left rear camera respectively... After lane line detection, fitting, and coordinate transformation, the first lane line curve and the second lane line curve corresponding to the first lane line curve and the second lane line curve respectively in the bird's-eye view coordinate system are obtained. After determining the first lane line curve and the second lane line curve to be stitched in the bird's-eye view coordinate system, stitching constraint points can be determined based on the first lane line curve and the second lane line curve. Based on the stitching constraint points, the first curve parameter corresponding to the first lane line curve is adjusted to obtain the adjusted first target curve parameter. Similarly, based on the stitching constraint points, the second curve parameter corresponding to the second lane line curve is adjusted to obtain the adjusted second target curve parameter. This allows the first target curve corresponding to the adjusted first target curve parameter and the second target curve corresponding to the second target curve parameter to be stitched smoothly and continuously. Thus, based on the first target curve parameter and the second target curve parameter, a continuous and smooth lane line after stitching can be determined, greatly improving the lane line effect after fusion from various perspectives. Understandably, when merging lane lines in the bird's-eye view coordinate system from various perspectives, there may be multiple sets of lane lines to be spliced, or multiple different line segments of the same lane line in the bird's-eye view coordinate system that need to be spliced ​​in pairs. Any two lane line curves to be spliced ​​can be used as the first lane line curve and the second lane line curve, respectively, and spliced ​​using the above method. The specifics will not be elaborated here.

[0043] Exemplary methods

[0044] Figure 2 This is a flowchart illustrating a lane line splicing method provided in an exemplary embodiment of this disclosure. This embodiment can be applied to electronic devices, specifically, for example, on an in-vehicle computing platform. Figure 2 As shown, it includes the following steps:

[0045] Step 201: Determine the first lane line curve and the second lane line curve to be spliced ​​in the first coordinate system.

[0046] The first coordinate system can be either the coordinate system corresponding to the bird's-eye view (BEV) or the world coordinate system. The specific system can be set according to actual needs, as long as it unifies the lane line curves captured by cameras from different viewpoints to the same coordinate system. The first lane line curve and the second lane line curve are curve segments, each with a start point and an end point. For example, the first lane line curve and the second lane line curve can be represented as follows:

[0047] f1(x,y)=0,x∈[x s1 ,x e1 ),y∈[y s1 ,y e1 )

[0048] f2(x,y)=0,x∈[x s2 ,x e2 ),y∈[y s2 ,y e2 )

[0049] Where f1(x,y)=0 represents the curve of the first lane, x s1 and y s1 Let x represent the x-coordinate and y-coordinate of the starting point of the first lane curve, respectively. e1 and y e1 Let f1, f2, and f3 represent the x and y coordinates of the endpoint of the first lane curve, respectively; f2(x,y) = 0 represents the x-coordinate of the second lane curve. s2 and y s2 Let x represent the x-coordinate and y-coordinate of the starting point of the second lane curve, respectively. e2 and y e2 These represent the x-coordinate and y-coordinate of the endpoint of the second lane curve, respectively.

[0050] To determine whether two lane curves can be joined, a preset matching rule can be used to match multiple lane lines in the first coordinate system pairwise. The preset matching rule can be set according to the characteristics of the lane lines. For example, in the bird's-eye view coordinate system, the vehicle center is the origin, the direction directly in front of the vehicle is the vertical axis (y-axis), and the direction directly to the right of the vehicle is the horizontal axis (x-axis). Lane lines from different perspectives extend in the vertical direction (y-direction) but are very close in the horizontal direction. The corresponding matching rule is set accordingly to determine the two lane curves that need to be joined.

[0051] Step 202: Determine the splicing constraint points based on the first lane line curve and the second lane line curve.

[0052] Due to potential errors in lane line detection, fitting, and coordinate transformation from different perspectives, the first and second lane line curves may be discontinuous or uneven at their connection points. Splicing constraint points are used to constrain the adjustment process of the first and second lane line curves, ensuring a smooth and continuous transition between them. These constraint points can be determined based on the start and end points of the first and second lane line curves, also known as endpoints. For example, the splicing point can be determined based on the first start and end points of the first lane line curve and the second start and end points of the second lane line curve. If the first end point of the first lane line curve and the second start point of the second lane line curve need to be spliced, the splicing constraint point is determined based on the positional relationship between the first end point and the second start point. For example, the first lane curve and the second lane curve can have two longitudinal directions: overlapping (i.e., the ordinate of the first termination point is within the ordinate range of the second lane curve, and the ordinate of the second starting point is within the ordinate range of the first lane curve) or gap (i.e., the ordinate of the first termination point is outside the ordinate range of the second lane curve, and the ordinate of the second starting point is outside the ordinate range of the first lane curve). The corresponding splicing constraint point is determined based on the different situations. For example, in the case of a gap, the center of the gap area can be determined based on the first termination point and the second starting point as the splicing constraint point. The specific setting can be determined according to actual needs.

[0053] Step 203: Based on the splicing constraint points, adjust the first curve parameters corresponding to the first lane line curve to obtain the adjusted first target curve parameters corresponding to the first curve parameters.

[0054] Among them, the first curve parameter corresponding to the first lane line curve is a multiple curve coefficient used to represent the first lane line curve, such as a cubic curve coefficient. The principle of adjusting the first curve parameter is to make the first target curve corresponding to the adjusted first target curve parameter be able to be continuously and smoothly spliced ​​with the adjusted second target curve at the splicing constraint point.

[0055] For example, the first lane line curve is a cubic curve, and the parameter of the first curve is C. 10 C 11 C 12 C 13 Then the curve of the first lane can be represented as:

[0056] x = C 10 +C 11 y+C 12 y 2 +C 13 y 3

[0057] Optionally, the parameters of the first curve can be adjusted using any feasible optimization method, such as constrained extended Kalman filter algorithm, nonlinear optimization, etc., which can be set according to actual needs.

[0058] Step 204: Based on the splicing constraint points, adjust the second curve parameters corresponding to the second lane line curve to obtain the adjusted second target curve parameters.

[0059] Here, the second curve parameter corresponding to the second lane line curve is a multiple curve coefficient, such as a cubic curve coefficient, used to represent the second lane line curve. Similarly, the principle of adjusting the second curve parameter is to ensure that the second target curve corresponding to the adjusted second target curve parameter can be continuously and smoothly spliced ​​with the adjusted first target curve at the splicing constraint point.

[0060] For example, the second lane line curve is a cubic curve, and the parameter of the second curve is C. 20 C 21 C 22 C 23 Then the curve of the second lane can be represented as:

[0061] x = C 20 +C 21 y+C 22 y 2 +C 23 y 3

[0062] It should be noted that steps 203 and 204 are not in any particular order.

[0063] Step 205: Determine the spliced ​​lane lines based on the first target curve parameters and the second target curve parameters.

[0064] The first target curve parameter and the second target curve parameter are curve parameters adjusted based on the splicing constraint points. The first target curve and the second target curve corresponding to them can be continuously and smoothly spliced ​​at the splicing constraint points to obtain the spliced ​​lane lines. Based on this, the lane line curves that need to be spliced ​​in the first coordinate system corresponding to each viewpoint can be spliced ​​to obtain a smooth and continuous surround view lane line map in the first coordinate system.

[0065] In one optional example, the first coordinate system is the coordinate system corresponding to the bird's-eye view. After obtaining the stitched lane lines, they can be merged with other lane lines to obtain a surround-view lane line map under the bird's-eye view. The surround-view lane line map under the bird's-eye view is a lane line map of a local area around the vehicle. In practical applications, the surround-view lane line map under the bird's-eye view can also be converted to a global coordinate system (such as the world coordinate system) so that a global lane line map can be obtained over time. The specific settings can be configured according to actual needs.

[0066] The lane line splicing method provided in this embodiment determines splicing constraint points based on the curves of the two lane lines to be spliced, and adjusts the curve parameters of the two lane line curves based on the splicing constraint points, so that the adjusted two lane lines can be spliced ​​smoothly and continuously, effectively improving the effect of the merged lane lines.

[0067] Figure 3 This is a flowchart illustrating a lane line splicing method provided in another exemplary embodiment of this disclosure.

[0068] In one optional example, step 203 may specifically include the following steps:

[0069] Step 2031: Based on the splicing constraint points, the parameters of the first curve are adjusted using the constrained extended Kalman filter algorithm to obtain the parameters of the first target curve.

[0070] The constrained extended Kalman filter algorithm uses the first curve parameters as the initial state variables. Based on the splicing constraint points and the conditions required for the continuous and smooth splicing of the first and second lane curves, corresponding constraints are set. Iterative filtering achieves state transition, updates the first curve parameters, and obtains the updated first target curve parameters, ensuring that the updated first target curve parameters satisfy the continuous and smooth splicing constraints. The constraints can include lateral coordinate constraints and first-order partial derivative constraints at the splicing constraint points. The lateral coordinate constraints ensure that the first target curve corresponding to the first target curve parameters includes the splicing constraint points. The first-order partial derivative constraints ensure that the first partial derivative of the first target curve at the splicing constraint points is equal to that of the second target curve, thus enabling the continuous and smooth splicing of the first and second target curves.

[0071] In an optional example, step 204 may specifically include the following steps:

[0072] Step 2041: Based on the splicing constraint points, the parameters of the second curve are adjusted using the constrained extended Kalman filter algorithm to obtain the parameters of the second target curve.

[0073] The specific principle of step 2041 is explained in step 2031, and will not be repeated here.

[0074] This disclosure employs a constrained extended Kalman filter algorithm to adjust the parameters of the two curves to be stitched, so that the adjusted target curves can effectively meet the requirements of smoothness and continuity of the stitching, thereby further improving the lane line fusion effect.

[0075] In one optional example, Figure 4 This is a flowchart illustrating step 2031 provided in an exemplary embodiment of this disclosure. In this example, step 2031, based on splicing constraint points, uses a constrained extended Kalman filter algorithm to adjust the first curve parameters to obtain the first target curve parameters, including:

[0076] Step 20311: Determine the initial state variables based on the parameters of the first curve.

[0077] The initial state variables are used as the initial state variables for the constrained extended Kalman filter algorithm.

[0078] For example, for the first curve parameter C 10 C 11 C 12 C 13 The determined initial state quantity S 10 It can be represented as:

[0079]

[0080] Step 20312: Based on the splicing constraint points, the first curve parameters, and the second curve parameters, determine the constraint quantities. The constraint quantities include the horizontal coordinate values ​​of the splicing constraint points and the first-order partial derivative constraint values ​​at the vertical coordinate values ​​of the splicing constraint points.

[0081] The constraint values ​​are used to constrain the adjustment of the first curve parameters, ensuring that the adjusted first target curve parameters satisfy the continuous and smooth splicing constraint. The lateral coordinate values ​​of the splicing constraint points serve as lateral coordinate constraints, ensuring that the first target curve corresponding to the first target curve parameters can contain the splicing constraint points, i.e., the splicing constraint points lie on the first target curve. The first-order partial derivative constraint values ​​serve as first-order partial derivative constraints, ensuring that the first-order partial derivative of the first target curve at the splicing constraint points is the same as that of the second target curve, thereby enabling the first and second target curves to be continuously and smoothly spliced.

[0082] For example, the splicing constraint point is represented as (x c ,y c The first curve parameter is C. 10 C 11 C 12 C 13 The parameter of the second curve is C. 20 C 21 C 22C 23 The curve of the first lane is then:

[0083] x = C 10 +C 11 y+C 12 y 2 +C 13 y 3

[0084] The curve of the second lane is:

[0085] x = C 20 +C 21 y+C 22 y 2 +C 23 y 3

[0086] The constraint is expressed as:

[0087]

[0088] Among them, f c ′(x,y) yc Represents the vertical coordinate value y c The first-order partial derivative constraint value at the point can be based on the curve of the first lane line at y. c The first partial derivative at y and the second lane curve c The first-order partial derivative at a given point is determined, and can be set according to actual needs. For example, the average of two first-order partial derivatives can be taken, as shown below:

[0089]

[0090] in, This indicates that the curve of the first lane is at y=y c The first-order partial derivative at that point, This indicates that the curve of the second lane is at y=y c The first-order partial derivative at point is expressed as follows:

[0091]

[0092]

[0093] When the curve parameters of the first lane line, the curve parameters of the second lane line, and y c Once determined, the first-order partial derivative constraint value can be determined.

[0094] Steps 20311 and 20312 are not in any particular order.

[0095] Step 20313: Determine the initial transition matrix from the initial state variables to the constraint variables.

[0096] The initial state transition matrix represents the transition relationship from the initial state quantity to the constraint quantity.

[0097] For example, the initial transition matrix is ​​represented as follows:

[0098]

[0099] Step 20314: Determine the Kalman gain based on the initial transition matrix.

[0100] Here, the Kalman gain characterizes the weighting of observation bias and can be expressed as K. t It is used to update the state variables, and through iterative adjustments, the state variables are made to meet the constraints of the constraints. The specific method for determining the Kalman gain can be set according to actual needs.

[0101] Step 20315: Determine the target state variables based on Kalman gain, constraint variables, initial transition matrix, and initial state variables.

[0102] After determining the Kalman gain, the state variables can be updated based on the Kalman gain, constraint variables, initial transition matrix, initial state variables, and preset update formula to obtain the updated state variables. If the updated state variables can meet the preset conditions or reach the preset number of iterations, the updated state variables can be used as the target state variables. Otherwise, the state variables are updated based on the above process until the updated state variables meet the preset conditions or reach the preset number of iterations to obtain the target state variables.

[0103] Step 20316: Determine the parameters of the first target curve based on the target state variables.

[0104] The target state variables include the state values, which are the updated parameters of the first target curve. For example, the target state variables are represented as follows:

[0105]

[0106] Where N represents the number of iterations, the parameters of the first target curve are then...

[0107] In one optional example, Figure 5 This is a flowchart illustrating step 2041 provided in an exemplary embodiment of this disclosure. In this example, step 2041, based on the splicing constraint points, uses a constrained extended Kalman filter algorithm to adjust the second curve parameters to obtain the second target curve parameters, including:

[0108] Step 20411: Determine the initial state variables based on the parameters of the second curve.

[0109] Step 20412: Based on the splicing constraint points, the first curve parameters, and the second curve parameters, determine the constraint quantities. The constraint quantities include the horizontal coordinate values ​​of the splicing constraint points and the first-order partial derivative constraint values ​​at the vertical coordinate values ​​of the splicing constraint points.

[0110] Step 20413: Determine the initial transition matrix from the initial state variables to the constraint variables.

[0111] Step 20414: Determine the Kalman gain based on the initial transition matrix.

[0112] Step 20415: Determine the target state variables based on Kalman gain, constraint variables, initial transition matrix, and initial state variables.

[0113] Step 20416: Determine the parameters of the first target curve based on the target state variables.

[0114] The specific operations of steps 20411 to 20416 in this example are the same as those of steps 20311 to 20316 mentioned above. The difference is that the initial state quantity in this example is the second curve parameter of the second lane line curve. The specific principle will not be repeated here.

[0115] In an optional example, step 20315, which determines the target state variable based on Kalman gain, constraint, initial transition matrix, and initial state variable, includes: iteratively updating the initial state variable based on Kalman gain, constraint, and initial transition matrix until a preset iteration termination condition is met, thereby obtaining the target state variable.

[0116] The preset iteration termination conditions can include the maximum number of iterations and / or the conditions that the covariance matrix needs to meet, which can be set according to actual needs.

[0117] In one optional example, Figure 6 This is a schematic diagram of a state variable update iteration process provided in an exemplary embodiment of this disclosure. In this example, based on Kalman gain, constraint quantities, and an initial transition matrix, the initial state variable is iteratively updated until a preset iteration termination condition is met to obtain the target state variable, including:

[0118] During the iteration process, taking any iteration as the current iteration, perform the following steps:

[0119] Step 301: Based on the new state variables obtained in the previous iteration, determine the current curve parameters corresponding to the first curve parameters.

[0120] Specifically, for the current iteration, the new state variables obtained in the previous iteration include the updated curve parameters corresponding to the first curve parameters obtained in the previous update, and these updated curve parameters are used as the current curve parameters for the current iteration. If the current iteration is the first iteration, then the new state variables obtained in the previous iteration are the initial state variables.

[0121] Step 302: Determine the current state variable based on the current curve parameters.

[0122] For example, the current state variable is represented as:

[0123]

[0124] Where i represents the current iteration as the i-th iteration.

[0125] In practical applications, the new state value obtained in the previous iteration can be directly used as the current state value of the current iteration.

[0126] Step 303: Determine the current transition matrix from the current state variable to the constraint variable.

[0127] The constraints are fixed and invariant, as shown in C_s above. Since the state variables represent curve parameters, changes in the curve parameters do not affect the transition matrix; therefore, the current transition matrix is ​​the same as the initial transition matrix.

[0128] For example, the current transition matrix is ​​represented as follows:

[0129]

[0130] The meanings of each symbol are as described above.

[0131] Step 304: Determine the current Kalman gain based on the current transition matrix, the new covariance matrix obtained in the previous iteration, and the preset constraint noise matrix.

[0132] The covariance matrix is ​​a diagonal matrix, and the initial covariance matrix can be a preset diagonal matrix, which is continuously updated during the iteration process until convergence. The preset constraint noise matrix represents the degree of confidence in the constraint. The preset constraint noise matrix is ​​a diagonal matrix; when all its values ​​are 0, it indicates a hard constraint, meaning the first target curve parameters are obtained through one iteration. When its diagonal values ​​are not 0, it indicates a soft constraint, and the end of the iteration needs to be controlled by the maximum number of iterations, which can be set according to actual needs. The current Kalman matrix can be determined based on the preset Kalman gain formula. If the current iteration is the first iteration, the new covariance matrix obtained in the previous iteration is the preset initial covariance matrix.

[0133] For example, the initial covariance matrix is ​​represented as:

[0134]

[0135] Where, λ 1,0 -λ 4,0 This is the default value.

[0136] The new covariance matrix obtained in the previous iteration is represented as:

[0137]

[0138] Where, λ 1,i-1 -λ 4,i-1 This is the value obtained in the previous iteration.

[0139] The preset constraint noise matrix C_n0 is represented as:

[0140]

[0141] β1 and β2 are preset values, such as 0.001 or 0, which can be set according to actual needs.

[0142] Then the current Kalman gain K ti It is expressed as follows:

[0143]

[0144] The superscript T indicates transpose.

[0145] Step 305: Based on the current Kalman gain, constraints, and current transition matrix, update the current state variables to obtain the new state variables for the current iteration.

[0146] For example, the state update formula is expressed as follows:

[0147] S 1i =S 1i-1 +K ti (C_s-C_t i *S 1i-1 )

[0148] Among them, S 1i This represents the new state variable for the current iteration. The meanings of the other symbols are explained above. If the current iteration is the first iteration, the current state variable is the initial state variable S. 10 .

[0149] Step 306: Based on the current Kalman gain and the current transition matrix, update the new covariance matrix obtained in the previous iteration to obtain the new covariance matrix for the current iteration.

[0150] For example, the formula for updating the covariance matrix is ​​expressed as follows:

[0151] E_ci =E_c i-1 -K t *C_t i *E_c i-1

[0152] Among them, E_c i This represents the new covariance matrix obtained in the current iteration.

[0153] Step 307: In response to the current iteration reaching the preset number of iterations and / or the new covariance matrix of the current iteration satisfying the preset condition, the iteration process ends, and the new state variable of the current iteration is taken as the target state variable.

[0154] The preset number of iterations and preset conditions can be set according to actual needs to control the end of the iteration process. After the iteration process is completed, the target state is obtained.

[0155] Understandably, the state update and iteration process of the second lane curve is similar to steps 301-307 above, as detailed above, and will not be repeated here.

[0156] It should be noted that the specific example in this disclosure uses a cubic curve as an example. In practical applications, the first and second lane curves to be spliced ​​can also be curves of other orders of equations, such as straight lines or higher-order curves. For different types of lane curves, the corresponding number of state variables and other required data can be determined according to the number of their curve parameters, without any specific limitation. For example, for a quartic curve, the curve parameters include 5 coefficients, and the corresponding initial state variables include 5 state values, which can be set according to actual needs.

[0157] In an optional example, step 20314, determining the Kalman gain based on the initial transition matrix, includes: determining the Kalman gain based on the initial transition matrix, a preset initial covariance matrix, and a preset constraint noise matrix, wherein the preset initial covariance matrix is ​​a diagonal matrix and the preset constraint noise matrix is ​​a diagonal matrix used to represent the confidence level of the constraint.

[0158] For the specific principle of determining the Kalman gain, please refer to the detailed explanation of step 304 above. This example illustrates the determination of the Kalman gain during the first iteration (i.e., i=1), and will not be elaborated further here.

[0159] This disclosure uses a constrained extended Kalman filter algorithm to adjust the curve parameters of the first and second lane curves to be spliced. By continuously updating the constraint and state variables, the two lane curves gradually approach the splicing constraint point until they can be connected continuously and smoothly at the splicing constraint point, making the spliced ​​lane lines more consistent with the continuous and smooth state of real lane lines.

[0160] In one optional example, Figure 7 This is a schematic diagram of the constraint quantity determination process provided by an exemplary embodiment of this disclosure. In this example, step 20412, which determines the constraint quantity based on the spliced ​​constraint points, the first curve parameter, and the second curve parameter, includes:

[0161] a. Based on the parameters of the first curve, determine the first-order partial derivative of the first lane line curve at the longitudinal coordinate value of the splicing constraint point.

[0162] For example, the first-order partial derivative It is expressed as follows:

[0163]

[0164] Among them, y c This represents the longitudinal coordinate value of the splicing constraint point, and the first curve parameter is C. 10 C 11 C 12 C 13 .

[0165] b. Based on the parameters of the second curve, determine the second first-order partial derivative of the second lane line curve at the longitudinal coordinate value of the splicing constraint point.

[0166] For example, the second first-order partial derivative is expressed as follows:

[0167]

[0168] Among them, y c This represents the longitudinal coordinate value of the splicing constraint point, and the second curve parameter is C. 20 C 21 C 22 C 23 .

[0169] c. Determine the constraint value of the first-order partial derivative based on the first and second-order partial derivatives.

[0170] For example, the first-order partial derivative constraint values ​​are expressed as follows:

[0171]

[0172] d. Determine the constraint quantities based on the lateral coordinates and first-order partial derivative constraint values ​​of the splicing constraint points.

[0173] For example, the constraint is represented as follows:

[0174]

[0175] This disclosure achieves a smooth connection between the first and second target curves at the splicing constraint point by adjusting the lateral coordinate values ​​of the splicing constraint point in the constraint quantity. This is accomplished by adjusting the first partial derivative constraint value in the constraint quantity, ensuring that the first and second target curves at the splicing constraint point are both equal to the first partial derivative constraint value. In other words, the first partial derivatives of the first and second target curves at the splicing constraint point are equal, thus achieving a smooth connection between the first and second target curves at the splicing constraint point. Therefore, through the constraint quantity, the obtained first and second target curves can be spliced ​​continuously and smoothly, further improving the splicing effect and making the spliced ​​lane line more consistent with the state of the real lane line.

[0176] In an optional example, step 202, determining the splicing constraint points based on the first lane line curve and the second lane line curve, includes:

[0177] Step 2021: Determine the longitudinal coordinate values ​​of the splicing constraint points based on the first lane line curve and the second lane line curve.

[0178] Specifically, the longitudinal coordinate value y of the splicing constraint point c It can be determined based on the ordinates of the endpoints of the first lane curve and the second lane curve, or it can be determined based on the overlapping area or gap area of ​​the first lane curve and the second lane curve. The specific setting can be determined according to actual needs.

[0179] Step 2022: Based on the longitudinal coordinate value, determine the first lateral coordinate value of the first lane line curve at the longitudinal coordinate value.

[0180] After determining the longitudinal coordinate value of the splicing constraint point, the first lateral coordinate value x1 of the first lane line curve at that longitudinal coordinate value can be determined based on the longitudinal coordinate value, for example, it can be represented as follows:

[0181] x1=C 10 +C 11 y c +C 12 y c 2 +C 13 y c 3

[0182] Among them, C 10 C 11 C 12 C 13 The first curve parameter is the first curve parameter of the first lane line curve.

[0183] Step 2023: Based on the longitudinal coordinate value, determine the second lateral coordinate value of the second lane line curve at the longitudinal coordinate value.

[0184] For example, the second horizontal coordinate value is represented as follows:

[0185] x2=C 20 +C 21 y c +C 22 y c 2 +C 23 y c 3

[0186] Among them, C 20 C 21 C 22 C 23 The second curve parameter is the second curve parameter of the second lane line curve.

[0187] It should be noted that steps 2022 and 2023 are not in any particular order.

[0188] Step 2024: Determine the horizontal coordinate values ​​of the splicing constraint points based on the first and second horizontal coordinate values.

[0189] It should be noted that since the longitudinal coordinates of the splicing constraint point are determined based on the first and second lane curves, when the first and second lane curves do not overlap longitudinally but have a certain gap, the splicing constraint point may not be on the first or second lane curves, but rather in the gap between them. Therefore, the determined first and second lateral coordinates are obtained by extending the corresponding lane curves longitudinally to the longitudinal coordinates of the splicing constraint point. The obtained first and second lateral coordinates may not be equal. Alternatively, when the two lane curves overlap longitudinally, the longitudinal coordinates are determined based on the overlapping area, or based on the two endpoints of the overlap. The determined first and second lateral coordinates are then the lateral coordinates of the cutoff points of the two lane curves at that longitudinal coordinate value, which may also result in the first and second lateral coordinates of the two lane curves at that longitudinal coordinate value being different. Therefore, it is necessary to determine the lateral coordinates of the splicing constraint point of the two lane curves based on the first and second lateral coordinates.

[0190] For example, Figure 8 This is a schematic diagram illustrating the longitudinal overlap of the first lane line curve and the second lane line curve provided in an exemplary embodiment of this disclosure, wherein (x1, y c ) indicates that the curve of the first lane is at y = yc The cutoff point at (x2, y) c ) indicates that the curve of the second lane is at y=y c The cutoff point at that location. Figure 9 This is a schematic diagram of the longitudinal gap between the first lane line curve and the second lane line curve provided in an exemplary embodiment of this disclosure, wherein (x1, y c ) indicates that the curve of the first lane is at y = y c The extension point at (x2, y) c ) indicates that the curve of the second lane is at y=y c The extension point at the location. The lateral coordinate x of the splicing constraint point. c It can be represented as follows:

[0191] x c = (x1+x2) / 2

[0192] Step 2025: Determine the splicing constraint points based on the horizontal and vertical coordinate values ​​of the splicing constraint points.

[0193] In summary, the obtained splicing constraint point is (x c ,y c ).

[0194] This disclosure first determines the longitudinal coordinate value of the splicing constraint point based on the longitudinal extension characteristics of the lane lines. Then, it determines the lateral coordinate value of the splicing constraint point based on the average of the lateral coordinate values ​​of the two lane line curves at the longitudinal coordinate value. This allows the splicing constraint point to constrain the two lane line curves to move closer to the splicing constraint point during the adjustment of the curve parameters, until the two lane line curves can be continuously and smoothly connected at the splicing constraint point. This effectively improves the adjustment effect and avoids reducing the accuracy of the lane lines due to excessive adjustment of one lane line curve.

[0195] In one optional example, Figure 10 This is a flowchart illustrating step 2021 provided in an exemplary embodiment of this disclosure. Determining the longitudinal coordinate values ​​of the splicing constraint point based on the first lane line curve and the second lane line curve includes:

[0196] Step 20211: Determine the ordinates of the first and second endpoints of the first lane curve, as well as the ordinates of the third and fourth endpoints of the second lane curve.

[0197] Wherein, the ordinates of the first endpoint and the second endpoint are respectively the ordinates of the first starting point of the first lane curve. s1 and the y-coordinate of the first termination point e1 The ordinates of the third and fourth endpoints are respectively the ordinates of the second starting points of the second lane curve. s2 The y-coordinate of the second termination pointe2 The ordinates of each endpoint can be obtained from the coordinate range of the first lane curve and the second lane curve; details will not be elaborated further.

[0198] Step 20212: Based on the ordinates of the first endpoint, the second endpoint, the third endpoint, and the fourth endpoint, determine the ordinates of the first target endpoint and the second target endpoint adjacent to the first lane line curve and the second lane line curve.

[0199] Specifically, the positional relationship between the two lane curves can be determined first by using the coordinates of each endpoint, such as overlap or gap. This can be determined by the range of the ordinates or by comparing the magnitudes of the ordinates of each endpoint. Then, the ordinates of the endpoints where the two lane curves need to be joined can be determined, which are the ordinates of the first target endpoint and the ordinates of the second target endpoint.

[0200] For example, if y s1 <y e1 <y s2 <y e2 , or y s1 <y s2 <y e1 <y e2 , then y e1 With y s2 These are the ordinates of the first target endpoint and the second target endpoint, respectively.

[0201] Step 20213: Determine the longitudinal coordinate values ​​of the splicing constraint points based on the longitudinal coordinates of the first target endpoint and the second target endpoint.

[0202] For example, once the ordinates of the first target endpoint and the second target endpoint are determined, the average of the ordinates of the first target endpoint and the second target endpoint can be used as the longitudinal coordinate value of the splicing constraint point.

[0203] In an optional example, step 201, determining the first lane line curve and the second lane line curve to be spliced ​​in the first coordinate system, includes:

[0204] Step 2011: Obtain first image data corresponding to at least two viewpoints.

[0205] The first image data can be acquired by cameras with at least two perspectives mounted on the vehicle.

[0206] For example, while the vehicle is in motion, the surround-view cameras installed on the vehicle collect first image data corresponding to each viewpoint of the surrounding road environment in real time. This first image data corresponding to at least two viewpoints can be collected by each camera simultaneously. For instance, it could be the first image data collected by each camera at the current moment the vehicle is traveling. Alternatively, it could be first image data from at least two viewpoints collected and stored in advance at the same time; the specific method is not limited.

[0207] Step 2012: Based on each first image data, determine the lane line region in the image coordinate system corresponding to each first image data.

[0208] The lane line region in the image coordinate system includes the set of pixels belonging to the lane line in the image coordinate system. The lane line region in the image coordinate system corresponding to each first image data can be detected by a pre-trained target detection model. The details will not be elaborated further.

[0209] Step 2013: Convert the lane line regions corresponding to each first image data to the first coordinate system to obtain the lane line coordinate points in the first coordinate system corresponding to each first image data.

[0210] The first coordinate system can be the coordinate system corresponding to the bird's-eye view (BEV). Since the lane line areas of each view are based on their respective image coordinate systems, in order to achieve the fusion of the views, they need to be transformed to a unified coordinate system. In order to utilize the longitudinal extension of the lane lines to determine the stitching constraint points, this disclosure adopts the coordinate system corresponding to the bird's-eye view as the first coordinate system, with the origin at the vehicle center. The lane line curves of each view are transformed to a unified first coordinate system so that the lane lines of each view can be fused to obtain a surround-view lane line map around the vehicle. The transformation from the image coordinate system to the first coordinate system can be achieved according to the mapping relationship between the two coordinate systems; the specific principle will not be elaborated here.

[0211] Step 2014: Based on the lane line coordinate points in the first coordinate system corresponding to each first image data, determine the lane line curve in the first coordinate system corresponding to each first image data.

[0212] In this process, pixels in the lane line region of the image coordinate system are transformed to lane line coordinate points in the first coordinate system. Then, the lane line curves in the first coordinate system corresponding to each first image data can be obtained through fitting. The specific fitting principle will not be elaborated further.

[0213] Step 2015: Based on the lane line curves in the first coordinate system corresponding to each of the first image data, determine the first lane line curve and the second lane line curve to be stitched in the first coordinate system.

[0214] Whether two lane curves can be spliced ​​can be determined by using preset matching rules to match multiple lane curves in the first coordinate system. The preset matching rules can be set according to the characteristics of the lane lines. For example, in the bird's-eye view coordinate system, lane curves from different perspectives extend in the longitudinal direction (y direction) and the distance in the lateral direction is relatively small. Accordingly, the matching rules are set to determine the two lane curves that need to be spliced.

[0215] For example, one lane curve in a two-lane road pattern can be sampled to obtain multiple sampling points. The distance between these multiple sampling points and the other lane curve is less than a certain threshold, and the difference between the coefficients of the first term of the two lane curves (such as the C of the first lane curve mentioned above) can also be considered. 11 C with the curve of the second lane 21 The gap between them meets certain conditions, and the specific matching principle will not be elaborated here.

[0216] The lane line stitching method provided in this disclosure determines stitching constraint points and uses a constrained extended Kalman filter algorithm to constrain the continuity and smoothness of the two lane lines at the stitching constraint points, thereby ensuring that the stitched lane lines are continuous and smooth at the stitching points. Furthermore, the constrained extended Kalman filter algorithm has low latency and computational cost, ensuring real-time and effective lane line stitching and providing better surround-view lane line data for subsequent applications. In addition, since each camera in this disclosure independently perceives lane lines, different segments of the same lane line can be stitched together to obtain a complete lane line during the fusion process. The determination of stitching constraint points allows lane lines from different perspectives to be extended or truncated, thus giving each camera a certain fault tolerance. When one or more cameras malfunction, the middle part of the lane line can be obtained by stitching together the segments of the same lane line from other cameras, thus not affecting the overall lane line output.

[0217] The embodiments or optional examples disclosed above can be implemented individually or in any combination without conflict. The specific implementation can be set according to actual needs, and this disclosure does not limit it.

[0218] Any lane line splicing method provided in this disclosure can be executed by any suitable device with data processing capabilities, including but not limited to: terminal devices and servers. Alternatively, any lane line splicing method provided in this disclosure can be executed by a processor, such as by a processor executing any lane line splicing method mentioned in this disclosure by calling corresponding instructions stored in memory. Further details will not be elaborated below.

[0219] Exemplary device

[0220] Figure 11This is a schematic diagram of a lane line splicing device provided in an exemplary embodiment of the present disclosure. The device in this embodiment can be used to implement corresponding method embodiments of the present disclosure, such as… Figure 11 The device shown includes: a first determining module 501, a first processing module 502, a second processing module 503, a third processing module 504, and a fourth processing module 505.

[0221] A first determining module 501 is used to determine the first lane line curve and the second lane line curve to be spliced ​​in a first coordinate system; a first processing module 502 is used to determine splicing constraint points based on the first lane line curve and the second lane line curve determined by the first determining module 501; a second processing module 503 is used to adjust the first curve parameter corresponding to the first lane line curve based on the splicing constraint points determined by the first processing module 502 to obtain the adjusted first target curve parameter corresponding to the first curve parameter; a third processing module 504 is used to adjust the second curve parameter corresponding to the second lane line curve based on the splicing constraint points determined by the first processing module 502 to obtain the adjusted second target curve parameter corresponding to the second curve parameter; and a fourth processing module 505 is used to determine the spliced ​​lane line based on the first target curve parameter and the second target curve parameter.

[0222] Figure 12 This is a schematic diagram of the structure of a lane line splicing device provided in another exemplary embodiment of this disclosure.

[0223] In an optional example, the second processing module 503 includes: a first processing unit 5031, used to adjust the first curve parameters based on the splicing constraint points using a constrained extended Kalman filter algorithm to obtain the first target curve parameters.

[0224] In an optional example, the third processing module 504 includes a second processing unit 5041, which is used to adjust the second curve parameters based on the splicing constraint points using a constrained extended Kalman filter algorithm to obtain the second target curve parameters.

[0225] In one optional example, Figure 13 This is a schematic diagram of the structure of a first processing unit 5031 provided in an exemplary embodiment of the present disclosure. In this example, the first processing unit 5031 includes: a first determining subunit 50311, a second determining subunit 50312, a third determining subunit 50313, a fourth determining subunit 50314, a fifth determining subunit 50315, and a sixth determining subunit 50316.

[0226] The first determining subunit 50311 is used to determine the initial state quantity based on the first curve parameter; the second determining subunit 50312 is used to determine the constraint quantity based on the splicing constraint point, the first curve parameter, and the second curve parameter, wherein the constraint quantity includes the horizontal coordinate value of the splicing constraint point and the first-order partial derivative constraint value at the vertical coordinate value of the splicing constraint point; the third determining subunit 50313 is used to determine the initial transition matrix from the initial state quantity to the constraint quantity; the fourth determining subunit 50314 is used to determine the Kalman gain based on the initial transition matrix; the fifth determining subunit 50315 is used to determine the target state quantity based on the Kalman gain, the constraint quantity, the initial transition matrix, and the initial state quantity; the sixth determining subunit 50316 is used to determine the first target curve parameter based on the target state quantity.

[0227] In an optional example, the fifth determining subunit 50315 is specifically used to: iteratively update the initial state quantity based on the Kalman gain, the constraint quantity, and the initial transition matrix until a preset iteration termination condition is met, thereby obtaining the target state quantity.

[0228] In an optional example, the fifth determining subunit 50315 is specifically used to: during the iteration process, taking any iteration as the current iteration, perform the following steps: based on the new state quantity obtained in the previous iteration, determine the current curve parameter corresponding to the first curve parameter; based on the current curve parameter, determine the current state quantity; determine the current transition matrix from the current state quantity to the constraint quantity; based on the current transition matrix, the new covariance matrix obtained in the previous iteration, and the preset constraint quantity noise matrix, determine the current Kalman gain; based on the current Kalman gain, the constraint quantity, and the current transition matrix, update the current state quantity to obtain the new state quantity of the current iteration; based on the current Kalman gain and the current transition matrix, update the new covariance matrix obtained in the previous iteration to obtain the new covariance matrix of the current iteration; in response to the current iteration reaching a preset number of iterations, and / or the new covariance matrix of the current iteration satisfying a preset condition, end the iteration process, and take the new state quantity of the current iteration as the target state quantity.

[0229] In an optional example, the fourth determining subunit 50314 is specifically used to: determine the Kalman gain based on the initial transition matrix, the preset initial covariance matrix, and the preset constraint noise matrix, wherein the preset initial covariance matrix is ​​a diagonal matrix and the preset constraint noise matrix is ​​a diagonal matrix used to represent the confidence level of the constraint.

[0230] In an optional example, the second determining subunit 50312 is specifically used to: determine the first first-order partial derivative of the first lane line curve at the longitudinal coordinate value of the splicing constraint point based on the first curve parameters; determine the second first-order partial derivative of the second lane line curve at the longitudinal coordinate value of the splicing constraint point based on the second curve parameters; determine the first-order partial derivative constraint value based on the first and second first-order partial derivatives; and determine the constraint amount based on the lateral coordinate value of the splicing constraint point and the first-order partial derivative constraint value.

[0231] In an optional example, the first processing module 502 includes: a first determining unit 5021, a second determining unit 5022, a third determining unit 5023, a fourth determining unit 5024, and a fifth determining unit 5025.

[0232] A first determining unit 5021 is used to determine the longitudinal coordinate value of the splicing constraint point based on the first lane line curve and the second lane line curve; a second determining unit 5022 is used to determine the first lateral coordinate value of the first lane line curve at the longitudinal coordinate value based on the longitudinal coordinate value; a third determining unit 5023 is used to determine the second lateral coordinate value of the second lane line curve at the longitudinal coordinate value based on the longitudinal coordinate value; a fourth determining unit 5024 is used to determine the lateral coordinate value of the splicing constraint point based on the first lateral coordinate value and the second lateral coordinate value; and a fifth determining unit 5025 is used to determine the splicing constraint point based on the lateral coordinate value and the longitudinal coordinate value of the splicing constraint point.

[0233] In an optional example, the first determining unit 5021 is specifically configured to: determine the ordinates of the first and second endpoints of the first lane line curve, and the ordinates of the third and fourth endpoints of the second lane line curve; based on the ordinates of the first, second, third, and fourth endpoints, determine the ordinates of the first and second target endpoints adjacent to the first and second lane line curves; and based on the ordinates of the first and second target endpoints, determine the longitudinal coordinate values ​​of the splicing constraint points.

[0234] In an optional example, the first determining module 501 includes: a first acquiring unit 5011, a sixth determining unit 5012, a third processing unit 5013, a seventh determining unit 5014, and an eighth determining unit 5015.

[0235] The first acquisition unit 5011 is used to acquire first image data corresponding to at least two viewpoints respectively; the sixth determination unit 5012 is used to determine lane line regions in the image coordinate system corresponding to each of the first image data based on each of the first image data; the third processing unit 5013 is used to transform the lane line regions corresponding to each of the first image data to the first coordinate system to obtain lane line coordinate points in the first coordinate system corresponding to each of the first image data; the seventh determination unit 5014 is used to determine lane line curves in the first coordinate system corresponding to each of the first image data based on the lane line coordinate points in the first coordinate system corresponding to each of the first image data; the eighth determination unit 5015 is used to determine the first lane line curve and the second lane line curve to be stitched in the first coordinate system based on the lane line curves in the first coordinate system corresponding to each of the first image data.

[0236] Exemplary electronic devices

[0237] This disclosure also provides an electronic device, including: a memory for storing computer programs;

[0238] A processor is configured to execute a computer program stored in the memory, and when the computer program is executed, to implement the lane line splicing method described in any of the above embodiments of the present disclosure.

[0239] Figure 14 This is a schematic diagram of an application embodiment of the electronic device disclosed herein. In this embodiment, the electronic device 10 includes one or more processors 11 and a memory 12.

[0240] The processor 11 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.

[0241] The memory 12 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 11 may execute the program instructions to implement the methods of the various embodiments of this disclosure described above and / or other desired functions. Various contents such as input signals, signal components, and noise components may also be stored in the computer-readable storage medium.

[0242] In one example, the electronic device 10 may also include an input device 13 and an output device 14, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0243] For example, the input device 13 may be the microphone or microphone array described above, used to capture the input signal of the sound source.

[0244] In addition, the input device 13 may also include, for example, a keyboard, a mouse, etc.

[0245] The output device 14 can output various information to the outside, including determined distance information, direction information, etc. The output device 14 may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0246] Of course, for the sake of simplicity, Figure 14 Only some of the components of the electronic device 10 relevant to this disclosure are shown, omitting components such as buses, input / output interfaces, etc. In addition, the electronic device 10 may include any other suitable components depending on the specific application.

[0247] Exemplary computer program products and computer-readable storage media

[0248] In addition to the methods and apparatus described above, embodiments of this disclosure may also be computer program products comprising computer program instructions that, when executed by a processor, cause the processor to perform the steps of the methods according to various embodiments of this disclosure as described in the "Exemplary Methods" section above.

[0249] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this disclosure. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on a user's computing device, partially on a user's computing device, as a standalone software package, partially on a user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0250] Furthermore, embodiments of this disclosure may also be computer-readable storage media having computer program instructions stored thereon, which, when executed by a processor, cause the processor to perform the steps in the methods according to various embodiments of this disclosure described in the "Exemplary Methods" section above.

[0251] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0252] The basic principles of this disclosure have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0253] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For system embodiments, since they largely correspond to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0254] The block diagrams of devices, apparatuses, devices, and systems disclosed herein are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0255] The methods and apparatus of this disclosure may be implemented in many ways. For example, they may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above-described order of steps for the methods is for illustrative purposes only, and the steps of the methods of this disclosure are not limited to the order specifically described above unless otherwise specifically stated. Furthermore, in some embodiments, this disclosure may also be implemented as a program recorded on a recording medium, the program including machine-readable instructions for implementing the methods according to this disclosure. Thus, this disclosure also covers recording media storing programs for performing the methods according to this disclosure.

[0256] It should also be noted that in the apparatus, devices, and methods of this disclosure, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions to this disclosure.

[0257] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of this disclosure. Therefore, this disclosure is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.

[0258] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this disclosure to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations therein.

Claims

1. A method for splicing lane lines, comprising: Determine the first lane line curve and the second lane line curve to be spliced ​​in the first coordinate system; the first lane line curve and the second lane line curve are different lane line segments of the same lane line corresponding to two different viewpoints. Based on the first lane line curve and the second lane line curve, determine the splicing constraint point; Based on the splicing constraint points, the first curve parameters corresponding to the first lane line curve are adjusted to obtain the adjusted first target curve parameters corresponding to the first curve parameters. Based on the splicing constraint points, the second curve parameters corresponding to the second lane line curve are adjusted to obtain the adjusted second target curve parameters corresponding to the second curve parameters; the splicing constraint points are used to constrain the adjustment process when adjusting the first lane line curve and the second lane line curve, so that the two lane line curves after adjustment can be continuously and smoothly spliced ​​at the splicing constraint points. Based on the first target curve parameters and the second target curve parameters, the spliced ​​lane lines are determined.

2. The method according to claim 1, wherein, The step of adjusting the first curve parameters corresponding to the first lane line curve based on the splicing constraint points to obtain the adjusted first target curve parameters corresponding to the first curve parameters includes: Based on the splicing constraint points, the first curve parameters are adjusted using a constrained extended Kalman filter algorithm to obtain the first target curve parameters.

3. The method of claim 2, wherein, The step of adjusting the first curve parameters based on the splicing constraint points to obtain the first target curve parameters includes: The initial state quantities are determined based on the parameters of the first curve. Based on the splicing constraint point, the first curve parameter, and the second curve parameter, a constraint quantity is determined, wherein the constraint quantity includes the horizontal coordinate value of the splicing constraint point and the first-order partial derivative constraint value at the vertical coordinate value of the splicing constraint point; Determine the initial transition matrix from the initial state quantity to the constraint quantity; Based on the initial transition matrix, determine the Kalman gain; The target state variable is determined based on the Kalman gain, the constraint, the initial transition matrix, and the initial state variable. Based on the target state quantity, the parameters of the first target curve are determined.

4. The method according to claim 3, wherein, The determination of the target state variable based on the Kalman gain, the constraint, the initial transition matrix, and the initial state variable includes: Based on the Kalman gain, the constraint quantity, and the initial transition matrix, the initial state quantity is iteratively updated until a preset iteration termination condition is met, thereby obtaining the target state quantity.

5. The method of claim 4, wherein, The step of iteratively updating the initial state variable based on the Kalman gain, the constraint, and the initial transition matrix until a preset iteration termination condition is met to obtain the target state variable includes: During the iteration process, taking any iteration as the current iteration, perform the following steps: Based on the new state variables obtained in the previous iteration, determine the current curve parameters corresponding to the first curve parameters; Based on the current curve parameters, determine the current state quantity; Determine the current transition matrix from the current state quantity to the constraint quantity; Based on the current transition matrix, the new covariance matrix obtained in the previous iteration, and the preset constraint noise matrix, the current Kalman gain is determined; Based on the current Kalman gain, the constraint quantity, and the current transition matrix, the current state quantity is updated to obtain the new state quantity for the current iteration; Based on the current Kalman gain and the current transition matrix, the new covariance matrix obtained in the previous iteration is updated to obtain the new covariance matrix for the current iteration. In response to the current iteration reaching a preset number of iterations, and / or the new covariance matrix of the current iteration satisfying a preset condition, the iteration process ends, and the new state variable of the current iteration is taken as the target state variable.

6. The method of claim 3, wherein, Determining the Kalman gain based on the initial transition matrix includes: Based on the initial transition matrix, the preset initial covariance matrix, and the preset constraint noise matrix, the Kalman gain is determined. The preset initial covariance matrix is ​​a diagonal matrix, and the preset constraint noise matrix is ​​a diagonal matrix used to represent the confidence level of the constraint.

7. The method according to claim 3, wherein, The step of determining the constraint amount based on the splicing constraint point, the first curve parameter, and the second curve parameter includes: Based on the first curve parameters, determine the first first-order partial derivative of the first lane line curve at the longitudinal coordinate value of the splicing constraint point. Based on the second curve parameters, determine the second first-order partial derivative of the second lane line curve at the longitudinal coordinate value of the splicing constraint point; The constraint value of the first-order partial derivative is determined based on the first-order partial derivative and the second-order partial derivative. The constraint quantity is determined based on the lateral coordinates of the splicing constraint points and the first-order partial derivative constraint values.

8. The method of claim 1, wherein, The step of adjusting the second curve parameters corresponding to the second lane line curve based on the splicing constraint points to obtain the adjusted second target curve parameters includes: Based on the splicing constraint points, the second curve parameters are adjusted using a constrained extended Kalman filter algorithm to obtain the second target curve parameters.

9. The method of claim 1, wherein, The step of determining the splicing constraint point based on the first lane line curve and the second lane line curve includes: Based on the first lane line curve and the second lane line curve, determine the longitudinal coordinate value of the splicing constraint point; Based on the longitudinal coordinate value, determine the first lateral coordinate value of the first lane line curve at the longitudinal coordinate value; Based on the longitudinal coordinate value, determine the second lateral coordinate value of the second lane line curve at the longitudinal coordinate value; Based on the first horizontal coordinate value and the second horizontal coordinate value, determine the horizontal coordinate value of the splicing constraint point; The splicing constraint points are determined based on the horizontal and vertical coordinate values ​​of the splicing constraint points.

10. The method of claim 9, wherein, Determining the longitudinal coordinate value of the splicing constraint point based on the first lane line curve and the second lane line curve includes: Determine the ordinates of the first and second endpoints of the first lane line curve, as well as the ordinates of the third and fourth endpoints of the second lane line curve; Based on the first endpoint ordinate, the second endpoint ordinate, the third endpoint ordinate, and the fourth endpoint ordinate, determine the first target endpoint ordinate and the second target endpoint ordinate adjacent to the first lane line curve and the second lane line curve; Based on the ordinates of the first and second target endpoints, the longitudinal coordinates of the splicing constraint points are determined.

11. The method of any one of claims 1-10, wherein, Determining the first lane line curve and the second lane line curve to be spliced ​​in the first coordinate system includes: Acquire the first image data corresponding to at least two viewpoints; Based on each of the first image data, determine the lane line region in the image coordinate system corresponding to each of the first image data; Transform the lane line regions corresponding to each of the first image data into the first coordinate system to obtain the lane line coordinate points in the first coordinate system corresponding to each of the first image data. Based on the lane line coordinate points in the first coordinate system corresponding to each of the first image data, the lane line curves in the first coordinate system corresponding to each of the first image data are determined. Based on the lane line curves in the first coordinate system corresponding to each of the first image data, the first lane line curves and the second lane line curves to be stitched together in the first coordinate system are determined.

12. A lane marking splicing device, comprising: The first determining module is used to determine the first lane line curve and the second lane line curve to be spliced ​​in the first coordinate system; the first lane line curve and the second lane line curve are different lane line segments of the same lane line corresponding to two different viewpoints. The first processing module is used to determine the splicing constraint point based on the first lane line curve and the second lane line curve; The second processing module is used to adjust the first curve parameters corresponding to the first lane line curve based on the splicing constraint points, so as to obtain the adjusted first target curve parameters corresponding to the first curve parameters. The third processing module is used to adjust the second curve parameters corresponding to the second lane line curve based on the splicing constraint points to obtain the adjusted second target curve parameters corresponding to the second curve parameters; the splicing constraint points are used to constrain the adjustment process when adjusting the first lane line curve and the second lane line curve, so that the two lane line curves after adjustment can be continuously and smoothly spliced ​​at the splicing constraint points. The fourth processing module is used to determine the spliced ​​lane lines based on the first target curve parameters and the second target curve parameters.

13. A computer-readable storage medium storing a computer program for performing the lane line splicing method according to any one of claims 1-11.

14. An electronic device, the electronic device comprising: processor; Memory used to store the processor's executable instructions; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the lane line splicing method according to any one of claims 1-11.

Images