Lane line fusion method, lane line fusion device, vehicle and storage medium

Abstract

A lane line fusion method, device, vehicle and storage medium, the lane line fusion method comprising: acquiring an environmental image around a movable platform, and obtaining an initial lane line set of the movable platform according to the environmental image (SllO); performing fitting optimization on initial lane line data in the initial lane line set to obtain a target lane line set; wherein the fitting optimization comprises: performing fitting optimization on the initial lane line data and historical lane line data in the historical lane line set to obtain a target lane line set (S120); the target lane line set comprises lane lines that are not parallel to each other.

Description

Technical Field

[0001] The present application relates to the field of vehicle assisted driving and autonomous driving, and in particular to a lane line fusion method, a lane line fusion device, a vehicle and a storage medium. Background Art

[0002] Lane detection is one of the main functions that need to be realized by assisted driving (ADAS) and autonomous driving technology. Based on the detected lane lines, the driving plan of mobile platforms, such as vehicles, can be planned to ensure safe and reliable driving. The existing lane detection method is mainly based on the environmental image around the current vehicle. It is unable to optimize the current lane line by integrating the historical lane lines detected at previous moments, and cannot effectively cope with driving scenarios with forked intersections. Therefore, the existing lane detection has poor versatility and low accuracy. Summary of the invention

[0003] Based on this, this specification provides a lane fusion method, a lane fusion device, a vehicle and a storage medium, aiming to solve technical problems such as the existing lane line detection cannot cope well with driving scenarios with forked intersections, has poor versatility and is not accurate enough.

[0004] In a first aspect, this specification provides a lane line detection method, including:

[0005] Acquire an environment image around the movable platform, and obtain an initial lane line set of the movable platform according to the environment image;

[0006] Performing fitting optimization on the initial lane line data in the initial lane line set to obtain a target lane line set;

[0007] Wherein, the fitting optimization includes: performing fitting optimization based on the initial lane line data and the historical lane line data in the historical lane line set to obtain a target lane line set, and the fitting optimization is used to make the target lane line set include lane lines that are not parallel to each other.

[0008] In a second aspect, this specification provides a lane detection device, including a sensor and a processor;

[0009] The sensor is used to obtain an image of the environment around the movable platform;

[0010] The processor is used to implement the following steps:

[0011] Acquire an environment image around the movable platform, and obtain an initial lane line set of the movable platform according to the environment image;

[0012] Performing fitting optimization on the initial lane line data in the initial lane line set to obtain a target lane line set;

[0013] Wherein, the fitting optimization includes: performing fitting optimization based on the initial lane line data and the historical lane line data in the historical lane line set to obtain a target lane line set, and the fitting optimization is used to make the target lane line set include lane lines that are not parallel to each other.

[0014] In a third aspect, the present specification provides a vehicle, comprising:

[0015] The lane line detection device mentioned above is used to determine the lane line;

[0016] Sports components, used for driving.

[0017] In a fourth aspect, the present specification provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the above-mentioned lane fusion method.

[0018] The embodiments of this specification provide a lane line fusion method, a lane line fusion device, a vehicle and a storage medium, which determine a number of initial lane line data through the environment image around the movable platform, and then perform fitting optimization based on the number of initial lane line data and the corresponding historical lane line data in the historical lane line set to obtain a number of target lane lines. Since the fitting optimization integrates the historical detection data of the lane line, the lane line detection can be independent of the lane line geometry parallel assumption and the road surface parallel assumption, so that it can be applied to the scene of the road including the fork intersection, and improves the versatility and detection accuracy.

[0019] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure of this specification. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of this specification, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0021] Figure 1 It is a flowchart of a lane line detection method provided in an embodiment of this specification;

[0022] Figure 2 is a schematic diagram of a lane detection scenario in an implementation manner;

[0023] Figure 3 It is a schematic diagram of determining the local lane lines in the local map through connected domain analysis;

[0024] Figure 4 yes Figure 1 Schematic diagram of the sub-process of obtaining the target lane line set through fitting optimization;

[0025] Figure 5 yes Figure 4 Schematic diagram of the sub-process of obtaining the initial lane line data corresponding to the target lane line by mid-fitting optimization;

[0026] Figure 6 is a schematic block diagram of a remote control device provided in an embodiment of this specification;

[0027] Figure 7 It is a schematic block diagram of a movable platform provided in one embodiment of this specification. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of this specification to clearly and completely describe the technical solutions in the embodiments of this specification. Obviously, the described embodiments are part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this specification.

[0029] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0030] In conjunction with the accompanying drawings, some implementation methods of this specification are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0031] See also Figure 1 , Figure 1 It is a flowchart of a lane line detection method provided in an embodiment of this specification.

[0032] The lane line detection method can be applied to a lane line detection device, which can be mounted on a movable platform and has a data connection with the movable platform. The movable platform can be some mobile devices that can travel on public transportation roads, such as an autonomous driving vehicle. The lane line detection device can also be an auxiliary driving device that has a data connection with the movable platform.

[0033] The lane line detection device may be built into the movable platform, such as a system integrated into the movable platform, or may be externally connected to the movable platform, such as an auxiliary driving device connected to the outside of the movable platform.

[0034] In some embodiments, the lane line detection method can be applied to Figure 2 In the lane line detection scenario shown, the movable platform 10 may be a vehicle traveling on a public transportation road, the lane line detection device 100 is built in the movable platform 10, and one or more sensors 101 are also installed on the movable platform 10, and the lane line detection device 100 can obtain an environmental image of the movable platform 10 through the sensor 101. For example, the sensor 101 includes a visual sensor for obtaining an environmental image around the movable platform.

[0035] Among them, the sensor 101 can be installed in the front, rear and / or roof of the movable platform 10. One or more sensors 101 installed in the movable platform 10 can be installed in the same position or in different positions, which is not limited in the embodiment of the present invention.

[0036] like Figure 1 As shown, the lane line detection method of this embodiment includes steps S110 to S120.

[0037] S110: Acquire an environment image around the movable platform, and obtain an initial lane line set of the movable platform according to the environment image.

[0038] Exemplarily, the sensor carried by the movable platform collects environmental images of a preset range around the movable platform in real time or at a preset period, and constructs a local map of the environment in which the movable platform is located based on the environmental images.

[0039] Exemplarily, the image in front of the movable platform, or the front view, is acquired by the sensor carried by the movable platform, and the lane line image points in the front view are identified based on the image model. The lane line image points can also be converted to a local map in the world coordinate system in combination with the pose information and 3D environment information of the movable platform to obtain a local map of the environment in which the movable platform is located, and the lane line position point corresponding to the lane line image point in the local map is determined.

[0040] For example, a convolutional neural network (CNN) may be used to perform preliminary lane line detection on the front view to obtain lane line image points.

[0041] For example, the position and posture information of the movable platform, such as the current position and posture information of the vehicle body, can be acquired through a visual inertial navigation system (such as Visual-Inertial Navigation System, VINS).

[0042] Exemplarily, the 3D environment perception information around the movable platform, that is, the 3D environment information, can be determined by using the binocular stereo matching algorithm SGBM.

[0043] In some implementations, after determining the lane line position points in the local map, an initial lane line set may be determined in the local map, where the initial lane line set includes a plurality of initial lane line data.

[0044] Exemplarily, a connected domain analysis is performed based on lane line position points in a local map to obtain a local lane line set corresponding to the local map.

[0045] Exemplarily, the initial lane line data included in the initial lane line set may be determined based on lane line position points recorded in a local map. In some embodiments, the local map may be a grid map in a world coordinate system, and the grid corresponding to the lane line position point is a lane line grid, and each lane line grid includes a semantic information, and the semantic information is used to characterize that the lane line grid is a grid corresponding to the lane line position point. In view of this situation, each lane line grid in the local map may be analyzed and processed to obtain a local lane line set corresponding to the local map, and the local lane line set includes at least one local lane line, and then an initial lane line set is determined from the local lane line set, and the initial lane line set includes at least one initial lane line data.

[0046] In some implementations, connected domain analysis can be performed on each lane line grid in the local map based on the semantic information of each lane line grid to obtain a local lane line set corresponding to the local map, and based on a preset optimization algorithm and the weight value of each local lane line in the local lane line set, an initial lane line is optimized and determined to obtain an initial lane line set. The weight value of the local lane line is determined based on the feature information of the lane line.

[0047] Exemplarily, when performing connected domain analysis processing on each lane line grid in a local map based on the semantic information of each lane line grid, each lane line grid can be first determined from the local map according to the semantic information of each lane line grid, and connected domain extraction can be performed on each lane line grid. The connected domain labels of the lane line grids in a small range can be determined according to the semantic information of each lane line grid. Lane line fitting can be performed based on the connected domain labels corresponding to each lane line grid to determine one or more local lane lines corresponding to the lane line position points in the small range. By analogy, all local lane lines in the local map can be determined, thereby obtaining a set of local lane lines corresponding to the local map.

[0048] The lane line grids belonging to the same lane line correspond to the same connected domain label, and the connected domain label is associated with the semantic information of the lane line grid. For example, the semantic information of the lane line grid G1 is used to indicate that the image point corresponding to the lane line grid G1 belongs to the lane line A, and the semantic information of the lane line grid G2 is used to indicate that the image point corresponding to the lane line grid G2 also belongs to the lane line A, then the lane line grid G1 and the lane line grid G2 correspond to the same connected domain label.

[0049] like Figure 3 The figure shows a schematic diagram of determining the local lane line in the local map through the connected domain analysis. As shown in the figure, each grid represents an image point, and the numbers in the grid represent the connected domain labels corresponding to each grid. Among them, the grid with a connected domain label of 0 may be a grid that does not belong to the lane line in the local lane line, and the grid with a connected domain label that is not 0 is a lane line grid that may belong to the lane line. The grid with a connected domain label of 1 is the image point belonging to the A lane line in the local map; the image point with a connected domain label of 2 is the image point corresponding to the B lane line in the local map, where the A lane line and the B lane line are two different lane lines.

[0050] Exemplarily, when performing connected domain analysis on lane line grids in a local map, a small range of grids can be selected from the local map according to an image detection window of a preset size, and the connected domain labels of each grid in the image detection window can be determined based on the semantic information of each grid, wherein the connected domain label corresponding to the non-lane line grid is 0, and the connected domain label of the lane line grid is determined by the corresponding semantic information. Since grids in the same connected domain have the same connected domain label, the connected domains of the image detection window can be determined based on the characteristics of whether the connected domain labels are the same, that is, the grids included in each connected domain can be determined, wherein the preset size of the image detection window can be, for example, 3×3 or 5×5.

[0051] After determining the grids included in each connected domain, the grids corresponding to each connected domain can be fitted according to the optimal solution algorithm to obtain the local lane lines corresponding to each connected domain. Similarly, the local lane lines corresponding to the lane lines included when the image detection window is located at different positions in the local map can be determined, thereby determining all the local lane lines in the local map to obtain a local lane line set.

[0052] For example, if the connected domain label corresponding to the grid located in the image detection window selected from the local lane line is Figure 3As shown, all lane line grids with a connected domain label of 1 can be fitted into a local lane line according to the optimal solution algorithm, as shown in 501 in the figure, and all lane line grids with a connected domain label of 2 can be fitted into a local lane line according to the optimal solution algorithm, as shown in 502 in the figure. By analogy, all lane line grids in the image detection window can be fitted into multiple local lane lines, so that all local lane lines in the local map can be determined.

[0053] Exemplarily, after determining the local lane line set corresponding to the local map, at least one initial lane line can be optimized and determined based on the weighted maximum clique algorithm and the weight values ​​of each local lane line in the local lane line set, and the at least one initial lane line can be filtered to obtain the initial lane line set.

[0054] Exemplarily, feature information of each local lane line in the local lane line set is analyzed to obtain a weight value of each local lane line, and then based on a preset optimization algorithm, such as a weighted maximum clique algorithm and the weight value of each local lane line, at least one initial lane line is optimized and determined to obtain an initial lane line set.

[0055] Among them, the weight value of the local lane line is determined based on the characteristic information of the lane line. In one embodiment, the characteristic information of the lane line includes: geometric features and / or color features of the lane line, and the geometric features include any one or more of length features, width features, and parallel features between lane lines.

[0056] As a feasible implementation method, the correspondence between the geometric features, color features and assumed scores of the lane line can be established in advance. The higher the sum of the assumed scores corresponding to a lane line, the greater the possibility that the lane line is the actual lane line. Exemplarily, the correspondence between the geometric features, color features and assumed scores of the lane line can be shown in Table 1. It can be seen that each local lane line can determine the sum of the assumed scores corresponding to itself through the correspondence shown in Table 1, and the sum of the assumed scores is the weight value corresponding to the local lane line.

[0057] Table 1. Lane feature information scores

[0058]

[0059] Among them, the hypothetical scores corresponding to the various feature dimensions in Table 1 are only examples, which are mainly used to illustrate the principle of determining the weight value of a local lane line through the feature information of the lane line, and cannot be a limitation of the embodiments of the present invention.

[0060] In one embodiment, the feature information of each local lane line can be analyzed, and the weight value of each local lane line can be determined based on the pre-established correspondence between the geometric features, color features and assumed scores of the lane lines. Furthermore, based on the weighted maximum clique algorithm, at least one local lane line with the highest weight value among the local lane lines is solved, that is, at least one initial lane line.

[0061] Furthermore, at least one initial lane line may be filtered to obtain an initial lane line set. In one embodiment, post-processing may be performed based on preset prior information for the initial lane line to filter all initial lane lines of the at least one initial lane line, filter out erroneous lane lines from all the initial lane lines, and obtain an initial lane line set.

[0062] The preset prior information for the initial lane line is a preset range or preset value set based on the national standard for lane lines, and the prior information specifically includes length information and width information, etc. Assuming that the length of the lane line in the national standard is 1.5 meters, the length information included in the prior information for the initial lane line can be set to a length that is less than the length of the standard lane line, such as a range of 10 cm to 15 cm, etc.; assuming that the width of the lane line in the national standard is 15 cm, the width information included in the prior information can be set to a range of 13 cm to 17 cm, etc. Correspondingly, the wrong lane line is a lane line that does not meet the preset prior information.

[0063] The initial lane line selected from the local lane line set can be considered as the lane line that best matches the actual lane line.

[0064] S120. Perform fitting optimization on the initial lane line data in the initial lane line set to obtain a target lane line set, wherein the fitting optimization includes: performing fitting optimization based on the initial lane line data and the historical lane line data in the historical lane line set to obtain a target lane line set.

[0065] Specifically, after determining the initial lane line set from the local map, the historical lane line data in the historical lane line set and the initial lane line data in the initial lane line set can be fitted and optimized to obtain the target lane line set. The current lane line detection result can be integrated with the historical lane line detection result in time and space, which is conducive to improving the accuracy of the lane line detection result.

[0066] Exemplarily, the filtered initial lane line data in the initial lane line set and the historical lane line data in the historical lane line set are fit-optimized to obtain a target lane line set to complete the detection of the lane lines.

[0067] In some implementations, after an initial lane line set is determined from a local map, a query may be made as to whether a historical lane line set exists in a storage area.

[0068] For example, if the historical lane line set does not exist, it can be determined that this lane line detection is the first lane line detection, and the historical lane line set is determined based on the initial lane line set obtained this time, and the determined historical lane line set is stored in the storage area for subsequent fitting optimization with the new initial lane line set. If the historical lane line set exists, the historical lane line data in the historical lane line set and the initial lane line data in the initial lane line set can be fitted and optimized to obtain the target lane line set.

[0069] In some implementations, the fitting optimization is used to make the target lane line set include lane lines that are not parallel to each other.

[0070] By fitting and optimizing the initial lane line data and the historical lane line data in the historical lane line set, the fitting optimization can be independent of the lane line geometry parallel assumption and the road surface parallel assumption, and can be applied to the scene where the road includes a fork intersection, such as Figure 2 As shown, the road includes lane lines that are not parallel to each other, which improves the versatility of the lane line detection method.

[0071] In some embodiments, the fitting optimization is performed based on the initial lane line data and the historical lane line data in the historical lane line set to obtain the target lane line set, including: based on overfitting constraints and parallel constraints, the fitting optimization is performed based on the initial lane line data and the historical lane line data in the historical lane line set to obtain the target lane line set.

[0072] Exemplarily, initial lane line data to be fitted is determined from an initial lane line set, and historical lane lines matching the initial lane line data are determined from a historical lane line set, and the initial lane line data to be fitted is any one of the initial lane line set.

[0073] Exemplarily, by matching the index information of the initial lane line data to be fitted with the index information of the historical lane line data in the historical lane line set, a target historical lane line that matches the initial lane line data is determined.

[0074] For example, the lane to which each initial lane line belongs can be determined based on the semantic information of the lane line grid constituting the initial lane line, and then index information is added to each initial lane line data, each index information indicating the lane to which the initial lane line data belongs.

[0075] After determining the historical lane line that matches the initial lane line to be fitted from the historical lane line set, a fitting optimization is performed based on the initial lane line data and the historical lane line data based on overfitting constraints and parallel constraints to obtain a target lane line corresponding to the initial lane line data.

[0076] The initial lane line data in the initial lane line set are sequentially determined as the initial lane line to be fitted, and the target lane line corresponding to each initial lane line data is determined; a target lane line set is obtained according to at least one target lane line; due to overfitting constraints and parallel constraints, the target lane line set includes lane lines that are not parallel to each other.

[0077] In some embodiments, the initial lane line data and the historical lane line data are fit-optimized using a lane model that includes an overfitting constraint and a parallel constraint.

[0078] Exemplarily, the mathematical function corresponding to the lane model can be composed of three parts. The first part is a basic least squares model, which is used to solve the minimum error between the lane line position point corresponding to the initial lane line data and the curve equation corresponding to the target lane line obtained by fitting optimization; the second part is the smoothness term of the curve (corresponding to the overfitting preset condition), and the integration of the square of the third-order derivative of the curve equation can be used as a smoothness constraint, which is mainly used to prevent overfitting; the third part is a weak parallel constraint term (corresponding to the parallel constraint condition), which is mainly used to constrain the parallel state between two adjacent lane lines.

[0079] For example, f i (t) represents the curve equation of the target lane line obtained by fitting and optimizing the initial lane line data of the i-th initial lane line.

[0080] Exemplarily, the curve equation of the target lane line is described by a quintic polynomial, specifically:

[0081] f i (t) = T i C i , where T i =(1,t,t 2 ,…,t 5 ) and C i =(c0,c1,c2,…,c5) T

[0082] Where, T i represents the polynomial basis vector, t represents the first coordinate of the lane line position point on the local map, C i represents the fitting coefficient data, which is the quantity to be solved, and the superscript T represents the transpose of the matrix; thus, the curve equation corresponding to the target lane line obtained by fitting optimization is expressed as follows:

[0083] f i (t) = T i C i =c0+c1×t+c2×t 2 +c3×t 3 +c4×t 4 +c5×t 5

[0084] The first part of the lane model is used to solve the minimum error between the lane line position points corresponding to the initial lane line data and the curve equation corresponding to the target lane line obtained by fitting optimization.

[0085] The second part of the lane model is the smoothness term of the curve (corresponding to the overfitting preset condition), and the integration of the square of the third-order derivative of the curve equation can be used as a smoothness constraint.

[0086] The third part of the lane model is the weak parallel constraint (corresponding to the parallel constraint condition), which is mainly used to constrain the parallel state between two adjacent lane lines. For example, it is used to constrain the parallel state between the i-th initial lane line and the i+1-th initial lane line.

[0087] For example, the first part, the second part, and the third part of the lane model may be processed according to the fifth-order polynomial description form of the target lane curve equation, and the processing results and the fitting coefficient data C i The irrelevant part is expressed as a constant, so that the fitting coefficient data C can be obtained. i The lane model in matrix form, such as the objective function in matrix form.

[0088] In some implementations, the objective function of the fitting optimization is expressed as obtaining the fitting coefficient data of the curve equations of two adjacent target lane lines so as to minimize the sum of the first part, the second part, and the third part of the lane model.

[0089] Exemplarily, by taking the derivative of the objective function in matrix form and setting the derivative to 0, an equation for the fitting coefficient data of the curve equations of two adjacent target lane lines can be obtained.

[0090] According to the equation, the fitting coefficient data of the curve equation of two adjacent target lane lines can be obtained. According to the obtained fitting coefficient data, the optimal curve equation of two adjacent target lane lines can be obtained, for example, the curve equation of the target lane line obtained by fitting the i-th initial lane line and the i+1-th initial lane line can be obtained.

[0091] For example, by calculating the polynomial basis vectors T on both sides of the equation i and the fitting coefficient data C iThe cumulative part of the cumulative sum can be used to retain historical observation information.

[0092] For example, regarding the polynomial basis vector T i The cumulative sum is the first aggregate data A i , about the polynomial basis vector T i and the fitting coefficient data C i The cumulative sum is the second aggregate data B i .

[0093] The first aggregate data A is obtained by accumulating the time series of historical observations i and the second aggregate data B i . First aggregate data A i The second aggregated data contains more historical observations, which can prevent the loss of valid observations caused by, for example, sliding windows, which may reduce the robustness of lane detection.

[0094] At the same time, although the first aggregate data A i and the second aggregate data B i It is obtained by the time series accumulation of observations, but the amount of data is only determined by the order of the lane curve equation and the number of segments of the curve.

[0095] Exemplarily, the historical lane line data is obtained by accumulating historical initial lane line data according to a preset fusion rule.

[0096] It can be understood that the historical lane line data has the same data volume.

[0097] In some embodiments, the polynomial basis vector and fitting coefficient data at the current time, such as the n+1th time, and the first aggregate data A at the nth time are determined. i and the second aggregate data B i The fitting coefficient of the curve equation of the target lane line that is optimized for fitting at the current moment can be solved.

[0098] In some embodiments, the increment of the cumulative sum is determined according to the polynomial basis vector and fitting coefficient data at the current moment, such as the n+1th moment, and the newly determined increment is correspondingly accumulated to the first aggregate data A. i , second aggregate data B i , the initial lane line data is compressed into the historical lane line data to obtain updated historical lane line data.

[0099] It can be understood that the initial lane line data and the historical lane line data are processed through a preset data structure. For example, the processing of the initial lane line data and the historical lane line data is achieved by accumulating the increments determined by the polynomial basis vectors and the fitting coefficient data. Among them, the preset data structure has a certain amount of calculation. For example, the accumulation of the increments at each moment and the historical first aggregated data and the second aggregated data has the same amount of calculation and will not increase with the accumulation of historical observation data, and the amount of calculation for solving the curve equation coefficients based on the first aggregated data and the second aggregated data will not increase with the accumulation of historical observation data.

[0100] In some embodiments, Figure 4 As shown, the fitting optimization is performed based on the initial lane line data and the historical lane line data in the historical lane line set to obtain the target lane line set, including steps S121 to S123.

[0101] S121. Determine historical lane line data that matches the initial lane line data from the historical lane line set.

[0102] Exemplarily, by matching the index information of the initial lane line data with the index information of the historical lane line data in the historical lane line set, the historical lane line data matching the initial lane line data is determined.

[0103] Exemplarily, if certain initial lane line data corresponds to the i-th lane line, then the historical lane line data corresponding to the i-th lane line in the historical lane line set matches the initial lane line data.

[0104] S122. Based on an overfitting constraint and a parallel constraint, the initial lane line data and the historical lane line data are fit-optimized to obtain a target lane line corresponding to the initial lane line data.

[0105] Exemplarily, the historical lane line data corresponding to the i-th lane line includes the first aggregated data A i and the second aggregate data B i .

[0106] Specifically, Figure 5 As shown, based on the overfitting constraint condition and the parallel constraint condition, the initial lane line data and the historical lane line data are fitted and optimized to obtain the target lane line corresponding to the initial lane line data, including steps S1221 to S1222.

[0107] S1221. Determine first incremental data and second incremental data according to the initial lane line data.

[0108] Exemplarily, the initial lane line data at the n+1th moment includes a number of lane line position points, so that the first incremental data for superimposing on the first aggregate data and the second incremental data for superimposing on the second aggregate data can be determined.

[0109] Exemplarily, the lane line position point can be represented on the map by a first coordinate value and a second coordinate value. For example, the first coordinate axis corresponding to the first coordinate value is a coordinate axis parallel to the left and right direction of the vehicle, and the second coordinate axis corresponding to the second coordinate value is a coordinate axis parallel to the direction of vehicle travel.

[0110] In some embodiments, determining the first incremental data based on the initial lane line data includes: determining the first incremental data based on a first coordinate value of the initial lane line on a local map, wherein the local map is determined based on the environment in which the movable platform is located.

[0111] In some embodiments, determining the second incremental data based on the initial lane line data includes: determining the second incremental data based on a first coordinate value of the initial lane line on the local map and a second coordinate value of the initial lane line on the local map.

[0112] S1222. Based on an equation determined according to an overfitting constraint and a parallel constraint, determine a target lane line corresponding to the initial lane line data according to the first incremental data, the second incremental data, the first aggregated data, and the second aggregated data.

[0113] In some embodiments, determining the target lane line corresponding to the initial lane line data based on the first incremental data, the second incremental data, the first aggregated data, and the second aggregated data includes: determining the polynomial coefficients corresponding to the initial lane line data based on the first incremental data, the second incremental data, the first aggregated data, and the second aggregated data; determining the target lane line corresponding to the initial lane line data based on the polynomial coefficients.

[0114] Specifically, the curve equation corresponding to the i-th lane line obtained by fitting is expressed as follows:

[0115] f i (t) = T i C i =c0+c1×t+c2×t 2 +c3×t 3 +c4×t 4 +c5×t 5

[0116] S123. Form a target lane line set by at least one of the target lane lines.

[0117] For example, the target lane line set is formed according to the two closest target lane lines on both sides of the vehicle, or the target lane line set is formed according to the closest target lane line on the left side of the vehicle, or the target lane line set is formed according to the closest target lane line on the right side of the vehicle, or all the target lane lines fitted within the sensor field of view are formed into a target lane line set. Accordingly, the corresponding number of lane lines in the target lane line set can be fitted and optimized later.

[0118] Since the lane model adds a parallel state constraint between two adjacent lane lines when fitting the initial lane line and the target historical lane line, it can better ensure the accuracy of lane line detection for complex lane detection scenarios, such as bifurcated road scenarios and urban scenarios, so that it can be better applied to a variety of lane line scenarios and improve versatility.

[0119] Exemplarily, after obtaining the target lane line set, the target lane lines in the target lane line set may be combined to obtain at least one lane, and a lane centerline of the lane may be generated to assist the movable platform in driving. For example, the target lane lines may be combined according to attribute information, such as geometric features and / or color features of the lane, and the geometric features include any one or more of length features, width features, and parallel features between lanes.

[0120] In some implementations, the lane line detection method further includes: updating the historical lane line data according to the initial lane line data, wherein the updated historical lane line data has the same data volume as the historical lane line data before the update.

[0121] Exemplarily, the updating of the historical lane line data according to the initial lane line data includes: accumulating the first incremental data to the first aggregated data, and accumulating the second incremental data to the second aggregated data. The initial lane line data is compressed to the historical lane line data to obtain updated historical lane line data. The updated historical lane line data has the same data volume as the historical lane line data before the update, so that the computational complexity is not increased due to the increase in the accumulated observations, thereby achieving the purpose of constraining the computational complexity of the fitting optimization.

[0122] The lane line detection method provided in the embodiment of this specification determines a number of initial lane line data through the environment image around the movable platform, and then performs fitting optimization based on the number of initial lane line data and the corresponding historical lane line data in the historical lane line set to obtain a number of target lane lines. Since the fitting optimization integrates the historical detection data of the lane line, the lane line detection does not rely on the lane line geometry parallel assumption and the road surface parallel assumption, so it can be applied to the scene of the road including the fork intersection, improving the versatility and detection accuracy.

[0123] Please refer to the above examples. Figure 6 , Figure 6 1 is a schematic block diagram of a lane line detection device 100 provided in an embodiment of the present specification. The lane line detection device 100 includes a sensor 101 and a processor 102. The sensor 101 may include a visual sensor, for example, for acquiring an environment image around a movable platform.

[0124] Specifically, the processor 102 may be a micro-controller unit (MCU), a central processing unit (CPU), or a digital signal processor (DSP).

[0125] Exemplarily, the lane line detection device 100 also includes a memory 103, which may include, for example, a Flash chip, a read-only memory (ROM) disk, an optical disk, a U disk, or a mobile hard disk.

[0126] The processor 102 is used to run a computer program stored in the memory 103, and implement the aforementioned lane line detection method when executing the computer program.

[0127] Exemplarily, the processor is used to run a computer program stored in the memory 103, and implements the following steps when executing the computer program:

[0128] Acquire an environment image around the movable platform, and obtain an initial lane line set of the movable platform according to the environment image;

[0129] Performing fitting optimization on the initial lane line data in the initial lane line set to obtain a target lane line set;

[0130] Wherein, the fitting optimization includes: performing fitting optimization based on the initial lane line data and the historical lane line data in the historical lane line set to obtain a target lane line set, and the fitting optimization is used to make the target lane line set include lane lines that are not parallel to each other.

[0131] Exemplarily, when the processor implements the fitting optimization based on the initial lane line data and the historical lane line data in the historical lane line set to obtain the target lane line set, the following is implemented:

[0132] Based on the overfitting constraint condition and the parallel constraint condition, fitting optimization is performed according to the initial lane line data and the historical lane line data in the historical lane line set to obtain a target lane line set.

[0133] Exemplarily, the initial lane line data and the historical lane line data are processed by a preset data structure, wherein the preset data structure has a certain amount of calculation.

[0134] Exemplarily, the historical lane line data is obtained by accumulating historical initial lane line data according to a preset fusion rule.

[0135] Exemplarily, when the processor implements the fitting optimization based on the initial lane line data and the historical lane line data in the historical lane line set to obtain the target lane line set, the following is implemented:

[0136] Determining historical lane line data matching the initial lane line data from the historical lane line set;

[0137] Based on an overfitting constraint condition and a parallel constraint condition, the initial lane line data and the historical lane line data are fitted and optimized to obtain a target lane line corresponding to the initial lane line data;

[0138] A target lane line set is composed of at least one of the target lane lines.

[0139] Exemplarily, the historical lane line data includes first aggregated data and second aggregated data;

[0140] The processor implements the fitting optimization of the initial lane line data and the historical lane line data based on the overfitting constraint condition and the parallel constraint condition to obtain the target lane line corresponding to the initial lane line data, thereby implementing:

[0141] Determine first incremental data and second incremental data according to the initial lane line data;

[0142] Based on an equation determined according to an overfitting constraint and a parallel constraint, a target lane line corresponding to the initial lane line data is determined according to the first incremental data, the second incremental data, the first aggregated data, and the second aggregated data.

[0143] Exemplarily, when the processor implements the step of determining the target lane line corresponding to the initial lane line data, the processor implements:

[0144] Determining polynomial coefficients corresponding to the initial lane line data;

[0145] A target lane line corresponding to the initial lane line data is determined according to the polynomial coefficients.

[0146] Exemplarily, the processor further implements:

[0147] The historical lane line data is updated according to the initial lane line data, and the updated historical lane line data has the same data volume as the historical lane line data before the update.

[0148] Exemplarily, when the processor implements the updating of the historical lane line data according to the initial lane line data, the following steps are implemented:

[0149] Adding the first incremental data to the first aggregated data;

[0150] The second incremental data is added to the second aggregated data.

[0151] Exemplarily, when the processor implements the determining of the first incremental data according to the initial lane line data, the processor implements:

[0152] The first incremental data is determined according to a first coordinate value of the initial lane line on a local map, and the local map is determined according to an environment in which the movable platform is located.

[0153] Exemplarily, when the processor implements the determining of the second incremental data according to the initial lane line data, the following is implemented:

[0154] The second incremental data is determined according to a first coordinate value of the initial lane line on the local map and a second coordinate value of the initial lane line on the local map.

[0155] The specific principles and implementation methods of the lane line detection device provided in the embodiments of this specification are similar to the lane line detection method of the aforementioned embodiments, and will not be repeated here.

[0156] Please refer to the above examples. Figure 7 , Figure 7 It is a schematic block diagram of a movable platform 200 provided in an embodiment of this specification.

[0157] like Figure 7 As shown, the movable platform 200 includes the aforementioned lane line detection device 100 for determining the lane line and further includes a motion component 210 for traveling.

[0158] In some embodiments, the movable platform 200 may be a vehicle, for example, a passenger vehicle, a cargo vehicle, an unmanned vehicle, etc., such as a manually driven vehicle or an autonomous driving vehicle.

[0159] The specific principles and implementation methods of the movable platform provided in the embodiments of this specification are similar to the lane line detection method in the aforementioned embodiments, and will not be repeated here.

[0160] A computer-readable storage medium is also provided in an embodiment of the present specification. The computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the processor implements the steps of the lane line detection method provided in the above embodiment.

[0161] The computer-readable storage medium may be an internal storage unit of the lane detection device described in any of the foregoing embodiments, such as a hard disk or memory of the lane detection device. The computer-readable storage medium may also be an external storage device of the lane detection device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (FlashCard), etc., equipped on the lane detection device.

[0162] The lane line fusion device, movable platform and storage medium provided in the above embodiments of this specification determine a number of initial lane line data through the environment image around the movable platform, and then perform fitting optimization based on the number of initial lane line data and the corresponding historical lane line data in the historical lane line set to obtain a number of target lane lines. Since the fitting optimization integrates the historical detection data of the lane line, the lane line detection does not rely on the lane line geometry parallel assumption and the road surface parallel assumption, so it can be applied to the scene of the road including the fork intersection, which improves the versatility and detection accuracy.

[0163] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the specification.

[0164] It will also be understood that the term "and / or" used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0165] The above is only a specific implementation of this specification, but the protection scope of this specification is not limited to this. Any technician familiar with this technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this specification, and these modifications or replacements should be included in the protection scope of this specification. Therefore, the protection scope of this specification should be based on the protection scope of the claims.

Claims

1. A lane line detection method, characterized in that: include: Acquire an environment image around the movable platform, and obtain an initial lane line set of the movable platform according to the environment image; Performing fitting optimization on the initial lane line data in the initial lane line set to obtain a target lane line set; The fitting optimization includes: performing fitting optimization according to the initial lane line data and the historical lane line data in the historical lane line set to obtain a target lane line set, wherein the fitting optimization is used to make the target lane line set include lane lines that are not parallel to each other; The step of performing fitting optimization based on the initial lane line data and the historical lane line data in the historical lane line set to obtain a target lane line set includes: Based on overfitting constraints and parallel constraints, fitting optimization is performed according to the initial lane line data and the historical lane line data in the historical lane line set to obtain a target lane line set. The parallel constraint is used to constrain the parallel state between two adjacent lane lines.

2. The method according to claim 1, characterized in that The initial lane line data and the historical lane line data are processed through a preset data structure, wherein the preset data structure has a certain amount of calculation.

3. The method according to claim 2, characterized in that The historical lane line data is accumulated by historical initial lane line data according to a preset fusion rule.

4. The method according to claim 1, characterized in that: The performing fitting optimization according to the initial lane line data and the historical lane line data in the historical lane line set to obtain the target lane line set includes: Determining historical lane line data matching the initial lane line data from the historical lane line set; Based on an overfitting constraint condition and a parallel constraint condition, the initial lane line data and the historical lane line data are fitted and optimized to obtain a target lane line corresponding to the initial lane line data; A target lane line set is composed of at least one of the target lane lines.

5. The method according to claim 4, characterized in that The historical lane line data includes first aggregated data and second aggregated data; The step of performing fitting optimization on the initial lane line data and the historical lane line data based on the overfitting constraint condition and the parallel constraint condition to obtain a target lane line corresponding to the initial lane line data includes: Determine first incremental data and second incremental data according to the initial lane line data; Based on an equation determined according to an overfitting constraint and a parallel constraint, a target lane line corresponding to the initial lane line data is determined according to the first incremental data, the second incremental data, the first aggregated data, and the second aggregated data.

6. The method according to claim 5, characterized in that The determining of the target lane line corresponding to the initial lane line data includes: Determining polynomial coefficients corresponding to the initial lane line data; A target lane line corresponding to the initial lane line data is determined according to the polynomial coefficients.

7. The method according to claim 5, characterized in that The method further comprises: The historical lane line data is updated according to the initial lane line data, and the updated historical lane line data has the same data volume as the historical lane line data before the update.

8. The method according to claim 7, characterized in that The updating of the historical lane line data according to the initial lane line data comprises: The first incremental data is accumulated to the first aggregate data; and the second incremental data is accumulated to the second aggregate data.

9. The method according to any one of claims 5 to 8, characterized in that: The determining the first incremental data according to the initial lane line data comprises: The first incremental data is determined according to a first coordinate value of the initial lane line on a local map, and the local map is determined according to an environment in which the movable platform is located.

10. The method according to claim 9, characterized in that The determining the second incremental data according to the initial lane line data comprises: The second incremental data is determined according to a first coordinate value of the initial lane line on the local map and a second coordinate value of the initial lane line on the local map.

11. A lane line detection device, characterized in that: It includes a sensor and a processor; the sensor is used to obtain an image of the environment around the movable platform; The processor is used to implement the following steps: Acquire an environment image around the movable platform, and obtain an initial lane line set of the movable platform according to the environment image; Performing fitting optimization on the initial lane line data in the initial lane line set to obtain a target lane line set; wherein the fitting optimization includes: The lane line data is fitted and optimized to obtain a target lane line set, wherein the fitting optimization is used to make the target lane line set include lane lines that are not parallel to each other; When the processor implements the fitting optimization based on the initial lane line data and the historical lane line data in the historical lane line set to obtain the target lane line set, the processor implements: Based on overfitting constraints and parallel constraints, fitting optimization is performed according to the initial lane line data and the historical lane line data in the historical lane line set to obtain a target lane line set. The parallel constraint is used to constrain the parallel state between two adjacent lane lines.

12. The device according to claim 11, characterized in that The initial lane line data and the historical lane line data are processed through a preset data structure, wherein the preset data structure has a certain amount of calculation.

13. The device according to claim 12, characterized in that The historical lane line data is accumulated by historical initial lane line data according to a preset fusion rule.

14. The device according to claim 11, characterized in that When the processor implements the fitting optimization based on the initial lane line data and the historical lane line data in the historical lane line set to obtain the target lane line set, the processor implements: Determining historical lane line data matching the initial lane line data from the historical lane line set; Based on an overfitting constraint condition and a parallel constraint condition, the initial lane line data and the historical lane line data are fitted and optimized to obtain a target lane line corresponding to the initial lane line data; A target lane line set is composed of at least one of the target lane lines.

15. The device according to claim 14, characterized in that The historical lane line data includes first aggregated data and second aggregated data; The processor implements the fitting optimization of the initial lane line data and the historical lane line data based on the overfitting constraint condition and the parallel constraint condition to obtain the target lane line corresponding to the initial lane line data, thereby implementing: Determine first incremental data and second incremental data according to the initial lane line data; Based on an equation determined according to an overfitting constraint and a parallel constraint, a target lane line corresponding to the initial lane line data is determined according to the first incremental data, the second incremental data, the first aggregated data, and the second aggregated data.

16. The device according to claim 15, characterized in that The processor implements the step of determining the initial lane When the target lane line corresponding to the line data is realized: Determining polynomial coefficients corresponding to the initial lane line data; A target lane line corresponding to the initial lane line data is determined according to the polynomial coefficients.

17. The device according to claim 15, characterized in that The processor also implements: The historical lane line data is updated according to the initial lane line data, and the updated historical lane line data has the same data volume as the historical lane line data before the update.

18. The device according to claim 17, characterized in that When the processor implements updating the historical lane line data according to the initial lane line data, the processor implements: The first incremental data is accumulated to the first aggregate data; and the second incremental data is accumulated to the second aggregate data.

19. The device according to any one of claims 15 to 18, characterized in that When the processor implements the determining of the first incremental data according to the initial lane line data, the processor implements: The first incremental data is determined according to a first coordinate value of the initial lane line on a local map, and the local map is determined according to an environment in which the movable platform is located.

20. The device according to claim 19, characterized in that When the processor implements the determining of the second incremental data according to the initial lane line data, the processor implements: The second incremental data is determined according to a first coordinate value of the initial lane line on the local map and a second coordinate value of the initial lane line on the local map.

21. A vehicle, characterized in that: include: The lane line detection device according to any one of claims 11 to 20, used to determine the lane line; Sports components, used for driving.

22. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the lane line fusion method according to any one of claims 1 to 10.

Images