Road element detection method, medium, program product, and electronic device

Abstract

The application relates to the technical field of assisted driving, in particular to a road element detection method, a medium, a program product and an electronic device. The method comprises the following steps: based on M target contour points of a road element at an (i-1)th moment, M prediction contour points of the road element at an ith moment are predicted, wherein i is a positive integer greater than 1, and M is a positive integer greater than 3; based on M observation contour points at the ith moment, a first correction operation is performed on the M prediction contour points at the ith moment, so that M corrected contour points at the ith moment are obtained, wherein the M observation contour points are determined based on sensor information collected when a target vehicle travels at the ith moment; based on geometric constraints corresponding to the road element, a second correction operation is performed on the M corrected contour points, so that L target contour points of the road element at the ith moment are obtained, wherein L is a positive integer less than or equal to M.

Description

Technical Field

[0001] This application relates to the field of driver assistance technology, specifically to a road element detection method, medium, program product, and electronic device. Background Technology

[0002] In assisted driving scenarios, accurate identification and stable tracking of road features (such as lane lines, curbs, and paths) are crucial technologies for ensuring driving safety. However, complex road environments (such as worn road markings, varying lighting, vehicle occlusion, sensor noise, and vehicle dynamic disturbances) can severely interfere with lane line identification, leading to significant errors in the identified perception data (such as the set of points corresponding to road curves). For example, the geometric continuity of road feature curves may be insufficient, and the lateral position estimation of road feature curves constructed from multiple road images may fluctuate greatly between each frame. This makes it difficult to balance accuracy and real-time performance requirements in the identified perception data. Summary of the Invention

[0003] This application provides a method for detecting road features, a readable storage medium, a program product, and an electronic device.

[0004] The technical solution provided in this application has at least the following beneficial effects: it helps to improve the accuracy of road element identification in complex environmental conditions or when the geometric data of road elements identified in the image fluctuates greatly.

[0005] Firstly, this application provides a road element detection method, comprising: predicting M predicted contour points of the road element at time i based on M target contour points of the road element at time i-1, where i is a positive integer greater than 1 and M is a positive integer greater than 3; performing a first correction operation on the M predicted contour points at time i based on the M observed contour points at time i to obtain M corrected contour points at time i, where the M observed contour points are determined based on sensor information collected by the target vehicle at time i; performing a second correction operation on the M corrected contour points based on the geometric constraints corresponding to the road element to obtain L target contour points of the road element at time i, where L is a positive integer less than or equal to M; and determining the assisted driving strategy of the target vehicle at time i based on the road element fitted by the L target contour points at time i.

[0006] According to the road element detection method provided in this application, not only are corrected contour points obtained by correcting predicted contour points through observed contour points, but also the geometric information of corrected contour points is corrected through preset constraint information, so as to avoid the phenomenon of contour point jump caused by the error of observed contour points, thereby improving the accuracy of road element detection.

[0007] In one possible implementation of the first aspect described above, the road elements include at least one of the following: road edge lines, guardrails, retaining walls, guide lines, merging lines, or shoulder lines.

[0008] In one possible implementation of the first aspect described above, the first correction operation includes: determining the correspondence between the M predicted contour points of the road element at time i and the M observed contour points at time i, to obtain M sets of target contour point pairs, wherein each set of target contour point pairs includes one predicted contour point and one observed contour point; calculating the first correction amount for each set of target contour point pairs; and correcting the predicted contour points in each set of target contour point pairs based on the first correction amount for each set of target contour point pairs, to obtain the M corrected contour points at time i.

[0009] In one possible implementation of the first aspect above, the M factorial arrangement of M sets of contour point pairs consisting of the M predicted contour points of the road element at time i and the M observed contour points at time i is determined.

[0010] From M factorial permutations, select L permutations from the M groups of contour point pairs where the distance between each contour point pair is less than a preset distance. The distance between contour point pairs is the distance between the predicted contour point and the observed contour point in the contour point pair. The M groups of contour point pairs with the smallest sum of the distances between the M contour point pairs among the L permutations are selected as the M target contour point pairs.

[0011] In one possible implementation of the first aspect described above, the first cumulative prediction error value of the predicted contour point in the prediction process at the first i-1 time steps and the first cumulative observation error value of the observed contour point in the observation process at the first i-1 time steps are obtained in the target contour point pair; the ratio of the first cumulative prediction error value to the sum of the first cumulative prediction error value and the first cumulative observation error value is used as the first error ratio; and the product of the contour point pair distance of the target contour point pair and the first error ratio is used as the first correction amount of the target contour point pair.

[0012] In one possible implementation of the first aspect described above, the first position of the predicted contour point in the target contour point pair is corrected to the second position to obtain the corrected contour point, wherein the second position is the sum of the first position and the first correction amount.

[0013] In one possible implementation of the first aspect described above, the second correction operation includes: determining the predicted geometric information of each corrected contour point based on the position information of M corrected contour points, wherein the predicted geometric information includes the curvature and slope of the corrected contour points; determining the constraint geometric information corresponding to the predicted geometric information of the M corrected contour points of the road element at time i, wherein the constraint geometric information is the prediction range boundary value of the predicted geometric information; calculating the second correction amount of each corrected contour point based on the predicted geometric information and the constraint geometric information of each corrected contour point; and correcting or deleting each corrected contour point based on the second correction amounts of the M corrected contour points to obtain N target contour points at time i.

[0014] In one possible implementation of the first aspect described above, corresponding to the correction contour point not being located at the endpoint of the road feature, a first slope and a first distance between the correction contour point and the first adjacent contour point among the two adjacent contour points on both sides are obtained, as well as a second slope and a second distance between the correction contour point and the second adjacent contour point among the two adjacent contour points on both sides, wherein the endpoint of the road feature is located at the detection boundary of the road feature; the slope and curvature of the correction contour point are calculated based on the first slope, the first distance, the second slope, and the second distance; the slope and curvature of the correction contour point are weighted and summed to obtain the predicted geometric information at the correction contour point.

[0015] In one possible implementation of the first aspect described above, the average of the first slope and the second slope is determined as the slope of the corrected profile point; the ratio of the difference between the first slope and the second slope to the sum of the first distance and the second distance projected onto the horizontal axis is determined as the curvature of the corrected profile point.

[0016] In one possible implementation of the first aspect described above, the second cumulative prediction error of the predicted geometric information used by each corrected contour point during the correction process in the first i-1 time steps is obtained, and the second cumulative observation error of the constraint geometric information used by the corrected contour point during the correction process in the first i-1 time steps is obtained; the second cumulative prediction error and the second error ratio of the sum of the second cumulative prediction error and the first cumulative preset error are calculated; and the second correction amount of each corrected contour point is determined based on the product of the difference between the predicted geometric information and the constraint geometric information of the corrected contour point and the second error ratio.

[0017] In one possible implementation of the first aspect above, if the second correction amount is greater than the first threshold, the correction contour point corresponding to the second correction amount is deleted, and if the second correction amount is less than or equal to the first threshold, the second position of the correction contour point is corrected to the third position to obtain the target contour point, wherein the third position is the sum of the second position and the second correction amount.

[0018] In one possible implementation of the first aspect above, the road element fitted by the N target contour points of the road element at time i is obtained, and the road element is divided to obtain P division intervals of the same length, where P is a positive integer greater than 2; P+1 endpoints of the P division intervals of the same length are obtained, and the P+1 endpoints are used as the P+1 target contour points of the road element at time i; the M observed contour points of the road element at time i+1 are corrected based on the P+1 target contour points.

[0019] In a second aspect, this application provides a readable storage medium including one or more programs, which, when executed on an electronic device, cause the electronic device to implement any of the methods that may be implemented in the first aspect.

[0020] Thirdly, this application provides a program product that, when executed on an electronic device, enables the electronic device to implement any of the possible methods described in the first aspect.

[0021] Fourthly, this application provides an electronic device, comprising: at least one memory for storing one or more programs; and at least one processor for executing the one or more programs to cause the electronic device to implement any of the possible methods described in the first aspect. Attached Figure Description

[0022] Figure 1 A schematic diagram of road elements provided for embodiments of this application;

[0023] Figure 2 An exemplary flowchart of the road element detection method provided in this application embodiment;

[0024] Figure 3 An exemplary flowchart of the first correction operation provided in the embodiments of this application;

[0025] Figure 4 An exemplary flowchart of the second correction operation provided in the embodiments of this application;

[0026] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions in the embodiments of this application will be described in detail below with reference to the accompanying drawings and specific implementation methods.

[0028] For ease of understanding, this application first explains the terms that may be involved in the embodiments.

[0029] (1) Road elements

[0030] Road elements are components of a road system, typically categorized into discrete road elements and continuous road elements. Discrete road elements can include traffic signs, traffic lights, or guardrails. Continuous road elements can include road edges, guardrails, retaining walls, guide lines, merging lines, or shoulder lines.

[0031] (2) Contour points

[0032] Contour points are discrete point sets used to represent continuous road features. Taking lane lines as an example, the contour points corresponding to lane lines can usually be obtained by discretizing and sampling continuous lane lines. The purpose of contour points is to approximate the contour of continuous lane lines using multiple discrete points, thereby reducing the overhead of computational and storage resources.

[0033] The following section will describe the road element detection method proposed in the embodiments of this application, using lane lines as the sensing target and an image sensor as the sensor for sensing lane lines.

[0034] Figure 1 A schematic diagram of road elements provided in an embodiment of this application is shown. For example... Figure 1 As shown, target vehicle 110 is a vehicle normally traveling on road 120. At time T1, target vehicle 110 is located at the first position 130 on road 120. At time T1, the range that the image sensor of target vehicle 110 can perceive is the first range 140. At time T2, target vehicle 110 has moved to the second position 150. At time T2, the range that the image sensor of target vehicle 110 can perceive is the second range 160.

[0035] It is understandable that lane lines have relatively high geometric continuity and temporal consistency. In other words, during the vehicle's journey, the geometric shape of the lane lines within the vehicle's perception range changes relatively smoothly between two relatively close moments, without any drastic jumps or abrupt changes.

[0036] Therefore, it is understandable that in some embodiments, algorithms such as Kalman filtering can be used to predict and correct the lane lines within the second range of 160 at time T2 based on the lane lines within the first range of 140 at time T1, in order to complete the lane line detection process.

[0037] In some embodiments, the lane lines can be represented by discrete contour points. However, the contour points may be subject to errors during prediction, resulting in the prediction of outliers. In addition, the contour points predicted by the Kalman filter algorithm may also have problems such as deviation and jaggedness, which do not conform to the geometric constraints of the lane lines. These problems will reduce the accuracy of lane line determination.

[0038] To address this, this application proposes a lane line determination method, which includes: predicting M predicted contour points of the road element at time i based on M target contour points of the road element at time i-1, where i is a positive integer greater than 1 and M is a positive integer greater than 3; performing a first correction operation on the M predicted contour points at time i based on the M observed contour points at time i to obtain M corrected contour points at time i, where the M observed contour points are determined based on sensor information collected by the target vehicle at time i; and performing a second correction operation on the M corrected contour points based on the geometric constraints corresponding to the road element to obtain L target contour points of the road element at time i, where L is a positive integer less than or equal to M.

[0039] The technical solution provided in this application has at least the following beneficial effects:

[0040] The embodiments of this application realize the use of the geometric constraints of lane lines at the current moment to correct the predicted contour points, thereby improving the accuracy of road element perception.

[0041] The road element prediction method of this application will be described below with reference to the accompanying drawings. Figure 2 This is an exemplary flowchart of the road feature detection method provided in the embodiments of this application, with reference to... Figure 2 An exemplary process for a road feature detection method includes:

[0042] S200: Obtain the M target contour points of road features within the perception range of the target vehicle at time i-1.

[0043] It is understood that in the embodiments of this application, the target vehicle may be a vehicle that can measure or perceive scene information (such as road elements) in the driving scene through sensors, such as a car, truck or bus, and this application does not limit it.

[0044] The sensors used by the target vehicle to measure scene information may include at least one of lidar, camera, inertial measurement unit or global navigation satellite system, and this application does not limit them.

[0045] It is understandable that, due to the limited accuracy of sensors (for example, 4D imaging radar can detect objects within a range of 300 meters), the range that the target vehicle can perceive is limited. In other words, the target vehicle can only detect road elements within its perception range.

[0046] It is understood that, in the embodiments of this application, the road elements within the perception range of the target vehicle are continuous road elements, such as lane lines, center lines, or guide lines, which are linear and continuous road elements.

[0047] It is understandable that, due to the limited computing resources of the onboard unit in the target vehicle, road features can be discretized using contour points in some embodiments. By fitting multiple discrete contour points to the aforementioned road features, a road feature (e.g., lane lines) can be represented by multiple discrete contour points.

[0048] In some embodiments, the contour points of road features at time i can be detected using a Kalman filter algorithm. It is understood that the Kalman filter algorithm utilizes two recursive steps—a prediction step and an update step—to predict the target value of the contour point at time i+1 based on the target value of the contour point at time i and the observed value of the contour point at time i+1.

[0049] For the sake of uniformity, this application uses target contour points to represent the target value of contour points, predicted contour points to represent the predicted value of contour points obtained from the prediction step, observed contour points to represent the observed value of contour points observed through sensor information, and corrected contour points to represent the corrected value of contour points obtained after correction based on the observed value of contour points.

[0050] As can be understood, in the Kalman filter method, the prediction step is the step of predicting the predicted contour point at time i based on the target contour point of the road feature at time i-1. Therefore, the target contour point at time i-1 must be obtained before performing the prediction step.

[0051] In some embodiments, the M target contour points can be represented by a state vector as shown in formula (1), specifically, formula (1) includes:

[0052] (1)

[0053] in, It is a state vector composed of the coordinates of the target contour points at time i. It is the lateral coordinate of the contour point at time i. I is the vertical coordinate of the contour point at time i, and M is the number of contour points. I is a positive integer that increases sequentially from 1 to M, such as 1, 2, 3, 4...M. M is a positive integer greater than 1, such as 1, 2, 3, 5, 8, 15 or 20, which is not limited in this application.

[0054] Thus, after obtaining the state vector at time i-1, the predicted contour point at time i can be predicted based on the state vector.

[0055] S210: Based on the M target contour points of the road element at time i-1, predict the M predicted contour points of the road element at time i.

[0056] According to S200, M target contour points of the road element at time i-1 can be obtained. After obtaining the M target contour points of the road element at time i-1, the M predicted contour points at time i can be predicted based on the prediction matrix using formula (2). Specifically, formula (2) includes:

[0057] (2)

[0058] in, It is the predicted contour point at time i, determined based on the target contour point at time i-1. Let F be the target contour point at time i-1. F is the prediction matrix, which represents the state transition relationship between the predicted contour point at time i and the target contour point at time i-1.

[0059] For example, when M=2, the specific value of F can be expressed by the following formula (3). Specifically, formula (3) includes:

[0060] (3)

[0061] It can be understood that in the above formula (3), the prediction matrix F is the identity matrix, which means that the position information of the predicted contour point corresponding to the target contour point at the next moment does not change compared with its own position information.

[0062] S220: Obtain M observation contour points of road elements within the perception range of the target vehicle at time i.

[0063] It is understandable that the observed contour points are the location information of the contour points used to fit the road features, obtained by the target vehicle based on the sensors (such as cameras) described in S200 to perceive and analyze the road features within the target vehicle's perception range.

[0064] S230: Based on the M observed contour points, perform the first correction operation on the M predicted contour points of the target vehicle at time i to obtain the M corrected contour points at time i.

[0065] It is understood that the Kalman filter algorithm corrects the predicted value by using the error relationship between the observed value and the predicted value to obtain the target value. Therefore, in this embodiment, M predicted contour points of road elements within the perception range of the target vehicle at time i can be corrected using M observed contour points to obtain M corrected contour points at time i.

[0066] The correction process can be implemented through a first correction operation. According to one embodiment, the first correction operation includes, as follows: Figure 3 Specifically, S231 to S233 shown include:

[0067] S231: Determine the correspondence between the M predicted contour points of the road element at time i and the M observed contour points at time i, and obtain M sets of target contour point pairs.

[0068] It is understandable that a predicted contour point can only be corrected by an observed contour point. In other words, a predicted contour point and the observed contour point that corrects the predicted contour point can form a pair of target contour points.

[0069] Therefore, in order to obtain the above M sets of target contour point pairs, it is also necessary to determine the correspondence between the M predicted contour points and the M observed contour points at time i. The predicted contour points and observed contour points that have a correspondence can form a set of target contour point pairs.

[0070] It is understandable that, since there are M observed contour points and M predicted contour points, in order to determine the correspondence, we can determine the M factorial arrangement of the M sets of contour point pairs consisting of the M predicted contour points of the road element at time i and the M observed contour points at time i.

[0071] For example, for three observed contour points A, B, C and three predicted contour points D, E, F, there are 3! = 6 possible arrangements: [(A, D), (B, E), (C, F)], [(A, D), (C, E), (B, F)], [(B, D), (C, E), (A, F)], [(B, D), (A, E), (C, F)], [(C, D), (B, E), (A, F)] or [(C, D), (A, E), (B, F)].

[0072] It is understood that, in some embodiments, one arrangement can be determined from the above M factorial arrangements to serve as the correspondence between the M predicted contour points and the M observed contour points.

[0073] In some embodiments, the correspondence between M predicted contour points and M observed contour points can be determined from M factorial permutations based on the contour point distance, wherein the contour point pair distance is the distance between the predicted contour point and the observed contour point in the contour point pair.

[0074] For example, firstly, from M factorial permutations, select L permutations where the distance between each pair of contour points is less than a preset distance from the M groups of contour point pairs.

[0075] For example, one of the M factorial permutations can be generated using the K-dimensional tree (Kd-tree) algorithm, and the predicted contour points in each contour point pair can be projected onto the road elements fitted by the observed contour points. If the distance between the projection of a predicted contour point and the observed contour point is less than a preset distance (e.g., 5cm, 6cm, 10cm, 15cm or 20cm, which is not limited in this application), it indicates that the confidence level of the correspondence between the observed contour point and the predicted contour point is low, and this permutation cannot be used as the correspondence between the M predicted contour points and the M observed contour points.

[0076] It is understandable that for M contour point pairs, the smaller the average distance between the contour point pairs, the higher the confidence level that there is a correspondence between the predicted contour point and the observed contour point. Therefore, the M groups of contour point pairs with the smallest sum of contour point pair distances among the L arrangements can be taken as the M groups of target contour point pairs.

[0077] S232: Calculate the first correction for each pair of target contour points.

[0078] It is understandable that, according to S231, target contour point pairs can be obtained. Thus, based on the error between the predicted contour points and the observed contour points in each target contour point pair, the first correction amount corresponding to each predicted contour point can be obtained.

[0079] It is understood that the first correction amount mentioned above can be determined by the first Kalman gain in the Kalman filtering algorithm, and the first Kalman gain is related to the cumulative prediction error value of each predicted contour point and the cumulative observation error value of the observed contour point.

[0080] Therefore, in order to determine the first correction amount for each pair of contour points, it is first necessary to obtain the first cumulative prediction error value of the predicted contour point in the prediction process at the first i-1 time steps, and the first cumulative observation error value of the observed contour point in the observation process at the first i-1 time steps.

[0081] It is understood that the first cumulative prediction error value indicates the prediction error accumulated by the predicted contour points in the historical prediction process, and the second cumulative prediction error value indicates the observation error accumulated by the observed contour points in the historical observation process.

[0082] It is understandable that the larger the first cumulative prediction error, the less accurate the predicted location information of the contour points. Similarly, the larger the first cumulative observation error, the less accurate the observed location information of the contour points.

[0083] It is understandable that the first cumulative observation error and the first cumulative prediction error can be calculated using an error matrix. The error matrix can be represented by the following formula (4), specifically, formula (4) includes:

[0084] (4)

[0085] in, for The matrix represents the error matrix of the j-th contour point, where the value of j can be a positive integer increasing sequentially from 1 to M. Specifically, it can be represented by formula (5):

[0086] (5)

[0087] The elements on the diagonal represent the variance (uncertainty) of the x-coordinate or y-coordinate of the contour points, for example, in... In the diagram, the variance of the x-coordinates of the contour points is 1, and the variance of the y-coordinates is 3. Off-diagonal elements are equal and represent the covariance (indicating correlation) between the x and y coordinates; for example, the covariance between the x and y coordinates is 2.

[0088] It can be understood that the error matrix at time i is obtained by predicting the target error matrix at time i-1 according to formula (6). Specifically, formula (6) includes:

[0089] (6)

[0090] in, This is the prediction error matrix at time i, obtained from the target error matrix at time i-1. Let F be the target error matrix at time i-1, which represents the target value of the error matrix at time i-1 after the prediction error matrix at time i-1 is corrected. F is the prediction matrix introduced in formula (2), and Q is the prediction noise matrix, which represents the process error that may exist in the prediction process.

[0091] Thus, the first Kalman gain can be determined by the relationship between the first cumulative prediction error value and the first cumulative observation error value.

[0092] For example, the first cumulative prediction error value can be used as the ratio of the first cumulative prediction error value to the sum of the first cumulative prediction error value and the first cumulative observation error value as the first error ratio, where the first error ratio is the first Kalman gain.

[0093] It is understood that in some embodiments, the first error ratio as described above can be represented by a first gain matrix, wherein each element in the first gain matrix represents the first error ratio of the contour point corresponding to that element.

[0094] For example, the first gain matrix described above can be represented by the following formula (7), specifically, formula (7) includes:

[0095] (7)

[0096] Wherein, K is used to indicate the first error ratio (first gain matrix); Let H be the prediction error matrix at time i; and let H be the observation matrix at time i. R is the transpose of the observation matrix, used to align the prediction noise in the error matrix to the observation space. For example, when the predicted contour points are represented by three-dimensional coordinates, if the observed contour points are represented by two-dimensional coordinates, the error matrix can be mapped from three-dimensional space to two-dimensional space, realizing the comparability of prediction error and observation error in a unified metric space; R is the observation noise matrix, indicating the process error in the observation process.

[0097] It is understandable that in formula (7), This represents the first cumulative prediction error. K is the sum of the first cumulative prediction error and the first cumulative observation error. Therefore, the larger the first cumulative prediction error, the less accurate the predicted contour points, and the closer K is to 1; the smaller the first cumulative prediction error, the more accurate the predicted contour points, and the closer K is to 0.

[0098] After obtaining the first error ratio, the product of the distance between the target contour point pairs and the first error ratio can be used as the first correction amount for the target contour point pairs.

[0099] In some embodiments, a first correction amount can be obtained according to formula (8), specifically, formula (8) includes:

[0100] (8)

[0101] in, Let K be the first correction value at time i, and K be the first error ratio (first gain matrix). Let H be the observed contour point at time i, and H be the observation matrix. Let be the predicted contour point at time i.

[0102] Understandable. For the distance between contour points, the closer K is to 1, the closer the obtained first correction amount is to the distance between contour points, and the target contour point obtained after correcting the predicted contour point should be closer to the observed contour point; the closer K is to 0, the closer the obtained first correction amount is to 0, and the target contour point obtained after correcting the predicted contour point should be closer to the predicted contour point.

[0103] S233: Based on the first correction amount of each group of target contour point pairs, the predicted contour points in each target contour point pair are corrected to obtain the M corrected contour points at time i.

[0104] In some embodiments, M correction contour points can be determined according to formula (9), specifically, formula (9) includes:

[0105] (9)

[0106] in, Let i be the target contour point at time i. Let i be the predicted contour point at time i. This is the first correction value at time i. It can be understood that, based on the first correction value, the position information of the predicted contour points can be corrected to obtain the corrected contour points at time i.

[0107] In some embodiments, after updating the target contour points, the error matrix is ​​further corrected in reverse according to the first error ratio to obtain a corrected error matrix. In some embodiments, this correction process can be represented by formula (10), specifically, formula (10) includes:

[0108] (10)

[0109] in, The correction error matrix is ​​defined by I, the identity matrix, K, the first error ratio (first Kalman gain), and H, the observation matrix. Let be the prediction error matrix at time i.

[0110] S240: Based on the geometric constraints corresponding to the road features, perform a second correction operation on the M modified contour points to obtain the L target contour points of the road features at time i.

[0111] Where L is a positive integer less than or equal to M, such as 1, 2, 3, 5, 10 or M-1, which is not limited herein.

[0112] It is understandable that some of the corrected contour points obtained in S210 may not conform to the geometric constraints. For example, the obtained corrected contour points may exhibit abrupt changes or deviations from the fitted road features. Therefore, to prevent such situations from occurring, this application adds a second correction operation on the corrected contour points through geometric constraints in the prediction step of the Kalman filter. The geometric constraints may include one or more of the following, such as slope constraints, distance constraints, or curvature constraints, etc., which are not limited in this application.

[0113] The second correction operation will be introduced below using slope constraint and curvature constraint as examples. According to the second correction operation provided in this application, the geometric information of the target contour points can be obtained that is closer to reality.

[0114] For example, the second correction operation can be performed as follows: Figure 4 S241 to S244 shown herein specifically include:

[0115] S241: Based on the position information of M corrected contour points, the predicted geometric information of each corrected contour point is determined. It can be understood that in some embodiments, the predicted geometric information includes the curvature and slope of the corrected contour points. The curvature and slope of each corrected contour point are determined based on its adjacent contour points.

[0116] In some embodiments, corresponding to the modified contour point not being located at the endpoint of the road feature, a first slope and a first distance between the modified contour point and a first adjacent contour point among the two adjacent contour points are obtained, as well as a second slope and a second distance between the modified contour point and a second adjacent contour point among the two adjacent contour points.

[0117] It is understood that the endpoints of road features are located at the detection boundaries of road features. Corresponding to contour points not located at the endpoints of road features, there are two adjacent contour points on both sides (such as the first adjacent contour point and the second adjacent contour point). Therefore, the slope and curvature of the corrected contour point can be calculated by correcting the first unit vector (corresponding to the first slope) of the line connecting the corrected contour point and the first adjacent contour point, the second unit vector (corresponding to the second slope) of the line connecting the corrected contour point and the second adjacent contour point, the first distance between the corrected contour point and the first adjacent contour point, and the second distance between the corrected contour point and the second adjacent contour point; specifically, the slope between the corrected contour point and the first adjacent contour point can be expressed by formula (11), specifically, formula (11) includes:

[0118] (11)

[0119] in, This represents the first unit vector corresponding to the first slope between the corrected contour point and the first adjacent contour point. Indicates the correction of contour points, Indicates the first adjacent contour point, This represents the vector pointing from the corrected contour point to the first adjacent contour point. This represents the magnitude of the vector pointing from the corrected contour point to the first adjacent contour point, i.e., the first distance.

[0120] The second slope between the corrected contour point and the second adjacent contour point can be expressed by formula (12), specifically, formula (12) includes:

[0121] (12)

[0122] in, This represents the second unit vector corresponding to the second slope between the corrected contour point and the second adjacent contour point. Indicates the correction of contour points, Indicates the second adjacent contour point, This represents the vector pointing from the corrected contour point to the second adjacent contour point. This represents the magnitude of the vector pointing from the corrected contour point to the second adjacent contour point, i.e., the second distance.

[0123] Thus, the slope at each corrected contour point can be determined according to the first slope and the second slope, based on the following formula (13). Specifically, formula (13) includes:

[0124] (13)

[0125] in, Let be a unit vector corresponding to the slope at each corrected contour point. and These are the first unit vector and the second unit vector between each corrected contour point and the first and second contour points, respectively.

[0126] It is understandable that in formula (13), the molecule The sum of the first and second unit vectors. This represents the magnitude of the sum of the first and second unit vectors. Thus, the unit vector corresponding to the slope at each corrected contour point can be obtained by comparing the ratio of the sum of the first and second unit vectors to the magnitude of the sum of the first and second unit vectors.

[0127] In some embodiments, the curvature vector of each modified contour point can be represented by formula (14), specifically, formula (14) includes:

[0128] (14)

[0129] Where c is the curvature vector. and These are the first unit vector and the second unit vector, respectively. Indicates the first distance. Indicates the second distance.

[0130] After obtaining the curvature vector, the curvature vector can be calculated using formula (14) to obtain the Euclidean norm value of the curvature vector in formula (14). and the Euclidean norm values The curvature value of the correction contour point is determined.

[0131] After obtaining the slope and curvature of the corrected contour points, the slope and curvature of the corrected contour points can be weighted and summed to obtain the predicted geometric information at the corrected contour points.

[0132] Specifically, the predicted geometric information at the corrected contour points can be represented by the following formula (15), which includes:

[0133] (15)

[0134] Where h represents the predicted geometric information. This is a unit vector corresponding to the slope of the corrected contour points. To correct the curvature of the contour points, α is a preset weight, where the value of α ranges from 0 to 1, such as 0, 0.1, 0.2, 0.5, 0.8 or 1, and this application does not limit it.

[0135] S242: Determine the constraint geometry information corresponding to the predicted geometry information of the M modified contour points of the road element at time i.

[0136] It can be understood that the constraint geometry information is the boundary value of the prediction range of the predicted geometry information. This boundary value is used to subsequently correct the predicted geometry information of each modified contour point.

[0137] S243: Calculate the second correction amount for each corrected contour point based on the predicted geometric information and constraint geometric information of each corrected contour point.

[0138] It is understandable that the second correction value in S243 is also obtained through the update step of the Kalman filter algorithm. Specifically, the predicted value in the Kalman filter process corresponds to the predicted geometric information of the corrected contour points, while the observed value corresponds to the geometric constraint information of the corrected contour points. Thus, the predicted geometric information can be corrected using the geometric constraint information to obtain the target geometric information, and consequently, the target contour points.

[0139] It is understandable that, in order to determine the degree of correction of the predicted geometric information based on the geometric constraint information, that is, the second Kalman gain, the second cumulative prediction error of the predicted geometric information used by each corrected contour point in the correction process of the first i-1 time steps can be obtained, and the second cumulative observation error of the constraint geometric information used by the corrected contour point in the correction process of the first i-1 time steps can be obtained.

[0140] Thus, the second Kalman gain can be determined by calculating the second cumulative prediction error and the second error ratio to the sum of the second cumulative prediction error and the second cumulative observation error.

[0141] It is understood that in some embodiments, the second error ratio as described above can be represented by a second gain matrix, wherein each element in the second gain matrix represents the second error ratio of the contour point corresponding to that element.

[0142] For example, the second gain matrix described above can be represented by the following formula (16), specifically, formula (16) includes:

[0143] (16)

[0144] in, This is the second gain matrix; C is the correction error matrix in formula (11); C is the constraint matrix, which is used to establish the mapping relationship between the state space of the correction contour points and the geometric constraint space, and to convert the geometric states such as the position and tangent direction of the contour points into measurable constraint residuals. This is the noise covariance matrix corresponding to the constraint matrix.

[0145] in, Used to represent the second cumulative prediction error It is used to represent the sum of the second cumulative prediction error and the second cumulative observation error.

[0146] After obtaining the second error ratio, the product of the difference between the predicted geometric information and the constrained geometric information of the corrected contour point and the second error ratio can be used as the second correction amount of the corrected contour point.

[0147] In some embodiments, the second correction amount can be determined by formula (17), specifically, formula (17) includes:

[0148] (17)

[0149] in, This indicates the second correction amount. This represents the second error ratio. This represents the error between the predicted geometric information and the geometric constraints of the corrected contour points, where d represents the constraint target vector, which is usually taken as a 0 vector, meaning that the constraint error is required to be 0.

[0150] In some embodiments, after obtaining the second correction amount, the error matrix is ​​further updated a second time. The update method can be represented by formula (18), specifically, formula (18) includes:

[0151] (18)

[0152] in, Let be the correction error matrix at time i. This represents the target error matrix after the second correction at time i. Let C be the second gain matrix and C be the constraint matrix.

[0153] S244: Based on the second correction amount of M correction contour points, each correction contour point is corrected or deleted to obtain N target contour points at time i.

[0154] In some embodiments, when the magnitude of the second correction amount is less than or equal to the correction threshold, the second position of the correction contour point can be corrected to the third position to obtain the target contour point, wherein the third position is the sum of the second position and the second correction amount.

[0155] In some embodiments, each modified contour point can be corrected using formula (19), specifically, formula (19) includes:

[0156] (19)

[0157] in, To correct the contour points, This is the second correction amount. The target contour point at time i.

[0158] In some embodiments, If the corresponding correction range is greater than the correction threshold, the correction contour points corresponding to the second correction amount can also be deleted.

[0159] In some embodiments, if the uniformity of the spacing between the target contour points obtained in S240 is poor, the position information of the target contour points can be corrected after S240 to make the target contour points evenly distributed on the road elements.

[0160] It is understandable that, in order to obtain the corrected spacing of the contour points, we can obtain the road features fitted by the N target contour points of the road features at time i, and divide the road features to obtain P division intervals of the same length.

[0161] Where P is a positive integer greater than 2, such as 3, 4, 5, 8, 10, 15 or 20, which is not limited in this application.

[0162] After obtaining the division interval, P+1 endpoints of P division intervals with the same length can be obtained according to the division interval, and the P+1 endpoints can be used as the P+1 target contour points of the road element at time i.

[0163] Thus, based on the P+1 target contour points at time i, the P+1 predicted contour points at time i+1 can be predicted. Then, using the P+1 observed contour points at time i+1, the P+1 predicted contour points at time i+1 are corrected to obtain the P+1 corrected contour points at time i+1. Finally, using the geometric information of these P+1 corrected contour points, they are corrected again to obtain the P+1 target contour points at time i+1.

[0164] S250: Determine the assisted driving strategy for the target vehicle at time i based on the road features fitted by the L target contour points at time i.

[0165] It is understandable that after S240 obtains L target contour points at time i, the road elements at time i can be fitted based on the L target contour points, and then the road elements at time i can be used as a reference for subsequent assisted driving strategies.

[0166] The method for fitting the road element at time i based on L target contour points can include least squares fitting or quadratic polynomial fitting. According to the above methods, discrete contour points can be fitted into continuous and smooth road elements.

[0167] This application also provides a computer-readable storage medium. The computer-readable storage medium can be any storage medium (e.g., magnetic medium, optical medium, semiconductor medium, etc.) capable of storing and / or retrieving data by a computing device. The computer-readable storage medium includes instructions that direct a computing device to implement the road element detection method provided in this application.

[0168] This application also provides a program product, which may be a software or program product including instructions, capable of running on a computing device or stored on any usable medium. When the computer program product runs on at least one computing device, the at least one computing device implements the road element detection method provided in this application.

[0169] This application also provides an electronic device, including one or more processors and one or more memories, wherein the one or more processors can be used to execute instructions to implement the road element detection method provided in this application.

[0170] Figure 5 This diagram illustrates the structure of an electronic device 500 according to some embodiments of this application.

[0171] like Figure 5 As shown, the electronic device 500 includes one or more processors 501, system memory 502, non-volatile memory (NVM) 503, communication interface 504, input / output (I / O) device 505, and system control logic 506 for coupling the processor 501, system memory 502, non-volatile memory 503, communication interface 504, and input / output (I / O) device 505. Wherein:

[0172] Processor 501 may include one or more processing units, such as data processing units or processing circuits that may include a central processing unit (CPU), graphics processing unit (GPU), digital signal processor (DSP), micro-programmed control unit (MCU), artificial intelligence (AI) processor or field programmable gate array (FPGA), neural-network processing unit (NPU), etc., and may include one or more single-core or multi-core processors.

[0173] System memory 502 is volatile memory, such as random-access memory (RAM), double data rate synchronous dynamic random access memory (DDR SDRAM), etc. System memory is used for temporary storage of data and / or instructions. For example, in some embodiments, system memory 502 can be used to store instructions, as well as raw data objects and modified data objects.

[0174] The non-volatile memory 503 may include one or more tangible, non-transitory computer-readable media for storing data and / or instructions. In some embodiments, the non-volatile memory 503 may include any suitable non-volatile memory and / or any suitable non-volatile storage device, such as a hard disk drive (HDD), compact disc (CD), digital versatile disc (DVD), solid-state drive (SSD), etc. In some embodiments, the non-volatile memory 503 may also be a removable storage medium, such as a secure digital (SD) memory card. In other embodiments, the non-volatile memory 503 may be used to store instructions, or to store original data objects and modified data objects.

[0175] In some embodiments, system memory 502 and non-volatile memory 503 may each include a temporary copy and a permanent copy of instructions 507. Instructions 507 may include, when executed by at least one of processors 501, causing electronic device 500 to implement the methods provided in the embodiments of this application.

[0176] The communication interface 504 may include a transceiver for providing a wired or wireless communication interface for the electronic device 500, thereby enabling communication with any other suitable device via one or more networks. In some embodiments, the communication interface 504 may be integrated into other components of the electronic device 500, for example, the communication interface 504 may be integrated into the processor 501. In some embodiments, the electronic device 500 may communicate with other devices through the communication interface 504. For example, the electronic device 500 may establish a communication connection with other devices through the communication interface 504 to send data change requests, obtain original data objects, and send changed data objects to other devices via the communication connection.

[0177] Input / output (I / O) device 505 may include input devices such as keyboards and mice, and output devices such as monitors. Users can interact with electronic devices 500 through input / output (I / O) device 505. For example, business personnel can input / select the content to be changed through input / output (I / O) device 505.

[0178] System control logic 506 may include any suitable interface controller to provide any suitable interface to other modules of electronic device 500. For example, in some embodiments, system control logic 506 may include one or more memory controllers to provide an interface to system memory 502 and non-volatile memory 503.

[0179] In some embodiments, at least one of the processors 501 may be packaged together with the logic of one or more controllers for system control logic 506 to form a system in package (SiP). In other embodiments, at least one of the processors 501 may also be integrated on the same chip with the logic of one or more controllers for system control logic 506 to form a system-on-chip (SoC).

[0180] Understandable. Figure 5 The structure of the electronic device 500 shown is merely an example. In other embodiments, the electronic device 500 may include more or fewer components than illustrated, or combine or separate certain components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of both.

[0181] Various embodiments of the mechanisms disclosed in this application can be implemented in hardware, software, firmware, or a combination of these implementation methods. Embodiments of this application can be implemented as computer modules or module code executing on a programmable system, the programmable system including at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device.

[0182] Module code can be applied to input instructions to execute the functions described in this application and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, the processing system includes any system having a processor such as, for example, a digital signal processor (DSP), a microcontroller, an application-specific integrated circuit (ASIC), or a microprocessor.

[0183] Module code can be implemented using a high-level modular language or an object-oriented programming language to communicate with the processing system. Assembly language or machine language can also be used to implement module code when needed. In fact, the mechanisms described in this application are not limited to any particular programming language. In either case, the language can be a compiled language or an interpreted language.

[0184] In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions carried on or stored thereon by one or more transient or non-transitory machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors. For example, the instructions may be distributed via a network or through other computer-readable media. Therefore, machine-readable media can include any mechanism for storing or transmitting information in a machine-readable (e.g., computer-readable) form, including but not limited to floppy disks, optical disks, CD-ROMs, compact disc read-only memory (CD-ROMs), magneto-optical disks, read-only memory (ROM), random access memory (RAM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic cards or optical cards, flash memory, or tangible machine-readable storage for transmitting information (e.g., carrier waves, infrared signals, digital signals, etc.) using the Internet in the form of electrical, optical, acoustic, or other forms of propagation signals. Therefore, machine-readable media includes any type of machine-readable medium suitable for storing or transmitting electronic instructions or information in a machine-readable (e.g., computer-readable) form.

[0185] In the accompanying drawings, some structural or methodological features may be shown in a specific arrangement and / or order. However, it should be understood that such a specific arrangement and / or order may not be necessary. Rather, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the illustrative drawings. Furthermore, the inclusion of structural or methodological features in a particular figure does not imply that such features are required in all embodiments, and in some embodiments, these features may be omitted or may be combined with other features.

[0186] Various embodiments of the mechanisms disclosed in this application can be implemented in hardware, software, firmware, or combinations of these implementation methods. Embodiments of this application can be implemented as computer programs or program code executable on a programmable system, the programmable system including at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device.

[0187] Program code can be applied to input instructions to execute the functions described in this application and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, the processing system includes any system having a processor such as, for example, a digital signal processor (DSP), a microcontroller, an application-specific integrated circuit (ASIC), or a microprocessor.

[0188] The program code can be implemented using a high-level procedural language or an object-oriented programming language to communicate with the processing system. Assembly language or machine language can also be used when needed. In fact, the mechanisms described in this application are not limited to any particular programming language. In either case, the language can be a compiled language or an interpreted language.

[0189] It should be noted that all units / modules mentioned in the device embodiments of this application are logical units / modules. Physically, a logical unit / module can be a physical unit / module, a part of a physical unit / module, or a combination of multiple physical units / modules. The physical implementation of these logical units / modules themselves is not the most important factor; the combination of functions implemented by these logical units / modules is the key to solving the technical problems proposed in this application. Furthermore, to highlight the innovative aspects of this application, the above-described device embodiments of this application have not introduced units / modules that are not closely related to solving the technical problems proposed in this application. This does not mean that the above-described device embodiments do not contain other units / modules.

[0190] It should be noted that in the examples and description of this application, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one" does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0191] In this specification, references to "some embodiments" or "embodiments" mean that a specific feature, structure, or characteristic described in connection with an embodiment is included in at least one exemplary implementation or technology disclosed according to embodiments of this application. The phrase "in some embodiments" appearing in various places in the specification does not necessarily refer to the same embodiment.

[0192] Furthermore, the language used in this specification has been primarily chosen for readability and instructional purposes and may not have been chosen to depict or limit the disclosed subject matter. Therefore, the embodiments disclosed herein are intended to illustrate, and not limit, the scope of the concepts discussed herein.

Claims

1. A method for detecting road elements, characterized in that, The method includes: Based on the M target contour points of the road element at time i-1, predict the M predicted contour points of the road element at time i, where i is a positive integer greater than 1 and M is a positive integer greater than 3. Based on the M observed contour points at time i, a first correction operation is performed on the M predicted contour points at time i to obtain the M corrected contour points at time i. The M observed contour points are determined based on sensor information collected by the target vehicle at time i. Based on the geometric constraints corresponding to the road elements, a second correction operation is performed on the M corrected contour points to obtain L target contour points of the road elements at time i, where L is a positive integer less than or equal to M. Based on the road features fitted by the L target contour points at time i, determine the assisted driving strategy for the target vehicle at time i. The method further includes: Obtain the road features fitted by the N target contour points of the road features at time i, and divide the road features to obtain P division intervals of the same length, where P is a positive integer greater than 2; Obtain P+1 endpoints of the P division intervals with the same length, and use the P+1 endpoints as the P+1 target contour points of the road element at time i. Based on the P+1 target contour points fitted by the road elements at time i+1, the P+1 target contour points at time i are corrected.

2. The method according to claim 1, characterized in that, The road elements include at least one of the following: Roadside lines, guardrails, retaining walls, guide lines, merging lines, or shoulder lines.

3. The method according to claim 1, characterized in that, The first correction operation includes: Determine the correspondence between the M predicted contour points of the road element at time i and the M observed contour points at time i to obtain M sets of target contour point pairs, wherein each set of target contour point pairs includes one predicted contour point and one observed contour point. Calculate the first correction for each pair of target contour points; Based on the first correction amount of each group of target contour point pairs, the predicted contour points in each target contour point pair are corrected to obtain the M corrected contour points at the i-th time.

4. The method according to claim 3, characterized in that, The process involves determining the correspondence between the M predicted contour points of the road element at time i and the M observed contour points at time i, resulting in M ​​pairs of target contour points, including: Determine the M factorial arrangements of the M predicted contour points of the road element at time i and the M observed contour points at time i. From the M factorial permutations, select L permutations from the M groups of contour point pairs where the distance between each contour point pair is less than a preset distance, wherein the distance between the contour point pairs is the distance between the predicted contour point and the observed contour point in the contour point pair. Among the L arrangements, the M groups of contour point pairs with the smallest sum of distances among the M contour point pairs are taken as the M groups of target contour point pairs.

5. The method according to claim 4, characterized in that, The calculation of the first correction amount for each pair of target contour points includes: In the target contour point pair, the first cumulative prediction error value of the predicted contour point in the prediction process at the first i-1 time steps, and the first cumulative observation error value of the observed contour point in the observation process at the first i-1 time steps; The first error ratio is the ratio of the first cumulative prediction error value to the sum of the first cumulative prediction error value and the first cumulative observation error value. The product of the distance between the target contour points and the first error ratio is used as the first correction amount for the target contour points.

6. The method according to claim 5, characterized in that, The first correction amount based on each pair of target contour points is used to correct the predicted contour points in each pair of target contour points to obtain the M corrected contour points at the i-th time, including: The first position of the predicted contour point in the target contour point pair is corrected to the second position to obtain the corrected contour point, wherein the second position is the sum of the first position and the first correction amount.

7. The method according to claim 1, characterized in that, The second correction operation includes: Based on the position information of the M corrected contour points, the predicted geometric information of each corrected contour point is determined, wherein the predicted geometric information includes the curvature and slope of the corrected contour point; Determine the constraint geometric information corresponding to the predicted geometric information of the M corrected contour points of the road element at time i, where the constraint geometric information is the prediction range boundary value of the predicted geometric information; Based on the predicted geometric information and constraint geometric information of each modified contour point, calculate the second correction amount for each modified contour point; Based on the second correction amount of the M corrected contour points, each corrected contour point is corrected or deleted to obtain the N target contour points at the i-th time.

8. The method according to claim 7, characterized in that, The step of determining the predicted geometric information of each corrected contour point based on the position information of the M corrected contour points includes: Corresponding to the fact that the corrected contour point is not located at the endpoint of the road element, the first slope and the first distance between the corrected contour point and the first adjacent contour point among the two adjacent contour points are obtained, as well as the second slope and the second distance between the corrected contour point and the second adjacent contour point among the two adjacent contour points, wherein the endpoint of the road element is located at the detection boundary of the road element. The slope and curvature of the corrected contour point are calculated based on the first slope, the first distance, the second slope, and the second distance. The slope and curvature of the corrected contour points are weighted and summed to obtain the predicted geometric information at the corrected contour points.

9. The method according to claim 8, characterized in that, The step of calculating the slope and curvature of the corrected contour point based on the first slope, the first distance, the second slope, and the second distance includes: The average of the first slope and the second slope is determined as the slope of the corrected contour point; The curvature of the corrected contour point is determined by the ratio of the difference between the first slope and the second slope, and the sum of the first distance and the second distance projected onto the horizontal axis.

10. The method according to claim 9, characterized in that, Based on the predicted geometric information and constraint geometric information of each corrected contour point, the second correction amount for each corrected contour point is calculated, including: The second cumulative prediction error of the predicted geometric information used by each corrected contour point during the correction process in the first i-1 time steps is obtained, and the second cumulative observation error of the constraint geometric information used by the corrected contour point during the correction process in the first i-1 time steps is obtained. Calculate the second cumulative prediction error and the second error ratio to the sum of the second cumulative prediction error and the second cumulative observation error; The product of the difference between the predicted geometric information and the constrained geometric information of the corrected contour point and the second error ratio is used as the second correction amount of the corrected contour point.

11. The method according to claim 10, characterized in that, The second correction amount based on the M corrected contour points is used to correct or delete each corrected contour point to obtain the N target contour points at the i-th time, including: If the second correction amount is greater than the correction threshold, the correction contour points corresponding to the second correction amount will be deleted. Furthermore, if the second correction amount is less than or equal to the correction threshold, the second position of the correction contour point is corrected to the third position to obtain the target contour point, wherein the third position is the sum of the second position and the second correction amount.

12. A readable storage medium, characterized in that, The readable storage medium includes one or more programs that, when executed on an electronic device, cause the electronic device to perform the method according to any one of claims 1 to 11.

13. A program product, characterized in that, When the program product is executed on an electronic device, it causes the electronic device to perform the method described in any one of claims 1 to 11.

14. An electronic device, characterized in that, include: At least one memory for storing one or more programs; At least one processor is configured to execute the one or more programs to cause the electronic device to perform the method of any one of claims 1 to 11.

Images