Lane marking determination method and apparatus

Abstract

The present application discloses one or more embodiments of a lane marking determination method and device. The method comprises: acquiring feature data of an environmental reference located around a current driving road, wherein the environmental reference is a reference within a predetermined spatial range around a vehicle driving on the driving road, and the feature data comprises position information of the environmental reference; processing the feature data to obtain an auxiliary marking line of the current driving road; and inputting multi-point sampling data of the auxiliary marking line into a prediction model to predict lane markings on the current driving road. The lane marking determination device of the present application can be applied to an automatic driving system.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of automatic driving, in particular to a lane prediction technology. BACKGROUND

[0002] With the development of artificial intelligence and network technology, it is possible to realize automatic driving of vehicles. However, one problem to be faced in the automatic driving technology is the planning of a driving road, which not only involves the selection of a driving road, i.e. a route, but also involves the selection of a specific lane on the selected road for driving. Usually, a vehicle recognizes a lane based on lane markings, i.e. lane lines, on the lane or with the help of a high-precision map. However, when the lane markings are missing or the vehicle-mounted visual system cannot see the lane markings or the high-precision map cannot provide the lane markings, the vehicle cannot recognize the lane, which may result in the vehicle being unable to drive on a safe lane, thus bringing about a safety hazard in the driving process. SUMMARY

[0003] The present application aims to provide a lane marking prediction technology, which can determine a normal lane position by predicting lane markings when the lane markings are missing or the high-precision map cannot provide the lane markings.

[0004] According to one aspect of the present application, a lane marking determination method is provided, comprising: acquiring feature data of environmental reference objects located around a current driving road, wherein the environmental reference objects are reference objects within a predetermined spatial range around a vehicle driving on the driving road, and the feature data comprises position information of the environmental reference objects; processing the feature data to obtain an auxiliary marking line corresponding to at least one section of the current driving road; and inputting position data of multiple points sampled from the auxiliary marking line into a prediction model to predict lane markings of at least one section of the current driving road.

[0005] According to one aspect of the present application, a lane marking determination device is provided, comprising: a reference object acquisition module configured to acquire feature data of environmental reference objects located around a current driving road, wherein the environmental reference objects are reference objects within a predetermined spatial range around a vehicle driving on the driving road, and the feature data comprises position information of the environmental reference objects; an auxiliary marking determination module configured to process the feature data to obtain an auxiliary marking line corresponding to at least one section of the current driving road; and a lane marking determination module configured to input position data of multiple points sampled from the auxiliary marking line into a prediction model to predict lane markings of at least one section of the current driving road.

[0006] According to the present application, an automatic driving system can also be implemented, which utilizes the lane marking determination device of the present application to select a current driving lane based on the lane markings determined by the lane marking determination device under the control of a vehicle-mounted control unit.

[0007] The present application also provides a lane marking determination apparatus comprising a memory storing instructions, and a controller configured to implement the method according to the present application by executing the instructions.

[0008] According to the present application, there is also provided a machine-readable medium having stored thereon machine-readable instructions which, when executed by a machine, cause the machine to perform the method according to the present application. BRIEF DESCRIPTION OF DRAWINGS

[0009] Figure 1 An exemplary schematic diagram of a road scene is shown;

[0010] Figure 2A With 2B A block diagram of a lane marking determination apparatus according to an embodiment of the present application is shown;

[0011] Figures 3A-3E A schematic diagram showing the stages of a lane marking prediction process according to an embodiment of the present application is shown;

[0012] Figure 4 A block diagram of a model training system according to an embodiment of the present application is shown;

[0013] Figure 5 A schematic diagram showing model training according to an embodiment of the present application is shown;

[0014] Figure 6 A flowchart of a lane marking determination method according to an embodiment of the present application is shown;

[0015] Figure 7 A flowchart of a lane marking determination method according to another embodiment of the present application is shown;

[0016] Figure 8 A block diagram of a lane determination apparatus according to another embodiment is shown. DETAILED DESCRIPTION

[0017] The apparatus and methods according to embodiments of the present application will now be described in detail with reference to the accompanying drawings. Although preferred embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure will be thorough and complete, and will fully convey the scope of the present disclosure to those skilled in the art.

[0018] In the present disclosure, the "traveling road" refers to the current traveling route, which contains at least one or more lanes, and the driver or the automatic driving system of the vehicle will travel in a selected lane. Here, each lane is defined by the lane markings on both sides of the lane, and can be further represented by the center line determined by the lane markings on both sides.

[0019] With the development of cities and transportation construction, roads and their surrounding facilities are becoming more and more standardized. The standardization of roads includes the standardization of road specifications such as lane width and the setting of lane markings, i.e. lane dividers. The standardization of supporting facilities includes the setting of isolation barriers, isolation belts, and traffic signs on the road. In addition, supporting facilities also include beautifying the road, such as planting green isolation belts, trees, etc. Usually, these supporting facilities and roads are arranged according to certain requirements, including relative distance, interval, etc. For example, Figure 1 A schematic diagram of a one-way expressway is shown. As shown, the road includes three lanes, which are divided by lane marking lines (or lane lines). In addition, an isolation barrier is provided between the marking B-B of the innermost lane and the reverse lane, and a barrier is also provided outside the marking A-A of the outermost lane. In addition, as shown, the lane marking line A-A of the outermost lane is spaced apart from the barrier at a predetermined distance, and a "G108 entrance" sign, trees, etc. are provided outside the barrier. The distance of these fixed barriers and the like relative to the lane marking is fixed and does not change, which constitutes the surrounding environment reference of the road, so that it is possible to determine the lane marking according to this information. The present application utilizes these supporting facilities to predict the lane marking by referring to these supporting facilities in the case of losing lane line markings or the vehicle being unable to identify the lane marking, so as to ensure that the vehicle still travels along the correct lane.

[0020] Figure 2A A block diagram of a lane marking determination device 100 according to an embodiment of the present application is shown. As shown, the lane marking determination device 100 includes a reference object acquisition module 200, an auxiliary marking determination module 300, and a lane marking determination module 400. The reference object acquisition module 200 is designed to acquire feature data of environmental reference objects located in the surrounding of the current traveling road, wherein the environmental reference objects are reference objects within a predetermined spatial range around the current vehicle traveling on the traveling road, and the features include but are not limited to the position information of the environmental reference objects. As an example, the predetermined spatial range includes a scanning space of a certain angle range from left to right in front of the current vehicle. Still taking Figure 1For example, in the predetermined spatial range, the multiple reference objects include not only the left guardrail and the right guardrail, but also the "G108 entrance" sign on the upper right, and even the green trees in the distance. In addition to the guardrails, it is not difficult to understand that the environmental reference objects can also include other moving vehicles (not shown in the figure) that are just in the predetermined spatial range. For the sake of description, in the following description, the reference objects such as guardrails are simply referred to as static reference objects, and moving vehicles and other moving objects are referred to as dynamic reference objects or moving objects.

[0021] The auxiliary marker determination module 300 processes the feature data obtained by the reference object acquisition module 200 to obtain an auxiliary marker line corresponding to a road segment RS of the current driving road. The auxiliary marker line is a virtual reference line derived from the reference objects, and the lane marker of the current road can be predicted using the virtual reference line. Specifically, the lane marker determination module 400 uses a pre-trained prediction model PM to input the position data obtained by sampling multiple points of the auxiliary marker line into the prediction model PM to predict the lane marker of the road segment RS of the current driving road.

[0022] Figure 2B The structure block diagram of the lane marker determination device 100 according to an embodiment of the present application is shown. As shown in the figure, the reference object acquisition module 200 includes a transceiver unit 202 and a feature determination unit 204. The transceiver unit 202 can be a radar transmitter and receiver located in the vehicle, wherein the transmitter is used to emit radar wave detection signals at a predetermined radiation angle to a predetermined range for detecting environmental reference objects in the predetermined spatial range, and the receiver is used to receive radar reflection signals from the environmental reference objects. It is obvious that the features of the environmental reference objects, including but not limited to the position of the environmental reference objects, can be determined based on the reflection signals. Figure 1 For example, for the sake of simplicity, it is assumed here that in the reflection signals received by the receiver, only the reflection signals of the left guardrail, the reflection signals of the right guardrail, and the reflection signals of the vehicle driving in front (not shown in the figure) are included.

[0023] After receiving the radar reflection signals, the feature determination unit 204 analyzes the reflection signals using a predetermined algorithm to determine the position features of the reference objects, such as the relative position between the current vehicle and the reference objects, including the relative distance and the direction. Here, any algorithm known in the prior art can be used to analyze and determine the position features of the reference objects. Subsequently, the feature determination unit 204 determines the actual physical positions of the reference objects based on the geographical position data of the current vehicle (which can be obtained from the positioning system built in the vehicle, such as GPS) and the relative positions of the reference objects determined above. Still taking Figure 1For example, the feature determination unit 204 can determine a position feature data set P of a left guardrail, a right guardrail, and other moving objects such as vehicles that can appear on the current driving road. For the purpose of description, the feature data set corresponding to the left guardrail is denoted as P L , the feature data set corresponding to the right guardrail is denoted as P R , and the feature data set corresponding to the moving objects is denoted as P M . That is, P = {P L ,

[0024] P M , P R}.

[0025] According to an embodiment of the present application, the auxiliary marker determination module 300 includes a filtering unit 302, a clustering module 304, and a fitting module 306. As mentioned above, since the moving objects such as other vehicles appearing on the current driving road are not helpful for determining the lane marker and are a kind of noise, the filtering unit 302 is used to perform speed filtering on the feature data set P to eliminate the noise data set P M with speed characteristics from the peripheral moving vehicles and other moving objects in the acquired position feature data set P L , thereby obtaining the feature data set P' of the static reference, i.e., P' = {P R , P L}. Figure 3A The data set P R representing a plurality of feature points of the static reference on the current road is shown.

[0026] The clustering module 304 performs clustering processing on the feature data set P' of the static reference to obtain a plurality of classes and the center position data of each class. Any clustering algorithm known in the prior art can be used to implement the clustering processing. For example, the feature data set P L of the road section RS on the current driving road can be clustered into m classes, i.e., m feature data subsets, as shown in Figure 3B . Then, the center point of each feature data subset is determined by using the feature data in the subset, so that the data subset of each class is clustered into one point representing the class. For the purpose of description, the positions of the m center points in the feature data set P m are denoted as p L , and the position of p i is represented by (x i , y i ), i = 1, 2,... m. Similarly, the feature data set P R is clustered into n classes, i.e., n feature data subsets, as shown in Figure 3BThen, using the feature data in each feature data subset, determine the center point of the subset, so that each class of data subset is clustered into a point representing the class. For ease of explanation, we use p1′, p2′, … p n ′ represents the feature dataset P R The positions of n central points in the i The position of ′ is given by (x i ′,y i ′) represents i=1, 2, ... n. It should be noted that the feature data set P obtained by the clustering module 304 is L The number of clusters m and feature dataset P L The number of clusters n can be the same or different and is completely determined by the clustering algorithm.

[0027] For the left reference feature dataset P L , the fitting module 306 calculates the center positions p1, p2, ... p of m classes m Line fitting is performed to obtain auxiliary marking lines, which can well fit the changing pattern of these position points. Considering that the actual lane is not always straight, for example, there may be curves, the position of the reference object will also change accordingly with the lane. In order to better fit the lane changes, according to one embodiment, a standard second-order function is used to fit the feature data set P L The standard function is expressed as: y = k2*x 2 +k1*x+k0. Thus, P L The data in {(x1,y

[0028] 1),(x2,y2)…(x m ,y m By training the standard second-order function to determine the parameters k2, k1, and k0, an auxiliary marking line L reflecting the change law of the environmental reference object on the road section RS can be obtained. L It is not difficult to understand that for a relatively straight lane or guardrail, on the road section RS, the auxiliary marking line is a linear function, such as Figure 3C As shown, y=k1*x+k0, where y represents the longitudinal coordinate and x represents the transverse coordinate.

[0029] Similarly, using the standard function y = k2'*x 2 +k1′*x+k0′ to the feature dataset P R The midpoint matrix is ​​fitted by P R {(x1′,y1′),(x2′,y2′)…(x n ′,y nAt least a portion of the data in ')} is trained by the standard function to determine the parameters k2', k1', k0', thereby obtaining the auxiliary marking line L reflecting the change law of the right environmental reference object over the predetermined distance D R . Figure 3C It is easy to understand that in practice, the auxiliary marking line L R With L L The curve characteristics are basically the same, so in a simplified scheme, the same curve or straight line function can also be used to represent L R With L L .

[0030] The lane marking determination module 400 respectively determines the lane marking line L from the auxiliary marking line L R 、L L Upsample N pairs of relative points. Here, each pair of "relative points" refers to points with the same vertical coordinate y, that is, each pair of "relative points" is located on the same horizontal line. Here, each pair of relative points is represented as [(y i ,x i ),(y i ,x i ′)]. It should be noted that the N pairs of relative points selected here can be the auxiliary marking lines L obtained by fitting the road section RS. R 、L L N points are randomly sampled on the .

[0031] Then, the lane marking determination module 400 inputs the N pairs of sampling point data into the trained prediction model PM to predict the lane marking on the current driving road. The lane marking can be the leftmost or rightmost lane marking on the current driving road, for example Figure 1 The prediction model PM utilized in the present invention is trained based on a large amount of actual measurement data by learning the relationship between environmental reference objects and lane positions or markings. In one embodiment, the trained prediction model PM is a linear model, i.e., X = A·θ. Here, the output X represents the lateral coordinate of the actual lane marking, and θ is the trained linear coefficient matrix, which can be expressed as follows:

[0032]

[0033] Where d1 represents the distance between the left auxiliary marking line and the leftmost lane marking, and d2 represents the distance between the right auxiliary marking line and the rightmost lane marking. The values ​​of d1 and d2 are obtained through training. A is the distance between the auxiliary marking line L and the leftmost lane marking. R With L LThe matrix formed by the horizontal coordinates of the sampling points, where A = 0.5*[(x+x′)-q*d,1,-1], where x represents L L The horizontal coordinate of the sampling point on L R . q represents the number of lanes on the current road and can be estimated based on the left and right markings x and x′, for example, q = (x′-x-d²) / d, where d represents the width of each lane. Typically, the lane width d is the default value d0 as specified by national traffic regulations. In another embodiment of the present invention, based on the road on which the current vehicle is located, when either an offline or online map is available, the number of lanes q on the current road can also be directly obtained from the map, denoted as q0 (this situation occurs, for example, when lane markings are missing or the autonomous vehicle cannot "see" the lane markings for various reasons, but the map is still available).

[0034] Therefore, for the auxiliary marking line L R With L L The points on the actual lane line corresponding to any two relative points at horizontal positions on the can be determined by the linear model PM. Figure 3D As shown, the two outermost solid lines represent auxiliary marking lines L R With L L Specifically, it is assumed that from each auxiliary marking line L on the road section RS R With L L The data of t points are sampled, that is, on the auxiliary marking line L L Upsampled the position coordinates (x1, y1), (x2, y2)… (x t ,y t ) of t points, and on the auxiliary marking line L R Upsampled the position coordinates (x1′,y1), (x2′,y2)…(x t ′,y t ) points, then the horizontal position coordinate X of the leftmost lane marker No. ① is 1 It can be determined by the following formula:

[0035] X 1 1 = A1·θ, where A1 = 0.5*[(x1+x1′)-q*d,1,-1]

[0036] X 1 2 = A2·θ, where A2 = 0.5*[(x2+x2′)-q*d,1,-1]

[0037] …

[0038] X 1 t =A t θ, where At =0.5*[(x t +x t ′)-q*d,1,-1]

[0039] From this, we can predict the corresponding coordinate position of lane marking line ① on the road section RS as follows: (X 1 1,y1),(X 1 2,y2)…(X 1 t ,y t ), these points form the leftmost lane marking line, Figure 3D The dotted line indicated by mark ① in the middle represents the predicted leftmost lane marking.

[0040] After determining the leftmost lane marking, the corresponding points on the lane markings on both sides of any lane can be predicted accordingly. For example, for any lane i from left to right, the horizontal coordinates of the corresponding points on the lane markings on both sides of the i-th lane can be expressed as:

[0041] X i,左 =X 1 +(i-1)*d

[0042] X i,右 =X 1 +i*d.

[0043] like Figure 3E As shown in FIG, a schematic diagram of calculating other lane markings using the leftmost lane marking based on the above algorithm is shown, as shown by the dotted lines indicated by marks ②③④, where mark ④ represents the rightmost lane marking line.

[0044] In the above embodiment, the leftmost lane marking is predicted first using feature data as an example. It is easy to understand that the above linear model can also be used to predict the rightmost lane marking first. However, in this case, A = 0.5*[(x+x′)+q*d,1,-1]. Thus, Figure 3E As shown, first determine the horizontal coordinate X of the rightmost lane marking line 4 After that, the corresponding points on the lane markings on both sides of any lane can be predicted accordingly. For example, for any lane i from right to left, the horizontal coordinates of the corresponding points on the lane markings on both sides of the i-th lane can be expressed as:

[0045] X i,右 =X 4 -(i-1)*d

[0046] X i,左 =X 4 -i*d.

[0047] It is to be noted that the linear prediction model X = A · θ used here can also vary with the road segment since the course of the driving road is not constant, and the model parameters adapted to each road segment can be stored in the memory of the vehicle itself or obtained online. According to an embodiment of the present application, when the lane marking determination module 400 invokes the prediction model PM, the current driving road segment RS i information can be obtained locally or from a remote server or cloud online, so as to obtain the corresponding linear model parameters θ i of the current driving road segment RS i , so as to utilize the matched model X = A · θ i to process the auxiliary marking line data to accurately predict the actual lane marking of the current driving road segment RS i .

[0048] In the above embodiment, the linear model X = A · θ is taken as an example to describe the use of the sampling points on the auxiliary marking line to realize the prediction of the actual lane marking, but the present application is not limited to the linear model, and a neural network model can also be used, which is trained by learning the relationship between the environmental reference and the lane position or marking. Preferably, the model is a recurrent neural network RNN model. When the neural model is used, the data processed by clustering can be directly input to the RNN model, or the sampling point data of the auxiliary marking line obtained by fitting processing can be input to the RNN model to generate the position output of the actual lane line. According to an embodiment of the present application, the RNN model can be trained to output the marking positions of multiple lanes.

[0049] In another embodiment, in order to better distinguish and utilize different environmental references, the difference in the signal pattern returned by each reference can be used for prediction. For example, the environmental reference is the surrounding trees instead of the guardrails, and this information provides valuable reference for determining the lane marking. Therefore, in addition to the physical position information of the environmental reference, the signal pattern features of the environmental reference, such as the radar image features of the reference, can also be generated by the feature determination unit 204 when predicting the lane marking. Thus, the lane marking determination module 400 can input the signal pattern features of the environmental reference combined with the position information to the trained neural network model RNN, thereby realizing the prediction of the lane marking.

[0050] Figure 4A block diagram of a system for training a prediction model PM according to the present invention is shown. Training system 500 includes a high-precision map database 502, a cloud data storage device 504, and a training module 506. Map database 502 stores a large amount of M traffic roads, such as those in one or more cities, and detailed map information for each lane, including the number of lanes on the road, the width of each lane, and lane markings, particularly the location of lane markings for the outermost and innermost lanes.

[0051] Cloud data storage device 504 records the characteristic data of environmental reference objects along the M roads. This characteristic data is also obtained by collecting and processing the environmental reference objects along the roads using transceiver unit 202 and characteristic determination unit 204 shown in FIG2 . It should be noted that the characteristic data stored in cloud data storage device 504 is characteristic data P′ of static reference objects after velocity filtering to eliminate noise data with velocity characteristics of moving objects.

[0052] According to one embodiment, the training module 506 can be configured to train the trainer according to the different road segments RS i To train the matching prediction model PM. Therefore, for the selected driving section RS i , read the data related to the driving section RS from the cloud data storage device 504 i The training module 506 then uses a clustering algorithm to perform clustering processing on the feature data P' of the static reference object to obtain multiple classes and the center position data of each class. For example, the feature data P of the left environmental reference object on the current driving section can be clustered. L ′ is clustered into m classes, i.e., m feature data subsets. Then, using the feature data in each feature data subset, the center point of the subset is determined, so that the data subset of each class is clustered into a point representing the class. For ease of explanation, p1, p2, ... p m Represents the feature dataset P L The positions of the m center points, and p i The position of (x i ,y i ) represents i=1,2,…m. Similarly, the feature dataset P of the reference object on the right is R Cluster into n classes, i.e. n feature data subsets. Then, use the feature data in each feature data subset to determine the center point of the subset. Thus, the data subset of each class is clustered into a point representing the class. For ease of explanation, we use p1′, p2′, … p n ′ represents the feature dataset P R The positions of n central points in the i The position of ′ is given by (x i ′,y i ′) indicates that i=1,2,…n.

[0053] For the left reference feature dataset P L The training module 506 trains the center positions p1, p2, ... p of m classes. m Line fitting is performed to obtain an auxiliary marking line, which can well fit the variation pattern of these position data. According to one embodiment, a standard second-order function is used to fit the feature data set P L The standard function is expressed as: y = k2*x 2 +k1*x+k 0。 Therefore, through P L The data in {(x1,y2),

[0054] (x2,y2)…(x m ,y m )}, by training the standard function to determine the parameters k2, k1, k0, the RS of the road section can be obtained. i Auxiliary marking line L of the change law of upper environmental reference L It is not difficult to understand that for a relatively straight lane or guardrail, over a predetermined distance, the auxiliary marking line is a linear function, ie, y=k1*x+k0.

[0055] Similarly, using the standard function y = k2'*x 2 +k1′*x+k0′ to the feature dataset P R The midpoint matrix is ​​fitted by P R {(x1′,y1′),(x2′,y2′)…(x n ′,y n ′)} is trained by the standard function to determine the parameters k2′, k1′, k0′, thereby obtaining the data reflecting the road section RS i Auxiliary marking line L on the upper right side of the environmental reference object change law R . Figure 5 The auxiliary marker line L is obtained by fitting the characteristic data in the radar cloud data storage device 504. L With L R , as shown by the solid lines on both sides of the figure. Figure 5 Label

[0056] The dotted lines shown by ①②③④ are actual lane marking lines obtained from the map database 502.

[0057] The training module 506 then respectively L 、L R Upsample N pairs of relative points, where each pair of relative points is represented as [(y i ,xi ),(y i ,x i ′)], that is, each pair of relative points has the same longitudinal coordinate. It should be noted that the N pairs of relative points selected here can be for the road section RS i The auxiliary marker line L obtained by fitting R 、L L N points are randomly sampled on the .

[0058] Then, the training module 506 extracts actual lane lines, such as the leftmost lane line No. ①, from the map database 502, and the points that have the same vertical coordinates as the N pairs of relative points. At the same time, the training module 506 also determines the actual lane width d0 and the number of lanes q0 of the current road from the map data in the map database 502. Here, it is assumed that the coordinates of the points on the leftmost lane line No. ① are (y i ,X 1 i ), then at each relative point [(y i ,x i ),(y i ,x i ′)], the leftmost lane marking point (y i ,X 1 i ) and auxiliary line L L Point (y i ,x i ) and the distance d1 between the rightmost lane marking point (y i ,X 4 i ) and auxiliary line L R Point (y i ,x i The distance d2 between the two can be determined as follows:

[0059]

[0060]

[0061] In another embodiment of the present invention, the coordinates of the points on the leftmost lane marker line (y i ,X 1 i ) and the coordinates of each point on the rightmost lane marking line ④ are (y i ,X 4 i ) to determine d1 and d2:

[0062]

[0063] Thus, the training module 506 determines the linear coefficients θ in the linear model X = A · θ as follows:

[0064]

[0065] Similarly, the training module 506 can generate training samples for the training of the RNN neural network based on the map database 502 and the radar cloud data in the cloud data storage 504. Specifically, based on the cloud data storage 504, the training module 506 can obtain the data features of the environmental references, denoted as x. Still taking the example of the left guardrail and the right guardrail, the features include the position coordinates and / or the signal pattern features of the left guardrail, and the position coordinates and / or the signal pattern features of the right guardrail. As mentioned above, in a preferred example, the feature data stored in the cloud data storage 504 is data that has been processed by velocity filtering. Figure 1

[0066] Meanwhile, based on the map database 502, the training module 506 can obtain the lane feature data at the same horizontal position as each point in the static references in the sample features, denoted as y, which represents the label of each sample. The label y can include the number of lanes on the road in the section, the physical position data of the lane markers including the innermost lane marker and the outermost lane marker. In another simplified example, the label y can only include the physical position of the outermost lane marker.

[0067] Thus, based on the cloud data storage 504 and the map database 502, the training module 506 can generate a large number of samples corresponding to different roads and different sections RS on the same road, forming a sample set {x, y}. Subsequently, the training module 506 trains an RNN neural network using the sample set {x, y} to obtain an RNN neural network prediction model PM. In another embodiment of the present application, the training module 506 can only use a part of the sample set {x, y} to train the RNN neural network model PM, and use another part of the sample set to validate the trained RNN prediction model.

[0068] Figure 6 ​A lane marking prediction flowchart according to one embodiment of the present application is shown. In step 602, a feature data of environmental references located in the periphery of a current driving road is acquired by a marking determination device, wherein the environmental references are references within a predetermined spatial range around a vehicle driving on the driving road, and the feature data includes position information of the environmental references. In step 604, the feature data is processed to obtain an auxiliary marking line of the current driving road, which represents a virtual line formed by the environmental references and can be used to locate the position of lane marking lines. The auxiliary marking line is obtained by line fitting of feature points of the environmental references. In step 606, a plurality of sampling points are extracted from the auxiliary marking line, and position feature data of the sampling points are input to a prediction model to predict lane markings on the current driving road.

[0069] Figure 7 A lane marking prediction flowchart according to one preferred embodiment is shown. As shown, in step 702, a probe signal is emitted at a predetermined spatial radiation angle within a predetermined range to probe environmental references within the predetermined range, while receiving response signals from the environmental references, wherein the environmental references are references within a predetermined spatial range around a vehicle driving on the driving road.

[0070] In step 704, after receiving the response signals, the response signals are analyzed by a predetermined algorithm to determine features of the references, including signal patterns of the environmental references and relative distances of the references from the current driving vehicle.

[0071] In step 706, based on current geographic position data (e.g. GPS coordinates) of the vehicle and the relative distances of the references determined in step 704, physical position data of the reflection points of the references are determined.

[0072] In step 708, the physical position data of the reflection points of the references are preprocessed, including performing velocity filtering on the physical position data to eliminate data from moving objects to obtain position data of static references. Then, a clustering process is performed on the feature data of the static references to obtain center position data of each class of a plurality of classes; and finally, line fitting is performed on the center position data of the plurality of classes to obtain an auxiliary marking line reflecting positions of the environmental references.

[0073] In step 710, a plurality of point sampling data sets on the auxiliary marking line are input to a trained prediction model PM to predict the position of the innermost or outermost lane marking on the current driving road; and thus, based on the position of the innermost or outermost lane marking and the lane width, the positions of lane markings of other lanes on the current driving road can be calculated. In one embodiment of the present application, the trained prediction model PM is selected based on a road section RS where the current geographic position of the vehicle is located.

[0074] In one example, the multi-point sampled data set can be input to a trained linear model to predict the location of the lane marker. In another embodiment of the present application, the multi-point sampled data set and the signal pattern features of the reference objects can be further input to a trained RNN neural network prediction model to predict the location of the lane marker.

[0075] In one application of the present application, the lane marker determination apparatus 100 can be applied in an autonomous driving system, in which a vehicle control unit in the autonomous driving system selects a corresponding driving lane based on the lane marker information provided by the lane marker determination apparatus 100 in combination with the current driving conditions. Here, the driving conditions include traffic information, designed route, etc.

[0076] It is to be noted that the present application is not limited to the above-mentioned preferred embodiments. For example, Figure 8 The lane marker determination apparatus 100 according to another embodiment of the present application is shown, which comprises a transceiver 802, a control unit 804 and a memory 806. The memory 806 stores programs or instructions for implementing the lane prediction process of the present application, and the control unit 804 executes the programs or instructions to control the transceiver 802 to transmit radar detection signals to a predetermined range and receive reflected signals, and then the control unit 804 analyzes the reflected signals to perform lane prediction. The lane prediction process comprises obtaining feature data of environmental reference objects located in the periphery of the current driving road, wherein the environmental reference objects are reference objects within a predetermined spatial range around the vehicle driving on the driving road, and the features include position information of the environmental reference objects; processing the features to obtain auxiliary marker lines of the current driving road; and inputting multi-point sampled data of the auxiliary marker lines to a prediction model to predict lane markers on the current driving road.

[0077] In addition, it is not difficult to understand that Figure 2A , 2B , the modules and units in the above-mentioned embodiments can include processors, electronic devices, hardware devices, electronic components, logic circuits, memories, software codes, firmware codes, etc., or any combination thereof. Those skilled in the art will also recognize that the various illustrative logical blocks, modules and method steps described in connection with the present disclosure can be implemented as electronic hardware, computer software or a combination of the two. As an example, in the case of software implementation, as a logical device, the corresponding computer program instructions in the non-volatile memory are read into the memory by the processor and executed to form.

[0078] Another embodiment of the present application provides a machine readable medium having stored thereon machine readable instructions which, when executed by a computer, cause the computer to perform any of the aforementioned methods disclosed herein. Specifically, a system or apparatus equipped with a machine readable medium having stored thereon software program codes implementing the functions of any of the aforementioned embodiments can be provided, and the computer of the system is caused to read out and execute the machine readable instructions stored in the machine readable medium. In this case, the program codes read out from the machine readable medium itself implement the functions of any of the aforementioned embodiments, and thus the machine readable codes and the machine readable medium storing the machine readable codes constitute a part of the present application.

[0079] The present application has been described in detail by the above drawings and preferred embodiments, however the present application is not limited to the disclosed embodiments, and as those skilled in the art can know, more embodiments of the present application can be obtained by combining the code auditing means in the different embodiments described above, and these embodiments are also within the protection scope of the present application.

Claims

1. A method of determining lane markings, comprising: obtaining feature data of environmental references located in a periphery of a current driving road, wherein the environmental references are references within a predetermined spatial range around a vehicle driving on the driving road, the feature data comprising position information of the environmental references; processing the feature data to obtain an auxiliary marking line corresponding to at least a segment of the current driving road, wherein the auxiliary marking line is a virtual reference line derived from the environmental references; inputting a plurality of sampled position data of the auxiliary marking line into a prediction model to predict lane markings of at least a segment of the current driving road.

2. The method of claim 1, wherein, the auxiliary marking line is located on a left side or a right side of the current driving road, wherein predicting the lane markings comprises: inputting a plurality of sampled position data of the auxiliary marking line into the prediction model to predict a most inner or most outer lane marking position on the driving road; calculating lane marking positions of other lanes on the current driving road based on the most inner or most outer lane marking position and a lane width. 3.The method of claim 2, wherein the prediction model depends on a segment geographical location of the driving road, the method further comprising: selecting the prediction model matching the segment geographical location. 4.The method of claim 3, wherein the prediction model is a trained recurrent neural network (RNN) model or a linear model.

5. The method according to any one of claims 1 to 4, wherein processing the feature data to obtain the auxiliary marking line corresponding to at least a segment of the current driving road further comprises: performing velocity filtering on the feature data to eliminate feature data from moving objects in the obtained feature data to obtain feature data of static references. 6.The method of claim 5, further comprising: performing clustering processing on the feature data of the static references to obtain center position data of each of a plurality of clusters; performing line fitting on the center position data of the plurality of clusters to obtain the auxiliary marking line. 7.The method of any one of claims 1-4, wherein obtaining the feature data of the environmental references located in the periphery of the driving road comprises: sending a probe signal by a vehicle-mounted radar and receiving a response signal from the environmental references, wherein the vehicle-mounted radar emits the probe signal to the predetermined spatial range with a predetermined spatial radiation angle; based on the response signal, obtaining the feature data of the environmental references, comprising: determining a relative distance between the vehicle and the environmental references; predicting the position information of the environmental references based on geographical position data of the vehicle and the relative distance. 8.A lane marking determination apparatus, comprising: a reference obtaining module configured to obtain feature data of environmental references located in a periphery of a current driving road, wherein the environmental references are references within a predetermined spatial range around a vehicle driving on the driving road, the feature data comprising position information of the environmental references; an auxiliary marker determination module configured to process the feature data to obtain an auxiliary marker line corresponding to at least a segment of the current driving road, wherein the auxiliary marker line is a virtual reference line derived from the environmental reference; and a lane marker determination module configured to input a plurality of sampled position data of the auxiliary marker line into a prediction model to predict lane markers of at least a segment of the current driving road.

9. The apparatus of claim 8, wherein, the auxiliary marker line is located on the left or right side of the current driving road, wherein the lane marker determination module is further configured to: input a plurality of sampled position data of the auxiliary marker line into the prediction model to predict a position of a most inner or outer lane marker on the driving road; calculate positions of other lane markers on the current driving road based on the position of the most inner or outer lane marker and a lane width.

10. The apparatus of claim 9, wherein the prediction model depends on a segment geographical location of the driving road; wherein the lane marker determination module is further configured to: select the prediction model based on the segment geographical location.

11. The apparatus of claim 10, wherein, the prediction model is a trained recurrent neural network (RNN) model or a linear model.

12. The apparatus of one of claims 8-11, wherein, the auxiliary marker determination module further comprises: a filtering module configured to perform velocity filtering on the feature data to eliminate feature data from moving objects in the acquired feature data to obtain feature data of static reference objects.

13. The apparatus of claim 12, wherein, the auxiliary marker determination module further comprises: a clustering module configured to perform clustering on the feature data of the static reference objects to obtain a center position data of each of a plurality of clusters; and a fitting module configured to perform line fitting on the center position data of the plurality of clusters to obtain the auxiliary marker line.

14. The apparatus of any one of claims 8-13, wherein the reference object acquisition module further comprises: a transceiver unit configured to transmit a probe signal through a vehicle-mounted radar and receive a response signal from the environmental reference object, wherein the vehicle-mounted radar transmits the probe signal to the predetermined spatial range with a predetermined spatial radiation angle; a feature determination unit configured to: determine a relative distance between the vehicle and the environmental reference object, and predict the position information of the environmental reference object based on geographical position data of the vehicle and the relative distance.

15. An autonomous driving system, comprising: the lane marker determination apparatus of any one of claims 8-14; a vehicle-mounted control unit configured to select a current driving lane based on lane markers determined by the lane marker determination apparatus.

16. A lane marker determination apparatus, comprising: a memory storing instructions; a controller configured to implement the method of any one of claims 1-7 by executing the instructions.

17. A machine-readable medium having stored thereon machine-readable instructions, wherein the instructions, when executed by the machine, cause the machine to perform the method of any one of claims 1-7. ​

Images