Road surface information display method and device, storage medium, electronic equipment and vehicle

Abstract

The invention discloses a road surface information display method and device, a storage medium, electronic equipment and a vehicle. The method comprises the steps that a road image collected based on a target vehicle and a road surface information sensing result obtained based on a road image sensing model are acquired; dividing the plurality of lane lines into a left lane line array and a right lane line array according to the relative position of each lane line and the target reference object; matching the target distance corresponding to each lane line in the left lane line array with each left reference distance range, adding a corresponding target mark to the successfully matched lane line, and matching the target distance corresponding to each lane line in the right lane line array with each right reference distance range, adding a corresponding target mark to the successfully matched lane line; and outputting and displaying target road surface information, wherein the target road surface information comprises the marked left side lane line array and the marked right side lane line array.

Description

Technical Field

[0001] The present application relates to the field of intelligent driving technology, and more specifically, to a road information display method, device, storage medium, electronic device and vehicle. Background Art

[0002] With the development of intelligent driving technology, major OEMs are paying more and more attention to how to display various types of information obtained by intelligent driving vehicles to drivers. For example, during the driving process, the road surface information around the vehicle will be displayed on the vehicle's instrument or large screen, including lane lines, road edge lines, stop lines, etc. However, this road surface information generally comes from map information. When the map information is not updated in time or the map information is missing, it will cause the road surface information to be displayed incorrectly or missing, which can easily cause the vehicle to cross the line, or because the lane line information ahead seen by the driver's naked eye is different from the lane line information displayed by the vehicle, it will bring unnecessary pressure to the driver's driving, thereby exacerbating the occurrence of traffic accidents. Summary of the invention

[0003] The present application provides a road surface information display method, device, storage medium, electronic device and vehicle, which can solve the problem of traffic accidents being aggravated due to errors or omissions in map information relied upon for displaying lane lines.

[0004] The specific technical solutions are as follows:

[0005] In a first aspect, an embodiment of the present application provides a method for displaying road surface information, the method comprising:

[0006] Acquire a road surface information perception result obtained based on a road image collected by the target vehicle and a road image perception model, wherein the road surface information perception result includes a plurality of lane lines and a plurality of road edge lines;

[0007] According to the relative position of each lane line and the target reference object, the multiple lane lines are divided into a left lane line array and a right lane line array, wherein when the road surface information perception result includes a reference lane center line, the target reference object is the reference lane center line, and when the road surface information perception result does not include the reference lane center line, the target reference object is the target vehicle, and the reference lane center line is a lane center line that meets the reference center line requirements;

[0008] Matching the target distance corresponding to each lane line in the left lane line array with each left reference distance range respectively, and adding a corresponding target mark to the successfully matched lane line, and matching the target distance corresponding to each lane line in the right lane line array with each right reference distance range respectively, and adding a corresponding target mark to the successfully matched lane line, wherein the target distance corresponding to each lane line is the distance between the lane line and the target reference object, the left reference distance range and the right reference distance range are ranges determined based on the distance between each lane line that meets the standard and the target reference object, and the target mark can indicate the relative positional relationship between the marked lane line and the target reference object and the positional relationship between the marked lane lines;

[0009] The target road surface information is output and displayed, wherein the target road surface information includes the marked left lane line array and the marked right lane line array.

[0010] It can be seen from the above scheme that the embodiment of the present application first senses multiple lane lines on the road surface through the road image perception model, and then uses the center line of the reference lane or the target vehicle as the target reference object, and based on the relative position of the target reference object and each lane line, the lane lines are grouped on the left and right sides, and the target distance corresponding to each lane line in the left lane line array is matched with each left reference distance range, and the target distance corresponding to each lane line in the right lane line array is matched with each right reference distance range, and the corresponding target mark is added to the successfully matched lane line, so as to obtain the left lane line array and the right lane line array for output. It can be seen from this that when determining the lane line for output display, the embodiment of the present application only relies on the road image collected by the target vehicle, and no longer relies on the map information, so that whether the map information is wrong or missing, it does not affect the accuracy of the lane line determination of the present application, and thus can reduce the probability of traffic accidents caused by lane line display errors or display omissions.

[0011] In a possible implementation, before dividing the plurality of lane lines into a left lane line array and a right lane line array according to the relative position of each lane line and the target reference object, the method further includes:

[0012] Selecting a lane line that meets the lane line requirement from the multiple lane lines, wherein the lane line requirement includes at least one of the following: whether the length of the lane line is greater than or equal to a first length threshold, the number of discrete points included in the lane line is greater than or equal to a first number threshold, and the direction of the lane line is consistent with the driving direction of the target vehicle;

[0013] The method of dividing the plurality of lane lines into a left lane line array and a right lane line array according to the relative position of each lane line and the target reference object comprises:

[0014] According to the relative position between each of the selected lane lines and the target reference object, the selected lane lines are divided into the left lane line array and the right lane line array.

[0015] It can be seen from the above scheme that before grouping multiple lane lines, the embodiment of the present application first filters out invalid lane lines or lane lines with poor display effects according to lane line requirements, and then groups lane lines that meet the lane line requirements, which can improve the accuracy and visual effect of subsequent output display of lane lines.

[0016] In a possible implementation, when the target reference object is the center line of the reference lane, the method for calculating the target distance includes:

[0017] Determine a first discrete point set for each lane line according to a ordinate range from a starting discrete point to an ending discrete point on the center line of the reference lane, wherein the first discrete point set includes all first discrete points on the lane line that satisfy the ordinate range;

[0018] Searching for the second discrete point closest to each of the first discrete points in the first discrete point set of each of the lane lines on the center line of the reference lane, and calculating the distance between each of the first discrete points and its corresponding second discrete point as the discrete point distance;

[0019] For each lane line, the target distance between the lane line and the center line of the reference lane is determined according to the distances of the discrete points of the lane line.

[0020] It can be seen from the above scheme that compared with directly selecting a discrete point from the lane line to draw a perpendicular line to the center line of the reference lane to determine the target distance between the lane line and the center line of the reference lane, the embodiment of the present application first uses the vertical coordinate range of the center line of the reference lane to intercept the first discrete point within the vertical coordinate range from each lane line, thereby avoiding redundant discrete points outside the vertical coordinate range from interfering with the calculation accuracy of the target distance, and then comprehensively determines the target distance between each lane line and the center line of the reference lane based on the distance between each first discrete point and the second discrete point closest to the center line of the reference lane, which can improve the accuracy of the target distance.

[0021] In a possible implementation, the reference centerline requirement includes at least one of the following:

[0022] The length of the lane center line is greater than or equal to a second length threshold, the number of discrete points included on the lane center line is greater than or equal to a second quantity threshold, the distance from the lane center line to the target vehicle is less than or equal to a first distance threshold, and the orientation of the lane center line is consistent with the driving direction of the target vehicle.

[0023] In a possible implementation manner, the output of the display of the target road surface information includes:

[0024] From the start of the function of displaying the target road surface information, when the lane lines can be sensed continuously for a first duration, the target road surface information sensed in the current period is output and displayed;

[0025] When it first appears that the lane lines can no longer be sensed after being sensed, the first road surface information is output and displayed until the lane lines cannot be sensed continuously for a second duration, at which time the function of displaying the target road surface information is paused, or when the lane lines are sensed again within the second duration, the target road surface information sensed in the current period is output and displayed. Herein, the first road surface information includes the target road surface information determined when the lane lines could be sensed last time.

[0026] It can be seen from the above solution that after the function of displaying the target road surface information is started in the embodiment of the present application, only when the lane lines can be stably sensed continuously for a first duration, the target road surface information sensed in the current period is output. When the lane lines suddenly cannot be sensed subsequently, the display screen does not disappear immediately with the disappearance of the sensing, but first outputs and displays the target road surface information determined when the lane lines could be sensed last time until the lane lines cannot be sensed continuously for a second duration, at which time the function of displaying the target road surface information is paused, or when the lane lines are sensed again within the second duration, the target road surface information sensed in real time is output and displayed. By means of this way of late display and late disappearance, the embodiment of the present application eliminates problems such as jitter and flicker caused by unclear road images, blocked lane lines, etc., thereby further improving driving safety.

[0027] In a possible implementation manner, the output of the display of the target road surface information includes:

[0028] When the cumulative duration of the output and display of the first road surface information has not reached the second duration and it is detected that the driving state of the target vehicle meets a first driving state, in the case where the lane lines are sensed again, the target road surface information sensed in the current period is output and displayed, and in the case where the lane lines are not sensed again, the function of displaying the target road surface information is paused, where the first driving state includes that the steering wheel angle of the target vehicle is greater than or equal to a first angle threshold, and / or the cornering rate of the target vehicle is greater than or equal to a first cornering rate threshold.

[0029] It can be seen from the above scheme that when the lane line is not sensed, the embodiment of the present application first keeps displaying the previous lane line. When it is detected that the target vehicle wants to change lanes or turn, and the duration has not reached the second duration, if the lane line is sensed again, the currently detected lane line is directly displayed. If the lane line is not sensed again, no information is displayed, so as to avoid the phenomenon that the difference between the previously displayed lane line and the re-sensed lane line is too large, resulting in a sudden change.

[0030] In a possible implementation, the outputting and displaying target road surface information includes:

[0031] When it is detected that the speed of the target vehicle is 0, the second road surface information is output and displayed, until the speed of the target vehicle is not 0, the target road surface information perceived in the current cycle is output and displayed, wherein the second road surface information includes the target road surface information perceived most recently before the speed of the target vehicle becomes 0.

[0032] It can be seen from the above scheme that when the target vehicle is stationary, the second road surface information is output and displayed, and the real-time data sensed is no longer updated. Only when the target vehicle starts to move, the target road surface information sensed in the current cycle is re-output and displayed, thereby avoiding the difference in the target road surface information output when the target vehicle is stationary due to the difference in the collected road images, and further avoiding the visual jitter phenomenon caused by the display difference.

[0033] In a possible implementation, the outputting and displaying target road surface information includes:

[0034] When it is first detected that the target vehicle is in a low-speed congestion state, and the driving state of the target vehicle does not satisfy the second driving state, the third road surface information is output and displayed until the target vehicle is in a non-low-speed congestion state, or is in the low-speed congestion state and satisfies the second driving state, the target road surface information perceived in the current cycle is output and displayed, wherein the second driving state includes the steering wheel angle of the target vehicle being greater than or equal to a second angle threshold, and / or the turning angle rate of the target vehicle being greater than or equal to a second turning angle rate threshold, and the third road surface information includes the target road surface information most recently perceived before the target vehicle enters the low-speed congestion state.

[0035] It can be seen from the above scheme that when the target vehicle is identified as being in a low-speed congestion scene, the real-time perceived lane line will be missing due to line of sight obstruction. At this time, the target road surface information most recently perceived before the target vehicle enters the low-speed congestion state can be displayed. The target road surface information perceived in the current cycle will not be displayed to the driver until the target vehicle leaves the low-speed congestion scene, or wants to change lanes or turn. This ensures that the target road surface information can still be displayed stably and completely in the low-speed congestion scene.

[0036] In a possible implementation, the outputting and displaying target road surface information includes:

[0037] When the angle between any lane line in the marked left lane line array and the marked right lane line array and the target vehicle is greater than or equal to the first angle threshold, or when it is detected that the steering wheel angle of the target vehicle is greater than or equal to the third angle threshold, the function of displaying the target road surface information is suspended until the angles between all lane lines in the marked left lane line array and the marked right lane line array and the target vehicle are less than the first angle threshold, and the steering wheel angle is less than the third angle threshold, then the target road surface information perceived in the current cycle is output and displayed.

[0038] It can be seen from the above scheme that when the target vehicle is in a U-turn scenario, that is, when the angle between any lane line in the marked left lane line array and the marked right lane line array and the target vehicle is greater than or equal to the first angle threshold, or when the steering wheel angle of the target vehicle is detected to be greater than or equal to the third angle threshold, the visual impact problem caused by the large change in the lane line during the U-turn period can be solved by pausing the display of the target road surface information, thereby further optimizing the display effect in the U-turn scenario.

[0039] In a possible implementation, the outputting and displaying target road surface information includes:

[0040] When there is an unknown lane line type in the marked left lane line array and the marked right lane line array determined in the current cycle, the lane line type of the same lane line detected in the previous cycle is replaced with an unknown lane line type.

[0041] From the above scheme, it can be seen that by replacing the lane line type of the same lane line detected in the previous cycle with an unknown lane line type, not only can the problem of unknown lane line type being unable to be displayed be solved, but the lane line type can also be ensured to a certain extent.

[0042] In a possible implementation, the road surface information perception result further includes a plurality of road edge lines. Before outputting and displaying the target road surface information, the method further includes:

[0043] Along the left and right sides of the target reference object, select a left road edge line and a right road edge line that are closest to the target reference object from the multiple road edge lines;

[0044] The target road surface information also includes: the left road edge line and the right road edge line.

[0045] In a possible implementation, the method further includes:

[0046] When the angle between the left road edge line and the lane line closest to the target reference object in the marked left lane line array is greater than or equal to a second angle threshold, canceling the display of the left road edge line; and / or,

[0047] When the angle between the right road edge line and the lane line closest to the target reference object in the marked right lane line array is greater than or equal to the second angle threshold, canceling the display of the right road edge line; and / or,

[0048] When the angle between the left road edge line and the driving direction of the target vehicle is greater than or equal to a third angle threshold, canceling the display of the left road edge line; and / or,

[0049] When the angle between the right road edge line and the driving direction of the target vehicle is greater than or equal to the third angle threshold, canceling the display of the right road edge line; and / or,

[0050] When the distance between the left road edge line and any lane line in the marked left lane line array is less than or equal to a second distance threshold, canceling the display of the left road edge line; and / or,

[0051] When the distance between the right road edge line and any lane line in the marked right lane line array is less than or equal to the second distance threshold, the right road edge line is cancelled from being displayed.

[0052] It can be seen from the above scheme that the embodiment of the present application can avoid displaying incorrectly detected road edge lines by canceling the display of road edge lines that have too large an angle with the lane line, and can prevent sudden changes and flickering of road edge lines by canceling the display of road edge lines that have too large an angle with the target vehicle's driving direction. By canceling the display of road edge lines that are too close to the lane line, it can prevent the road edge line from overlapping with the lane line display.

[0053] In a second aspect, an embodiment of the present application provides a road surface information display device, the device comprising:

[0054] An acquisition unit, used to acquire a road surface information perception result obtained based on a road image collected by a target vehicle and a road image perception model, wherein the road surface information perception result includes a plurality of lane lines and a plurality of road edge lines;

[0055] a dividing unit, for dividing the plurality of lane lines into a left lane line array and a right lane line array according to a relative position between each lane line and a target reference object, wherein when the road surface information perception result includes a reference lane center line, the target reference object is the reference lane center line, and when the road surface information perception result does not include the reference lane center line, the target reference object is the target vehicle, and the reference lane center line is a lane center line that meets the reference center line requirements;

[0056] a matching marking unit, for matching the target distance corresponding to each lane line in the left lane line array with each left reference distance range, and adding a corresponding target mark to the successfully matched lane line, and matching the target distance corresponding to each lane line in the right lane line array with each right reference distance range, and adding a corresponding target mark to the successfully matched lane line, wherein the target distance corresponding to each lane line is the distance between the lane line and the target reference object, the left reference distance range and the right reference distance range are ranges determined based on the distance between each lane line that meets the standard and the target reference object, and the target mark can indicate the relative positional relationship between the marked lane line and the target reference object and the positional relationship between the marked lane lines;

[0057] An output display unit is used to output and display target road surface information, wherein the target road surface information includes the marked left lane line array and the marked right lane line array.

[0058] In a possible implementation manner, the device further includes:

[0059] A first selection unit is used to select a lane line that meets a lane line requirement from the multiple lane lines before dividing the multiple lane lines into a left lane line array and a right lane line array according to a relative position of each lane line and a target reference object, wherein the lane line requirement includes at least one of the following: whether the length of the lane line is greater than or equal to a first length threshold, the number of discrete points included in the lane line is greater than or equal to a first number threshold, and the direction of the lane line is consistent with the driving direction of the target vehicle;

[0060] The division unit is used to divide the selected lane lines into the left lane line array and the right lane line array according to the relative position of each selected lane line and the target reference object.

[0061] In a possible implementation manner, the device further includes:

[0062] a determination unit, configured to determine, when the target reference object is the center line of the reference lane, a first discrete point set for each lane line according to a ordinate range from a starting discrete point to an ending discrete point on the center line of the reference lane, wherein the first discrete point set includes all first discrete points on the lane line that satisfy the ordinate range;

[0063] A searching unit, configured to search on the center line of the reference lane respectively a second discrete point which is closest to each of the first discrete points in the set of the first discrete points of each of the lane lines;

[0064] a calculating unit, configured to calculate the distance between each of the first discrete points and its corresponding second discrete point as a discrete point distance;

[0065] The determination unit is used to determine, for each lane line, the target distance between the lane line and the center line of the reference lane according to the distances of the discrete points of the lane line.

[0066] In a possible implementation, the reference centerline requirement includes at least one of the following:

[0067] The length of the lane centerline is greater than or equal to a second length threshold, the number of discrete points contained in the lane centerline is greater than or equal to a second number threshold, the distance from the lane centerline to the target vehicle is less than or equal to a first distance threshold, and the direction of the lane centerline is consistent with the driving direction of the target vehicle.

[0068] In a possible implementation, the output display unit includes:

[0069] A first display module is used to output and display the target road surface information sensed in the current period when the lane line can be sensed for a first continuous period of time after the function of displaying the target road surface information is started;

[0070] The second display module is used to output and display the first road surface information when the lane line is perceived to be not perceived for the first time, and to suspend the display of the target road surface information when the lane line is not perceived for a second consecutive period of time, or to output and display the target road surface information perceived in the current period when the lane line is perceived again within the second period of time, wherein the first road surface information includes the target road surface information determined when the lane line was most recently perceived.

[0071] In a possible implementation, the output display unit further includes:

[0072] The third display module is used for outputting and displaying the target road surface information perceived in the current cycle when the lane line is perceived again, and pausing the display of the target road surface information when the cumulative time for outputting and displaying the first road surface information does not reach the second time period and it is detected that the driving state of the target vehicle satisfies the first driving state, wherein the first driving state includes the steering wheel angle of the target vehicle being greater than or equal to the first angle threshold, and / or the turning angle rate of the target vehicle being greater than or equal to the first turning angle rate threshold.

[0073] In a possible implementation, the output display unit includes:

[0074] The fourth display module is used to output and display the second road surface information when it is detected that the speed of the target vehicle is 0, and to output and display the target road surface information perceived in the current cycle when the speed of the target vehicle is not 0, wherein the second road surface information includes the target road surface information perceived most recently before the speed of the target vehicle becomes 0.

[0075] In a possible implementation, the output display unit includes:

[0076] The fifth display module is used to output and display the third road surface information when it is first detected that the target vehicle is in a low-speed congestion state and the driving state of the target vehicle does not satisfy the second driving state, until the target vehicle is in a non-low-speed congestion state, or is in the low-speed congestion state and satisfies the second driving state, output and display the target road surface information perceived in the current cycle, wherein the second driving state includes the steering wheel angle of the target vehicle being greater than or equal to a second angle threshold, and / or the turning angle rate of the target vehicle being greater than or equal to a second turning angle rate threshold, and the third road surface information includes the target road surface information most recently perceived before the target vehicle enters the low-speed congestion state.

[0077] In a possible implementation, the output display unit includes:

[0078] A sixth display module, configured to pause the function of displaying the target road surface information when any lane line in the marked left lane line array and the marked right lane line array has an angle greater than or equal to a first angle threshold with the target vehicle, or when it is detected that the steering wheel angle of the target vehicle is greater than or equal to a third angle threshold, and output and display the target road surface information perceived in the current cycle until the angles between all lane lines in the marked left lane line array and the marked right lane line array and the target vehicle are less than the first angle threshold, and the steering wheel angle is less than the third angle threshold.

[0079] In a possible implementation manner, the output display unit includes:

[0080] A replacement module, configured to, when there is a lane line with an unknown type in the marked left lane line array and the marked right lane line array determined in the current cycle, replace the lane line type of the same lane line detected in the previous cycle with the lane line type with unknown content.

[0081] In a possible implementation manner, the device further includes:

[0082] A second selection unit, configured to, when the road surface information perception result further includes multiple road edge lines, before outputting and displaying the target road surface information, select the left road edge line and the right road edge line closest to the target reference object from the multiple road edge lines along the left and right sides of the target reference object;

[0083] The target road surface information further includes: the left road edge line and the right road edge line.

[0084] In a possible implementation manner, the device further includes:

[0085] A cancellation unit, configured to cancel the display of the left road edge line when the angle between the left road edge line and the lane line closest to the target reference object in the marked left lane line array is greater than or equal to a second angle threshold; and / or,

[0086] when the angle between the right road edge line and the lane line closest to the target reference object in the marked right lane line array is greater than or equal to the second angle threshold, cancel the display of the right road edge line; and / or,

[0087] when the angle between the left road edge line and the driving direction of the target vehicle is greater than or equal to a third angle threshold, cancel the display of the left road edge line; and / or,

[0088] When the angle between the right side road edge line and the driving direction of the target vehicle is greater than or equal to the third angle threshold, canceling the display of the right side road edge line; and / or,

[0089] When the distance between the left road edge line and any lane line in the marked left lane line array is less than or equal to a second distance threshold, canceling the display of the left road edge line; and / or,

[0090] When the distance between the right road edge line and any lane line in the marked right lane line array is less than or equal to the second distance threshold, the right road edge line is cancelled from being displayed.

[0091] It can be seen from the above scheme that the embodiment of the present application first senses multiple lane lines on the road surface through the road image perception model, and then uses the center line of the reference lane or the target vehicle as the target reference object, and based on the relative position of the target reference object and each lane line, the lane lines are grouped on the left and right sides, and the target distance corresponding to each lane line in the left lane line array is matched with each left reference distance range, and the target distance corresponding to each lane line in the right lane line array is matched with each right reference distance range, and the corresponding target mark is added to the successfully matched lane line, so as to obtain the left lane line array and the right lane line array for output. It can be seen from this that when determining the lane line for output display, the embodiment of the present application only relies on the road image collected by the target vehicle, and no longer relies on the map information, so that whether the map information is wrong or missing, it does not affect the accuracy of the lane line determination of the present application, and thus can reduce the probability of traffic accidents caused by lane line display errors or display omissions.

[0092] In a third aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any possible implementation manner of the first aspect.

[0093] In a fourth aspect, an embodiment of the present application provides an electronic device, the electronic device comprising:

[0094] one or more processors;

[0095] The processor is coupled to a storage device, and the storage device is used to store one or more programs;

[0096] When one or more programs are executed by one or more processors, the electronic device implements the method described in any possible implementation manner of the first aspect.

[0097] In a fifth aspect, an embodiment of the present application provides a vehicle, wherein the vehicle includes a device as described in any possible implementation of the second aspect, or includes an electronic device as described in the fourth aspect.

[0098] In a sixth aspect, an embodiment of the present application provides a computer program product, which includes instructions. When the instructions are executed on a computer or a processor, the computer or the processor executes the method described in any possible implementation of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0099] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the prior art description are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without creative work.

[0100] Figure 1 A schematic diagram of a process flow of a road surface information display method provided in an embodiment of the present application;

[0101] Figure 2 An example diagram of a target distance calculation method and a lane marking method provided in an embodiment of the present application;

[0102] Figure 3 An example diagram of eliminating jitter phenomenon provided by an embodiment of the present application;

[0103] Figure 4 A block diagram of a road surface information display device provided in an embodiment of the present application;

[0104] Figure 5 A schematic diagram of the structure of an electronic device or computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0105] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0106] It should be noted that, in the absence of conflict, the embodiments in the present application and the features in the embodiments can be combined with each other. The terms "including" and "having" in the embodiments of the present application and the accompanying drawings and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device comprising a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or devices.

[0107] Figure 1 The figure is a flow chart of a method for displaying road information. The method can be applied to electronic equipment or computer equipment, and can be applied to vehicles or servers. The method can include the following steps:

[0108] S110: Obtaining a road surface information perception result obtained based on a road image collected by the target vehicle and a road image perception model.

[0109] The road surface information perception result includes multiple lane lines and may also include multiple road edge lines. When the pre-trained road image perception model has the function of perceiving lane center lines, the road surface information perception result may also include multiple lane center lines.

[0110] The road image perception model is a neural network model used to perceive road surface information in road images. Road surface information includes lane lines, stop lines, lane center lines (a virtual line commonly used in the field of autonomous driving), road edge lines, etc. In order to obtain the road image perception model, multiple road images can be collected first, and the true value labels of lane lines and road edge lines can be added to these road images. The true value label of the lane center line can also be added to form a training sample set; then the initial road image perception model is trained using the training sample set, and the lane line prediction value, road edge line prediction value, and lane center line prediction value of each sample image are output, and the loss value of this training is calculated through the lane line prediction value, road edge line prediction value, lane center line prediction value, and the corresponding lane line true value, road edge line true value, and lane center line true value. If the loss value is greater than or equal to the loss threshold, the model parameters of the initial road image perception model are adjusted, and the initial road image perception model is continued to be trained until the current loss value is less than the loss threshold, and the final road image perception model is obtained.

[0111] When it is necessary to perceive road information, the road image collected by the target vehicle in real time can be obtained, and the road image can be input into the road image perception model to obtain the road information perception result. The road information perception result includes the line attributes of each line, which is used to distinguish different types of lines. In the following steps, the premise for processing different types of lines is that the specific type of line has been identified through the line attributes, which will not be repeated here.

[0112] When the embodiment of the present application is applied to a vehicle, the target vehicle is the vehicle itself; when the embodiment of the present application is applied to a server, the target vehicle is the vehicle that applies to the server for road information to be displayed.

[0113] S120: Divide the plurality of lane lines into a left lane line array and a right lane line array according to the relative position of each lane line and the target reference object.

[0114] Among them, when the road surface information perception result includes the reference lane centerline, the target reference object is the reference lane centerline; when the road surface information perception result does not include the reference lane centerline, the target reference object is the target vehicle, and the reference lane centerline is the lane centerline that meets the reference centerline requirements.

[0115] The reference centerline requirements include at least one of the following: the length of the lane centerline is greater than or equal to the second length threshold, the number of discrete points contained in the lane centerline is greater than or equal to the second number threshold, the distance from the lane centerline to the target vehicle is less than or equal to the first distance threshold, and the direction of the lane centerline is consistent with the driving direction of the target vehicle.

[0116] When there are multiple lane centerlines that meet the above-mentioned reference centerline requirements, the lane centerline closest to the target vehicle is selected from the lane centerlines that meet the reference centerline requirements as the reference lane centerline.

[0117] By judging whether each lane line is located on the left side or the right side of the target reference object, it is determined whether the lane line is divided into the left lane line array or the right lane line array.

[0118] S130: Match the target distance corresponding to each lane line in the left lane line array with each left reference distance range respectively, and add the corresponding target mark to the successfully matched lane line; and match the target distance corresponding to each lane line in the right lane line array with each right reference distance range respectively, and add the corresponding target mark to the successfully matched lane line.

[0119] Among them, the target distance corresponding to each lane line is the distance between the lane line and the target reference object. The target markings can represent the relative position relationship between the marked lane line and the target reference object and the position relationship between the marked lane lines. For example, the left lane lines from near to far from the target reference object (the lane lines on the left side of the target reference object) can be marked as "Left 1, Left 2, Left 3..." in sequence, and the right lane lines from near to far from the target reference object (the lane lines on the right side of the target reference object) can be marked as "Right 1, Right 2, Right 3..." in sequence. The left reference distance range and the right reference distance range are ranges determined based on the distances between the lane lines that meet the standards and the target reference object. For example, the range satisfied by the distance between the Left 1 lane line and the target reference object is [0.5 * lane width - a, 0.5 * lane width + a], where a is a floating range determined according to the actual situation. For example, a = 0.25 * lane width.

[0120] When the target distance corresponding to a certain lane line in the left lane line array is within a certain left reference distance range, it indicates that the lane line successfully matches the left reference distance range. When the target distances corresponding to a certain lane line in the left lane line array are all outside all the left reference distance ranges, it indicates that the lane line fails to match all the left reference distance ranges. The right matching process is the same and will not be elaborated here.

[0121] When the target reference object is the center line of the reference lane, the methods for calculating the target distance include: determining the first discrete point set of each lane line according to the vertical coordinate range from the starting discrete point to the ending discrete point on the center line of the reference lane, where the first discrete point set includes all the first discrete points on the lane line that satisfy the vertical coordinate range; respectively finding the second discrete point closest to each first discrete point in the first discrete point set of each lane line on the center line of the reference lane, and calculating the distance between each first discrete point and its corresponding second discrete point as the discrete point distance; for each lane line, determining the target distance between the lane line and the center line of the reference lane according to the discrete point distances of the lane line.

[0122] The lines included in the road surface information perception result are often composed of discrete points at fixed intervals, rather than continuous curves or straight lines. When calculating the target distance, the coordinate system used can be the image coordinate system of the road image where the lines are located, or the vehicle body coordinate system obtained by converting the lines from the image coordinate system.

[0123] Methods for determining the target distance between a lane line and the center line of a reference lane based on the distances of the discrete points of the lane line include, but are not limited to, the following methods: (1) taking the middle value of all the discrete point distances of each lane line as the target distance between the lane line and the center line of the reference lane; (2) taking the average value of all the discrete point distances of each lane line as the target distance between the lane line and the center line of the reference lane; (3) after removing the maximum distance and the minimum distance of all the discrete point distances of each lane line, taking the average value of the remaining discrete point distances as the target distance between the lane line and the center line of the reference lane.

[0124] like Figure 2 As shown, the left lane line array includes 3 lane lines, and the right lane line array also includes 3 lane lines. The dotted line in the middle is the center line of the reference lane. When calculating the target distance between each lane line and the center line of the reference lane, the ordinate range from the starting discrete point to the ending discrete point on the center line of the reference lane can be determined first, that is, the lane line discrete point range in the figure, and the first discrete point set of each lane line is determined according to the ordinate range, and the second discrete point closest to each first discrete point in the first discrete point set of each lane line is searched on the center line of the reference lane, and the distance between each first discrete point and its corresponding second discrete point is calculated as the discrete point distance; for each lane line, the target distance between the lane line and the center line of the reference lane is determined according to the distances of the discrete points of the lane line. Then, the distance 1, distance 2, and distance 3 corresponding to each lane line in the left lane line array are matched with each left reference distance range respectively, and the corresponding target mark is added to the successfully matched lane line. The target distance corresponding to each lane line in the right lane line array is matched with each right reference distance range respectively, and the corresponding target mark is added to the successfully matched lane line. Finally, the successfully matched lane lines include left 1, left 2, left 3, right 1, and right 3.

[0125] When the target reference object is a target vehicle, the method for calculating the target distance includes: calculating the distance from a reference point on the target vehicle to each lane line as the target distance corresponding to the lane line. The reference point can be the center point of the rear axle or the center of gravity of the target vehicle.

[0126] In addition, in order to improve the efficiency of subsequent display of target road surface information, after obtaining the target distance corresponding to each lane line, the left lane line array and the right lane line array can be sorted according to the target distance, so that the lane lines in the left lane line array are sorted in order from large to small according to the target distance, and the lane lines in the right lane line array are sorted in order from small to large according to the target distance. After the sorting is completed, the target distance corresponding to each lane line in the left lane line array is matched with each left reference distance range, and the corresponding target mark is added to the successfully matched lane line, and the target distance corresponding to each lane line in the right lane line array is matched with each right reference distance range, and the corresponding target mark is added to the successfully matched lane line.

[0127] It can be seen from the above scheme that compared with directly selecting a discrete point from the lane line to draw a perpendicular line from the center line of the reference lane to determine the target distance between the lane line and the center line of the reference lane, the embodiment of the present application first uses the vertical coordinate range of the center line of the reference lane to intercept the first discrete point within the vertical coordinate range from the lane line, thereby avoiding redundant discrete points outside the vertical coordinate range from interfering with the calculation accuracy of the target distance, and then comprehensively determines the target distance between the lane line and the center line of the reference lane based on the distance between each first discrete point and the second discrete point closest to the center line of the reference lane, which can improve the accuracy of the target distance.

[0128] S140: Output and display target road surface information.

[0129] The target road surface information includes an array of marked left lane lines and an array of marked right lane lines.

[0130] When the embodiment of the present application is applied to a vehicle, the target road surface information can be directly displayed on the output of the vehicle's display device, such as on the display screen of the vehicle computer, or on the front windshield based on the HUD (Dead Up Display). When the embodiment of the present application is applied to a server, the server can send the target road surface information to the target vehicle so that the target vehicle can output and display it on its own display device. Of course, the target road surface information can also be sent by the target vehicle or the server to a mobile terminal for output and display on the mobile terminal.

[0131] When displaying the target road surface information, it can be displayed according to the distance of each line from the target reference object. The lines located on the left side of the target reference object are displayed in order from left to right according to the distance from large to small, and the lines located on the right side of the target reference object are displayed in order from left to right according to the distance from small to large.

[0132] The road surface information display method provided by the embodiments of the present application first perceives multiple lane lines existing on the road surface through a road image perception model, and then uses the reference lane center line or the target vehicle as the target reference object. Based on the relative positions of the target reference object and each lane line, the lane lines are grouped into left and right arrays. By respectively matching the target distances corresponding to each lane line in the left lane line array with each left reference distance range, and respectively matching the target distances corresponding to each lane line in the right lane line array with each right reference distance range, target marks corresponding to the successfully matched lane lines are added to obtain the left lane line array and the right lane line array for output. It can be seen that when determining the lane lines for output display, the embodiments of the present application only rely on the road images collected by the target vehicle and no longer rely on map information. Therefore, regardless of whether the map information is incorrect or missing, it does not affect the accuracy of the lane line determination in the present application, and further reduces the probability of traffic accidents caused by incorrect or missing display of lane lines.

[0133] In a possible implementation manner, before dividing the multiple lane lines into a left lane line array and a right lane line array according to the relative positions of each lane line and the target reference object, lane lines that meet the lane line requirements are selected from the multiple lane lines. The lane line requirements include at least one of the following: whether the length of the lane line is greater than or equal to the first length threshold, the number of discrete points included in the lane line is greater than or equal to the first quantity threshold, and the orientation of the lane line is consistent with the driving direction of the target vehicle. According to the relative positions of the selected lane lines and the target reference object, the selected lane lines are divided into a left lane line array and a right lane line array.

[0134] Before grouping the multiple lane lines, the embodiments of the present application first filter out invalid or poorly displayed lane lines according to the lane line requirements, and then group the lane lines that meet the lane line requirements, which can improve the accuracy and visual effect of the subsequent output display of lane lines.

[0135] In a possible implementation manner, in practical applications, problems such as jitter and flicker often occur when displaying the target road surface information due to reasons such as insufficient clarity of the collected road images and occlusion of the lines. To avoid the unnecessary visual and psychological pressure brought by these problems to the driver and thus affect driving safety, the embodiments of the present application provide the following method:

[0136] From the start of the function of displaying the target road surface information, when the lane line can be perceived for the first consecutive period, the target road surface information perceived in the current cycle is output and displayed; when the lane line is not perceived for the first time, the first road surface information is output and displayed, until the lane line is not perceived for the second consecutive period, the function of displaying the target road surface information is suspended, or when the lane line is perceived again within the second period, the target road surface information perceived in the current cycle is output and displayed, wherein the first road surface information includes the target road surface information determined when the lane line was last perceived.

[0137] The period from collecting a frame of road image to determining and displaying the target road surface information corresponding to the road image can be called a cycle. The first duration is smaller than the second duration, and the specific value can be determined according to actual experience, for example, the first duration = 500ms, the second duration = 2s.

[0138] like Figure 3 As shown, there is an area in the middle where the lane lines cannot be perceived. In order to avoid flickering or jittering caused by a brief lack of display information, thereby causing unnecessary driving panic to the driver, the previously perceived lane lines can be displayed when the target vehicle is driving on this section of road, thereby achieving imperceptible filtering for the scene where the lane lines in the real world disappear briefly.

[0139] In addition, when the cumulative time for outputting and displaying the first road surface information does not reach the second time length, and it is detected that the driving state of the target vehicle satisfies the first driving state, when the lane line is perceived again, the target road surface information perceived in the current cycle is output and displayed, and when the lane line is not perceived again, the function of displaying the target road surface information is suspended, wherein the first driving state includes the steering wheel angle of the target vehicle being greater than or equal to the first angle threshold, and / or the turning angle rate of the target vehicle being greater than or equal to the first turning angle rate threshold.

[0140] When the target vehicle is in a scene such as turning or changing lanes, if there is a large position gap between the delayed target road information and the newly detected target road information, a jump will occur, causing a sudden change in the target road information on the display device. Therefore, when the cumulative time for outputting and displaying the first road information does not reach the second time, and it is detected that the driving state of the target vehicle meets the first driving state, the first road information will no longer be displayed, but the target road information perceived in the current cycle will be displayed.

[0141] In layman's terms, when the lane line disappears, the previous lane line will remain displayed first. When it is detected that the target vehicle wants to change lanes or turn, and the duration has not reached the second duration, if the lane line is sensed again, the currently detected lane line will be directly displayed. If the lane line is not sensed again, no information will be displayed to avoid the phenomenon that the difference between the previously displayed lane line and the re-sensed lane line is too large, resulting in a sudden change.

[0142] In a possible implementation, when the speed of the target vehicle is detected to be 0, the second road surface information is output and displayed, until the speed of the target vehicle is not 0, the target road surface information sensed in the current cycle is output and displayed, wherein the second road surface information includes the target road surface information sensed most recently before the speed of the target vehicle becomes 0. The speed of the target vehicle can be detected by a speed sensor.

[0143] When the target vehicle is stationary, the lane line should not have any form of jitter. If the result of the current cycle perception is displayed directly, a certain degree of jitter may occur. Therefore, when the target vehicle is stationary, the second road surface information is output and displayed, and the real-time data perceived is no longer updated. Only when the target vehicle starts to move, the target road surface information perceived in the current cycle is re-output and displayed.

[0144] In one possible implementation, when the target vehicle is first detected to be in a low-speed congestion state, and the driving state of the target vehicle does not satisfy the second driving state, the third road surface information is output and displayed until the target vehicle is in a non-low-speed congestion state, or is in a low-speed congestion state and satisfies the second driving state, the target road surface information perceived in the current cycle is output and displayed, wherein the second driving state includes the target vehicle's steering wheel angle being greater than or equal to a second angle threshold, and / or the target vehicle's turning angle rate being greater than or equal to a second turning angle rate threshold, and the third road surface information includes the target road surface information most recently perceived before the target vehicle enters the low-speed congestion state.

[0145] When it is detected that the speed of the target vehicle in a recent period of time is less than or equal to the speed threshold, and the displacement of the target vehicle in this period of time is less than or equal to the displacement threshold, it can be determined that the target vehicle is in a low-speed congestion state.

[0146] In the scenario of low-speed congestion of the target vehicle, the visual camera is easily blocked by the vehicle in front, resulting in unstable real-time lane line results of the perception output. Therefore, when a low-speed congestion scenario is identified, when the perceived lane line is missing, the target road surface information most recently perceived before the target vehicle enters the low-speed congestion state can be displayed. The target road surface information perceived in the current cycle will not be displayed to the driver until the target vehicle leaves the low-speed congestion scenario, or wants to change lanes or turn. This ensures that the target road surface information can still be displayed stably and completely in the low-speed congestion scenario.

[0147] In a possible implementation, when the angle between any lane line in the marked left lane line array and the marked right lane line array and the target vehicle is greater than or equal to a first angle threshold, or when the steering wheel angle of the target vehicle is detected to be greater than or equal to a third angle threshold, the function of displaying the target road surface information is suspended until the angles between all lane lines in the marked left lane line array and the marked right lane line array and the target vehicle are less than the first angle threshold, and the steering wheel angle is less than the third angle threshold, then the target road surface information sensed in the current cycle is output and displayed.

[0148] When the angle between any lane line in the marked left lane line array and the marked right lane line array and the target vehicle is greater than or equal to the first angle threshold, or when the steering wheel angle of the target vehicle is detected to be greater than or equal to the third angle threshold, it means that the target vehicle is in a U-turn scenario. When the lane lines displayed by the display device are based on the vehicle body coordinate system, when the target vehicle is in a U-turn scenario, the lane lines will change greatly in a short period of time, and the display is not user-friendly. Therefore, if the target vehicle turns too far, the function of displaying the target road surface information can be suspended first, and when the angle slowly becomes smaller, the function of displaying the target road surface information can be restored. In this way, the visual impact problem caused by the large change in the lane lines during the U-turn period can be solved, thereby further optimizing the display effect in the U-turn scenario.

[0149] In a possible implementation, when there is an unknown lane line type in the marked left lane line array and the marked right lane line array determined in the current cycle, the lane line type of the same lane line detected in the previous cycle is replaced with an unknown lane line type.

[0150] Lane line types include solid lines, dotted lines, and double solid lines. The perception data may have abnormal recognition when outputting lane line types, that is, the feedback type is unknown. Therefore, when the perception result shows that the lane line exists, and the lane line type of the marked left lane line array and the marked right lane line array determined in the current cycle is unknown, the lane line type of the same lane line detected in the previous cycle is replaced with the unknown type.

[0151] By replacing the lane line type of the same lane line detected in the previous cycle with an unknown lane line type, not only can the problem of the lane line type being unknown and unable to be displayed be solved, but the accuracy of the lane line type can also be ensured to a certain extent.

[0152] In a possible implementation, when the road surface information perception result also includes multiple road edge lines, before outputting and displaying the target road surface information, the left road edge line and the right road edge line that are closest to the target reference object can be selected from the multiple road edge lines along the left and right sides of the target reference object, so that when the target road surface information is output and displayed, the left road edge line and the right road edge line are also output.

[0153] When there is a road edge line on both the left and right sides of the target reference object, the road edge line closest to the target reference object can be selected from the multiple road edge lines on the left side of the target reference object as the left road edge line, and the road edge line closest to the target reference object can be selected from the multiple road edge lines on the right side of the target reference object as the right road edge line. When there is only one road edge line on one side of the target reference object, the road edge line can be directly used as the road edge line on the corresponding side.

[0154] In order to improve the accuracy and display effect of the left road edge line and the right road edge line, before selecting the left road edge line and the right road edge line closest to the target reference object from multiple road edge lines along the left and right sides of the target reference object, the road edge lines that meet the edge line requirements can also be preliminarily screened from the multiple road edge lines. The edge line requirements include at least one of the following: the length of the road edge line is greater than or equal to the third length threshold, and the number of discrete points of the road edge line is greater than or equal to the third number threshold.

[0155] After the left road edge line and the right road edge line are determined, the left road edge line and the right road edge line may be marked as left 1 and right 1 respectively.

[0156] In a possible implementation, the method further includes:

[0157] When the angle between the left road edge line and the lane line closest to the target reference object in the marked left lane line array is greater than or equal to the second angle threshold, canceling the display of the left road edge line; and / or,

[0158] When the angle between the right road edge line and the lane line closest to the target reference object in the marked right lane line array is greater than or equal to the second angle threshold, canceling the display of the right road edge line; and / or,

[0159] When the angle between the left road edge line and the driving direction of the target vehicle is greater than or equal to a third angle threshold, canceling the display of the left road edge line; and / or,

[0160] When the angle between the right side road edge line and the driving direction of the target vehicle is greater than or equal to a third angle threshold, canceling the display of the right side road edge line; and / or,

[0161] When the distance between the left road edge line and any lane line in the marked left lane line array is less than or equal to a second distance threshold, canceling the display of the left road edge line; and / or,

[0162] When the distance between the right road edge line and any lane line in the marked right lane line array is less than or equal to a second distance threshold, the right road edge line is cancelled from being displayed.

[0163] When the angle between the left road edge line and the left lane line exceeds the second angle threshold, it means that the left road edge line is detected incorrectly and the left road edge line is not displayed. Similarly, when the angle between the right road edge line and the right lane line exceeds the second angle threshold, it means that the right road edge line is detected incorrectly and the right road edge line is not displayed.

[0164] When the angle between the left road edge line or the right road edge line and the driving direction of the target vehicle is greater than or equal to the third angle threshold, it means that the target vehicle is turning around, changing lanes or turning. In order to prevent the road edge line from changing suddenly or flashing, the road edge line on the corresponding side may not be displayed first.

[0165] Since the display range of the display device is limited, when the road edge line and the lane line are too close, they will appear to overlap or even be indistinguishable on the display device. Therefore, when the distance between the road edge line and the surrounding lane lines is less than or equal to the second distance threshold, the road edge line is not displayed.

[0166] It should be noted that the first length threshold, the second length threshold, the third length threshold, the first quantity threshold, the second quantity threshold, the third quantity threshold, the first distance threshold, the second distance threshold, the first angle threshold, the second angle threshold, the third angle threshold, the first corner rate threshold, the second corner rate threshold, the first corner threshold, the second corner threshold, the third corner threshold, the loss threshold and other thresholds are all empirical values. The first length threshold, the second length threshold and the third length threshold can be the same or different, and can be determined according to actual conditions. The same applies to the quantity threshold, the distance threshold, the angle threshold, the corner rate threshold and the corner threshold.

[0167] Corresponding to the above method embodiment, another embodiment of the present application provides a road surface information display device, such as Figure 4 As shown, the device comprises:

[0168] An acquisition unit 210 is used to acquire a road surface information perception result obtained based on a road image collected by a target vehicle and a road image perception model, wherein the road surface information perception result includes a plurality of lane lines and a plurality of road edge lines;

[0169] A division unit 220, configured to divide the plurality of lane lines into a left lane line array and a right lane line array according to a relative position between each lane line and a target reference object, wherein when the road surface information perception result includes a reference lane center line, the target reference object is the reference lane center line, and when the road surface information perception result does not include the reference lane center line, the target reference object is the target vehicle, and the reference lane center line is a lane center line that meets the reference center line requirements;

[0170] A matching mark unit 230, used for matching the target distance corresponding to each lane line in the left lane line array with each left reference distance range, and adding a corresponding target mark to the successfully matched lane line, and matching the target distance corresponding to each lane line in the right lane line array with each right reference distance range, and adding a corresponding target mark to the successfully matched lane line, wherein the target distance corresponding to each lane line is the distance between the lane line and the target reference object, the left reference distance range and the right reference distance range are ranges determined based on the distance between each lane line that meets the standard and the target reference object, and the target mark can indicate the relative positional relationship between the marked lane line and the target reference object and the positional relationship between the marked lane lines;

[0171] The output display unit 240 is used to output and display target road surface information, wherein the target road surface information includes the marked left lane line array and the marked right lane line array.

[0172] In a possible implementation manner, the device further includes:

[0173] A first selection unit is used to select a lane line that meets a lane line requirement from the multiple lane lines before dividing the multiple lane lines into a left lane line array and a right lane line array according to a relative position of each lane line and a target reference object, wherein the lane line requirement includes at least one of the following: whether the length of the lane line is greater than or equal to a first length threshold, the number of discrete points included in the lane line is greater than or equal to a first number threshold, and the direction of the lane line is consistent with the driving direction of the target vehicle;

[0174] The division unit 220 is used to divide the selected lane lines into the left lane line array and the right lane line array according to the relative position of each selected lane line and the target reference object.

[0175] In a possible implementation manner, the device further includes:

[0176] a determination unit, configured to determine, when the target reference object is the center line of the reference lane, a first discrete point set for each lane line according to a ordinate range from a starting discrete point to an ending discrete point on the center line of the reference lane, wherein the first discrete point set includes all first discrete points on the lane line that satisfy the ordinate range;

[0177] A searching unit, configured to search on the center line of the reference lane respectively a second discrete point which is closest to each of the first discrete points in the set of the first discrete points of each of the lane lines;

[0178] a calculating unit, configured to calculate the distance between each of the first discrete points and its corresponding second discrete point as a discrete point distance;

[0179] The determination unit is used to determine, for each lane line, the target distance between the lane line and the center line of the reference lane according to the distances of the discrete points of the lane line.

[0180] In a possible implementation, the reference centerline requirement includes at least one of the following:

[0181] The length of the lane centerline is greater than or equal to a second length threshold, the number of discrete points contained in the lane centerline is greater than or equal to a second number threshold, the distance from the lane centerline to the target vehicle is less than or equal to a first distance threshold, and the direction of the lane centerline is consistent with the driving direction of the target vehicle.

[0182] In a possible implementation, the output display unit 240 includes:

[0183] A first display module is used to output and display the target road surface information sensed in the current period when the lane line can be sensed for a first continuous period of time after the function of displaying the target road surface information is started;

[0184] The second display module is used to output and display the first road surface information when the lane line is perceived to be not perceived for the first time, and to suspend the display of the target road surface information when the lane line is not perceived for a second consecutive period of time, or to output and display the target road surface information perceived in the current period when the lane line is perceived again within the second period of time, wherein the first road surface information includes the target road surface information determined when the lane line was most recently perceived.

[0185] In a possible implementation, the output display unit 240 further includes:

[0186] The third display module is used for outputting and displaying the target road surface information perceived in the current cycle when the lane line is perceived again, and pausing the display of the target road surface information when the cumulative time for outputting and displaying the first road surface information does not reach the second time period and it is detected that the driving state of the target vehicle satisfies the first driving state, wherein the first driving state includes the steering wheel angle of the target vehicle being greater than or equal to the first angle threshold, and / or the turning angle rate of the target vehicle being greater than or equal to the first turning angle rate threshold.

[0187] In a possible implementation, the output display unit 240 includes:

[0188] The fourth display module is used to output and display the second road surface information when it is detected that the speed of the target vehicle is 0, and to output and display the target road surface information perceived in the current cycle when the speed of the target vehicle is not 0, wherein the second road surface information includes the target road surface information perceived most recently before the speed of the target vehicle becomes 0.

[0189] In a possible implementation, the output display unit 240 includes:

[0190] The fifth display module is used to output and display the third road surface information when it is first detected that the target vehicle is in a low-speed congestion state and the driving state of the target vehicle does not satisfy the second driving state, until the target vehicle is in a non-low-speed congestion state, or is in the low-speed congestion state and satisfies the second driving state, output and display the target road surface information perceived in the current cycle, wherein the second driving state includes the steering wheel angle of the target vehicle being greater than or equal to a second angle threshold, and / or the turning angle rate of the target vehicle being greater than or equal to a second turning angle rate threshold, and the third road surface information includes the target road surface information most recently perceived before the target vehicle enters the low-speed congestion state.

[0191] In a possible implementation, the output display unit 240 includes:

[0192] The sixth display module is used to suspend the function of displaying the target road surface information when the angle between any lane line in the marked left lane line array and the marked right lane line array and the target vehicle is greater than or equal to the first angle threshold, or when the steering wheel angle of the target vehicle is detected to be greater than or equal to the third angle threshold, until the angle between all lane lines in the marked left lane line array and the marked right lane line array and the target vehicle is less than the first angle threshold, and the steering wheel angle is less than the third angle threshold, then output and display the target road surface information perceived in the current cycle.

[0193] In a possible implementation, the output display unit 240 includes:

[0194] A replacement module is used to replace the lane line type of the same lane line detected in the previous cycle with an unknown lane line type when there is an unknown lane line type in the marked left lane line array and the marked right lane line array determined in the current cycle.

[0195] In a possible implementation manner, the device further includes:

[0196] A second selection unit, for selecting, when the road surface information perception result further includes a plurality of road edge lines, a left road edge line and a right road edge line which are closest to the target reference object from the plurality of road edge lines along the left and right sides of the target reference object before outputting and displaying the target road surface information;

[0197] The target road surface information also includes: the left road edge line and the right road edge line.

[0198] In a possible implementation manner, the device further includes:

[0199] a cancelling unit, configured to cancel display of the left road edge line when an angle between the left road edge line and a lane line closest to the target reference object in the marked left lane line array is greater than or equal to a second angle threshold; and / or,

[0200] When the angle between the right road edge line and the lane line closest to the target reference object in the marked right lane line array is greater than or equal to the second angle threshold, canceling the display of the right road edge line; and / or,

[0201] When the angle between the left road edge line and the driving direction of the target vehicle is greater than or equal to a third angle threshold, canceling the display of the left road edge line; and / or,

[0202] When the angle between the right side road edge line and the driving direction of the target vehicle is greater than or equal to the third angle threshold, canceling the display of the right side road edge line; and / or,

[0203] When the distance between the left road edge line and any lane line in the marked left lane line array is less than or equal to a second distance threshold, canceling the display of the left road edge line; and / or,

[0204] When the distance between the right road edge line and any lane line in the marked right lane line array is less than or equal to the second distance threshold, the right road edge line is cancelled from being displayed.

[0205] The road surface information display device provided in the embodiment of the present application first senses multiple lane lines on the road surface through a road image perception model, then uses the center line of the reference lane or the target vehicle as a target reference object, and groups the lane lines on the left and right sides based on the relative position of the target reference object and each lane line, and matches the target distance corresponding to each lane line in the left lane line array with each left reference distance range, and matches the target distance corresponding to each lane line in the right lane line array with each right reference distance range, and adds corresponding target marks to the successfully matched lane lines, so as to obtain the left lane line array and the right lane line array for output. It can be seen that when determining the lane lines for output display, the embodiment of the present application only relies on the road image collected by the target vehicle, and no longer relies on the map information, so that whether the map information is wrong or missing, it does not affect the accuracy of the lane line determination of the present application, and thus can reduce the probability of traffic accidents caused by lane line display errors or display omissions.

[0206] Based on the above method embodiments, another embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, it implements the method described in any of the above embodiments.

[0207] Based on the above method embodiments, another embodiment of the present application provides an electronic device or a computer device, as Figure 5 shown, including:

[0208] One or more processors 310;

[0209] The processor 310 is coupled to a storage device 320, and the storage device 320 is used to store one or more programs;

[0210] Wherein, when the one or more programs are executed by the one or more processors 310, the electronic device or the computer device implements the method described in any of the above embodiments.

[0211] An embodiment of the present application provides a vehicle, which includes the device described in any of the above embodiments, or includes the electronic device described in the above embodiments.

[0212] The vehicle includes a CPU (Central Processing Unit), a T-Box (Telematics Box), an image sensor, etc. Among them, the image sensor is used to collect road images in front of the vehicle. After the CPU obtains the road images collected by the image sensor, it can display the target road information by executing the road surface information display method provided in any of the above embodiments. The CPU can also upload the road images collected by these image sensors to the server through the T-Box. The server can determine the target road information by executing the road surface information display method provided in any of the above embodiments, and send the determined target road information to the T-Box, so that the T-Box can transmit the target road information to the car machine for display.

[0213] Based on the above embodiments, another embodiment of the present application provides a computer program product, which contains instructions. When the instructions run on a computer or a processor, the computer or the processor executes the method described in any of the above embodiments.

[0214] The above device embodiments correspond to the method embodiments and have the same technical effects as the method embodiments. For specific descriptions, please refer to the method embodiments. The device embodiments are obtained based on the method embodiments. For specific descriptions, please refer to the method embodiment part and will not be elaborated here. Those of ordinary skill in the art can understand that the drawings are only schematic diagrams of one embodiment, and the modules or processes in the drawings are not necessarily essential for implementing the present application.

[0215] Those skilled in the art can understand that the modules in the device in the embodiment can be distributed in the device in the embodiment according to the description of the embodiment, or can be changed accordingly and located in one or more devices different from the embodiment. The modules in the above embodiment can be combined into one module, or can be further divided into multiple sub-modules.

[0216] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A road surface information display method, It is characterized in that The method comprises: Acquire a road surface information perception result obtained based on a road image collected by the target vehicle and a road image perception model, wherein the road surface information perception result includes a plurality of lane lines; According to the relative position of each lane line and the target reference object, the multiple lane lines are divided into a left lane line array and a right lane line array, wherein when the road surface information perception result includes a reference lane center line, the target reference object is the reference lane center line, and when the road surface information perception result does not include the reference lane center line, the target reference object is the target vehicle, and the reference lane center line is a lane center line that meets the reference center line requirements; Matching the target distance corresponding to each lane line in the left lane line array with each left reference distance range respectively, and adding a corresponding target mark to the successfully matched lane line, and matching the target distance corresponding to each lane line in the right lane line array with each right reference distance range respectively, and adding a corresponding target mark to the successfully matched lane line, wherein the target distance corresponding to each lane line is the distance between the lane line and the target reference object, the left reference distance range and the right reference distance range are ranges determined based on the distance between each lane line that meets the standard and the target reference object, and the target mark can indicate the relative positional relationship between the marked lane line and the target reference object and the positional relationship between the marked lane lines; The target road surface information is output and displayed, wherein the target road surface information includes the marked left lane line array and the marked right lane line array.

2. The method according to claim 1, It is characterized in that Before dividing the plurality of lane lines into a left lane line array and a right lane line array according to the relative position of each lane line and the target reference object, the method further includes: Selecting a lane line that meets the lane line requirement from the multiple lane lines, wherein the lane line requirement includes at least one of the following: whether the length of the lane line is greater than or equal to a first length threshold, the number of discrete points included in the lane line is greater than or equal to a first number threshold, and the direction of the lane line is consistent with the driving direction of the target vehicle; The method of dividing the plurality of lane lines into a left lane line array and a right lane line array according to the relative position of each lane line and the target reference object comprises: According to the relative position between each of the selected lane lines and the target reference object, the selected lane lines are divided into the left lane line array and the right lane line array.

3. The method according to claim 1, It is characterized in that When the target reference object is the center line of the reference lane, the method for calculating the target distance includes: Determine a first discrete point set for each lane line according to a ordinate range from a starting discrete point to an ending discrete point on the center line of the reference lane, wherein the first discrete point set includes all first discrete points on the lane line that satisfy the ordinate range; Searching for the second discrete point closest to each of the first discrete points in the first discrete point set of each lane line on the center line of the reference lane, and calculating the distance between each of the first discrete points and its corresponding second discrete point as the discrete point distance; For each lane line, the target distance between the lane line and the center line of the reference lane is determined according to the distances of the discrete points of the lane line.

4. The method according to claim 1, It is characterized in that The reference centerline requirements include at least one of the following: The length of the lane centerline is greater than or equal to a second length threshold, the number of discrete points contained in the lane centerline is greater than or equal to a second number threshold, the distance from the lane centerline to the target vehicle is less than or equal to a first distance threshold, and the direction of the lane centerline is consistent with the driving direction of the target vehicle.

5. The method according to claim 1, It is characterized in that The output display target road surface information includes: From the start of the function of displaying the target road surface information, when the lane line can be sensed for a first continuous period of time, outputting and displaying the target road surface information sensed in the current period; When the vehicle changes from sensing a lane line to not sensing a lane line for the first time, the first road surface information is output and displayed. When the vehicle fails to sense the lane line for a second consecutive period of time, the function of displaying the target road surface information is suspended. Or, when the vehicle senses the lane line again within the second period of time, the target road surface information sensed in the current cycle is output and displayed, wherein the first road surface information includes the target road surface information determined the last time the vehicle was able to sense the lane line.

6. The method according to claim 5, It is characterized in that The output display target road surface information also includes: When the cumulative time for outputting and displaying the first road surface information does not reach the second time length, and it is detected that the driving state of the target vehicle satisfies the first driving state, when the lane line is perceived again, the target road surface information perceived in the current cycle is output and displayed, and when the lane line is not perceived again, the function of displaying the target road surface information is suspended, wherein the first driving state includes the steering wheel angle of the target vehicle being greater than or equal to a first angle threshold, and / or the turning angle rate of the target vehicle being greater than or equal to a first turning angle rate threshold.

7. The method according to claim 1, It is characterized in that The output display target road surface information includes: When it is detected that the speed of the target vehicle is 0, the second road surface information is output and displayed, until the speed of the target vehicle is not 0, the target road surface information perceived in the current cycle is output and displayed, wherein the second road surface information includes the target road surface information perceived most recently before the speed of the target vehicle becomes 0.

8. The method according to claim 1, It is characterized in that The output display target road surface information includes: When it is first detected that the target vehicle is in a low-speed congestion state, and the driving state of the target vehicle does not satisfy the second driving state, the third road surface information is output and displayed until the target vehicle is in a non-low-speed congestion state, or is in the low-speed congestion state and satisfies the second driving state, the target road surface information perceived in the current cycle is output and displayed, wherein the second driving state includes the steering wheel angle of the target vehicle being greater than or equal to a second angle threshold, and / or the turning angle rate of the target vehicle being greater than or equal to a second turning angle rate threshold, and the third road surface information includes the target road surface information most recently perceived before the target vehicle enters the low-speed congestion state.

9. The method according to claim 1, It is characterized in that The output display target road surface information includes: When the angle between any lane line in the marked left lane line array and the marked right lane line array and the target vehicle is greater than or equal to the first angle threshold, or when it is detected that the steering wheel angle of the target vehicle is greater than or equal to the third angle threshold, the function of displaying the target road surface information is suspended until the angles between all lane lines in the marked left lane line array and the marked right lane line array and the target vehicle are less than the first angle threshold, and the steering wheel angle is less than the third angle threshold, then the target road surface information perceived in the current cycle is output and displayed.

10. The method according to claim 1, It is characterized in that The output display target road surface information includes: When there is an unknown lane line type in the marked left lane line array and the marked right lane line array determined in the current cycle, the lane line type of the same lane line detected in the previous cycle is replaced with an unknown lane line type.

11. The method according to any one of claims 1 to 10, It is characterized in that The road surface information perception result also includes a plurality of road edge lines. Before outputting and displaying the target road surface information, the method further includes: Along the left and right sides of the target reference object, select a left road edge line and a right road edge line that are closest to the target reference object from the multiple road edge lines; The target road surface information also includes: the left road edge line and the right road edge line.

12. The method according to claim 11, It is characterized in that The method further comprises: When the angle between the left road edge line and the lane line closest to the target reference object in the marked left lane line array is greater than or equal to a second angle threshold, canceling the display of the left road edge line; and / or, When the angle between the right road edge line and the lane line closest to the target reference object in the marked right lane line array is greater than or equal to the second angle threshold, canceling the display of the right road edge line; and / or, When the angle between the left road edge line and the driving direction of the target vehicle is greater than or equal to a third angle threshold, canceling the display of the left road edge line; and / or, When the angle between the right side road edge line and the driving direction of the target vehicle is greater than or equal to the third angle threshold, canceling the display of the right side road edge line; and / or, When the distance between the left road edge line and any lane line in the marked left lane line array is less than or equal to a second distance threshold, canceling the display of the left road edge line; and / or, When the distance between the right road edge line and any lane line in the marked right lane line array is less than or equal to the second distance threshold, the right road edge line is cancelled from being displayed.

13. A road surface information display device, It is characterized in that The device comprises: An acquisition unit, used to acquire a road surface information perception result obtained based on a road image collected by a target vehicle and a road image perception model, wherein the road surface information perception result includes a plurality of lane lines and a plurality of road edge lines; a dividing unit, for dividing the plurality of lane lines into a left lane line array and a right lane line array according to a relative position between each lane line and a target reference object, wherein when the road surface information perception result includes a reference lane center line, the target reference object is the reference lane center line, and when the road surface information perception result does not include the reference lane center line, the target reference object is the target vehicle, and the reference lane center line is a lane center line that meets the reference center line requirements; a matching marking unit, for matching the target distance corresponding to each lane line in the left lane line array with each left reference distance range, and adding a corresponding target mark to the successfully matched lane line, and matching the target distance corresponding to each lane line in the right lane line array with each right reference distance range, and adding a corresponding target mark to the successfully matched lane line, wherein the target distance corresponding to each lane line is the distance between the lane line and the target reference object, the left reference distance range and the right reference distance range are ranges determined based on the distance between each lane line that meets the standard and the target reference object, and the target mark can indicate the relative positional relationship between the marked lane line and the target reference object and the positional relationship between the marked lane lines; An output display unit is used to output and display target road surface information, wherein the target road surface information includes the marked left lane line array and the marked right lane line array.

14. The device according to claim 13, It is characterized in that The output display unit comprises: A first display module is used to output and display the target road surface information sensed in the current period when the lane line can be sensed for a first continuous period of time after the function of displaying the target road surface information is started; The second display module is used to output and display the first road surface information when the lane line is perceived to be not perceived for the first time, and to suspend the display of the target road surface information when the lane line is not perceived for a second consecutive period of time, or to output and display the target road surface information perceived in the current period when the lane line is perceived again within the second period of time, wherein the first road surface information includes the target road surface information determined when the lane line was most recently perceived.

15. The device according to claim 14, It is characterized in that The output display unit also includes: The third display module is used for outputting and displaying the target road surface information perceived in the current cycle when the lane line is perceived again, and pausing the display of the target road surface information when the cumulative time for outputting and displaying the first road surface information does not reach the second time period and it is detected that the driving state of the target vehicle satisfies the first driving state, wherein the first driving state includes the steering wheel angle of the target vehicle being greater than or equal to the first angle threshold, and / or the turning angle rate of the target vehicle being greater than or equal to the first turning angle rate threshold.

16. The device according to claim 13, It is characterized in that The output display unit comprises: The fourth display module is used to output and display the second road surface information when it is detected that the speed of the target vehicle is 0, and to output and display the target road surface information perceived in the current cycle when the speed of the target vehicle is not 0, wherein the second road surface information includes the target road surface information perceived most recently before the speed of the target vehicle becomes 0.

17. The device according to claim 13, It is characterized in that The output display unit comprises: The fifth display module is used to output and display the third road surface information when it is first detected that the target vehicle is in a low-speed congestion state and the driving state of the target vehicle does not satisfy the second driving state, until the target vehicle is in a non-low-speed congestion state, or is in the low-speed congestion state and satisfies the second driving state, output and display the target road surface information perceived in the current cycle, wherein the second driving state includes the steering wheel angle of the target vehicle being greater than or equal to a second angle threshold, and / or the turning angle rate of the target vehicle being greater than or equal to a second turning angle rate threshold, and the third road surface information includes the target road surface information most recently perceived before the target vehicle enters the low-speed congestion state.

18. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the program is executed by a processor, the method according to any one of claims 1 to 12 is implemented.

19. An electronic device, It is characterized in that The electronic device comprises: one or more processors; The processor is coupled to a storage device, and the storage device is used to store one or more programs; When the one or more programs are executed by the one or more processors, the electronic device implements the method according to any one of claims 1 to 12.

20. A vehicle, It is characterized in that The vehicle comprises the apparatus according to any one of claims 13 to 17, or comprises the electronic device according to claim 19.

Images