Lane offset distance determination method and device, equipment and storage medium

Abstract

The application relates to a lane deviation distance determination method and device, equipment and a storage medium, and relates to the technical field of automobiles. The method comprises the following steps: acquiring sensor data and positioning information of a target vehicle, and a preset map of the target vehicle in a target area, the sensor data comprising visual sensor data and laser radar sensor data, and the preset map comprising lane line data information; generating a visual map based on the visual sensor data, and determining a target map from the visual map and the preset map based on the positioning information, the target map comprising lane line data information of a target area where the target vehicle is located; determining a first line pressing amount, a second line pressing amount and a third line pressing amount of a neighboring vehicle corresponding to the target vehicle; and determining a lane deviation distance of the neighboring vehicle based on the first line pressing amount, the second line pressing amount and the third line pressing amount. Thus, the technical problem of low accuracy in judging whether the driving position of the neighboring vehicle is pressed against a line can be solved.

Description

Technical Field

[0001] This invention relates to the field of automotive technology, and more specifically to a method, apparatus, device, and storage medium for determining lane departure distance. Background Technology

[0002] With the continuous development of automobiles and the increasing number of vehicles on the road, many drivers disregard traffic rules, changing lanes arbitrarily and driving over lane lines. This leads to insufficient braking distance for other vehicles, and even traffic accidents such as scrapes and rear-end collisions. Therefore, determining whether adjacent vehicles are driving over lane lines and taking preventative measures such as slowing down in advance is crucial. Currently, determining whether adjacent vehicles are driving over lane lines is based on the road lane markings, combined with the collected positions of adjacent vehicles, or by fitting the chassis frames of adjacent vehicles.

[0003] However, in the methods described above, both determining the position of adjacent vehicles and fitting the chassis frames of adjacent vehicles require good visibility and judgment based on the road lane lines. In extreme weather conditions such as heavy rain or fog, or when the road lane lines are severely worn, it becomes impossible to accurately determine whether adjacent vehicles are driving over the lane lines. Therefore, the accuracy of determining whether adjacent vehicles are driving over the lane lines is relatively low. Summary of the Invention

[0004] The purpose of this invention is to provide a method, apparatus, device, and storage medium for determining lane departure distance, thereby solving the technical problem of low accuracy in judging whether adjacent vehicles are crossing the lane line. The technical solution of this application is as follows:

[0005] According to a first aspect of this application, a method for determining lane departure distance is provided. The method includes: acquiring sensor data and positioning information of a target vehicle, and a preset map of the target vehicle in a target area; the sensor data includes visual sensor data and lidar sensor data; the preset map includes lane line data information; generating a visual map based on the visual sensor data, and determining a target map from the visual map and the preset map based on the positioning information; the target map includes lane line data information of the target area where the target vehicle is located; determining a first lane crossing amount, a second lane crossing amount, and a third lane crossing amount of adjacent vehicles corresponding to the target vehicle; the first lane crossing amount is determined based on the visual sensor data and lane line data information in the target map; the second lane crossing amount is determined based on the lidar sensor data and lane line data information in the target map; and the third lane crossing amount is determined based on the visual sensor data, lidar sensor data, and lane line data information in the target map; and determining the lane departure distance of adjacent vehicles based on the first lane crossing amount, the second lane crossing amount, and the third lane crossing amount.

[0006] Based on the aforementioned technical means, this application first acquires visual sensor data, lidar sensor data, positioning information, and a preset map of the target vehicle. Then, it generates a visual map from the visual sensor data and, based on the positioning information, determines the target map from the visual map and the preset map. Next, it determines the first line-crossing amount of adjacent vehicles corresponding to the target vehicle based on the visual sensor data, the second line-crossing amount of adjacent vehicles based on the lidar sensor data, and the third line-crossing amount of adjacent vehicles based on the visual sensor data and the lidar sensor data. Further, it determines the lane departure distance from the first, second, and third line-crossing amounts using a preset model. Through this method, the preset map and the visual map determined by the visual sensors can be compared and judged to select the target map, thereby obtaining an accurate lane departure distance for adjacent vehicles. Furthermore, from multiple line-crossing amounts, based on various judgment criteria of the preset model, one line-crossing amount with higher accuracy is determined as the lane departure distance, improving the accuracy of judging whether the driving position of adjacent vehicles is crossed and obtaining an accurate lane departure distance for adjacent vehicles.

[0007] In one possible implementation, determining a target map from a visual map and a preset map based on positioning information includes: determining a first lane line point set data corresponding to the visual map and a second lane line point set data corresponding to the preset map; and determining the target map from the visual map and the preset map based on the amount of data included in the first lane line point set data and the amount of data included in the second lane line point set data.

[0008] Based on the above technical means, this application determines whether the amount of data included in the point set data is greater than the preset amount by determining the first lane line point set data corresponding to the visual map and the second lane line point set data corresponding to the preset map, and determines the target map from the visual map and the preset map. This can effectively avoid the situation where the amount of data included in the point set data is too small, so that accurate lane lines and the amount of lane crossing cannot be obtained, and improve the accuracy of determining whether adjacent vehicles have crossed the lines and the lane offset distance.

[0009] In one possible implementation, determining a target map based on the amount of data included in the first lane line point set data and the amount of data included in the second lane line point set data includes: if it is determined that the amount of data included in the first target point set data is less than a preset amount of data and the amount of data included in the second target point set data is greater than or equal to a preset amount of data, then determining the map corresponding to the second target point set data as the target map, wherein the first target point set data is either the first lane line point set data or the second lane line point set data, and the second target point set data is either the first lane line point set data or the second lane line point set data, and the first target point set data and the second target point set data are different point set data.

[0010] According to the above technical means, if the amount of data included in the point set data is less than the preset amount of data, it means that there is less data representing lane lines in the map and the lane lines are not accurate enough. Therefore, the map cannot be used as the target map. Conversely, if the amount of data included in the point set data is greater than or equal to the preset amount of data, it means that there is more data representing lane lines in the map and the lane lines are more accurate. Therefore, the map can be used as the target map, and then it can be determined whether adjacent vehicles are crossing the lines.

[0011] In one possible implementation, determining a target map based on the amount of data included in the first lane line point set data and the amount of data included in the second lane line point set data includes: if the amount of data included in the first lane line point set data and the amount of data included in the second lane line point set data are both greater than or equal to a preset amount of data, determining the deviation between the lane lines in the visual map and the lane lines in the preset map; if the deviation is less than a preset deviation threshold, determining the preset map as the target map.

[0012] Based on the above technical means, if the amount of data included in the lane line point set data of both the visual map and the preset map is greater than or equal to the preset data amount, it means that both the visual map and the preset map can be used as target maps. Furthermore, by determining the deviation between the lane lines in the visual map and the lane lines in the preset map, if the deviation is less than the preset deviation threshold, the preset map is determined as the target map. This can effectively combine the positioning function of the preset map with the target vehicle and improve the accuracy of determining the lane offset distance.

[0013] In one possible implementation, determining the first, second, and third line-crossing amounts of adjacent vehicles corresponding to the target vehicle includes: determining the driving data, simulated chassis frame data, and laser point cloud convex hull data of the adjacent vehicles based on visual sensor data and lidar sensor data, wherein the driving data includes at least one of the following: driving direction, position information, vehicle offset speed, and offset distance; determining the first line-crossing amount of the adjacent vehicles based on visual sensor data and target lane lines in the target map, wherein the target lane lines are the lane lines between the target vehicle and the adjacent vehicles; determining the second line-crossing amount of the adjacent vehicles based on laser point cloud convex hull data and target lane lines in the target map; and determining the third line-crossing amount of the adjacent vehicles based on the driving data, simulated chassis frame data, and target lane lines in the target map.

[0014] Based on the aforementioned technical means, this application determines multiple lane departure measurements using visual sensor data and lidar sensor data. By comparing these multiple lane departure measurements, the lane departure measurement with the highest confidence level is determined as the lane departure distance, thereby further improving the accuracy of lane departure distance determination.

[0015] In one possible implementation, determining the third lane crossing amount of adjacent vehicles based on driving data, simulated chassis frame data, and target lane lines in the target map includes: determining the position information of the four vertices of adjacent vehicles based on driving data and simulated chassis frame data; determining the target distance between adjacent vehicles and target lane lines based on the position information of the four vertices and target lane lines in the target map, and determining the target distance as the third lane crossing amount.

[0016] Based on the aforementioned technical means, in addition to using visual sensor data to determine the first line crossing amount and using lidar sensor data to determine the second line crossing amount, this application can also combine visual sensor data and lidar sensor data to determine the driving data of adjacent vehicles and the simulated chassis frame data, thereby determining the position information of the four vertices of adjacent vehicles. Thus, based on the position information of the four vertices and the target lane line in the target map, the maximum lateral difference between the adjacent vehicle and the target lane line can be determined as the target distance and determined as the third line crossing amount. Furthermore, the line crossing amount with the highest confidence among the three line crossing amounts can be determined as the lane deviation distance.

[0017] In one possible implementation, determining the lane departure distance of adjacent vehicles based on a first line crossing amount, a second line crossing amount, and a third line crossing amount includes: inputting the first line crossing amount, the second line crossing amount, and the third line crossing amount into a preset model; determining multiple confidence levels corresponding to any one of the first line crossing amount, the second line crossing amount, and the third line crossing amount; and determining a target confidence level corresponding to any one of the multiple confidence levels; and determining the line crossing amount corresponding to the largest target confidence level among the first target confidence level, the second target confidence level, and the third target confidence level as the lane departure distance of adjacent vehicles based on the first target confidence level, the second target confidence level, and the third target confidence level.

[0018] Based on the above technical means, this application inputs the first line crossing amount, the second line crossing amount, and the third line crossing amount into a preset model including a sensor module, a variance module, a motion state module, a scene module, a deviation module, and a multi-source data comparison module to determine the target confidence level corresponding to any line crossing amount, thereby using the line crossing amount with the highest target confidence level as the lane offset distance of adjacent vehicles.

[0019] According to a second aspect of this application, a lane departure distance determination device is provided. The device includes an acquisition module and a processing module. The acquisition module acquires sensor data and positioning information of a target vehicle, as well as a preset map of the target vehicle in a target area. The sensor data includes visual sensor data and lidar sensor data, and the preset map includes lane line data. The processing module generates a visual map based on the visual sensor data and determines a target map from the visual map and the preset map based on the positioning information. The target map includes lane line data of the target area where the target vehicle is located. The processing module further determines a first lane overlap, a second lane overlap, and a third lane overlap for adjacent vehicles corresponding to the target vehicle. The first lane overlap is determined based on the visual sensor data and lane line data in the target map, the second lane overlap is determined based on the lidar sensor data and lane line data in the target map, and the third lane overlap is determined based on the visual sensor data, lidar sensor data, and lane line data in the target map. The processing module further determines the lane departure distance of adjacent vehicles based on the first lane overlap, the second lane overlap, and the third lane overlap.

[0020] In one possible implementation, the processing module is specifically used to determine the first lane line point set data corresponding to the visual map and the second lane line point set data corresponding to the preset map; the processing module is specifically used to determine the target map from the visual map and the preset map based on the amount of data included in the first lane line point set data and the amount of data included in the second lane line point set data.

[0021] In one possible implementation, the processing module is specifically used to determine the map corresponding to the second target point set data as the target map when it is determined that the amount of data included in the first target point set data is less than the preset amount of data and the amount of data included in the second target point set data is greater than or equal to the preset amount of data. The first target point set data is either the first lane line point set data or the second lane line point set data, and the second target point set data is either the first lane line point set data or the second lane line point set data. The first target point set data and the second target point set data are different point set data.

[0022] In one possible implementation, the processing module is specifically used to determine the deviation between the lane lines in the visual map and the lane lines in the preset map when the amount of data included in the first lane line point set data and the amount of data included in the second lane line point set data are both greater than or equal to a preset amount of data; the processing module is specifically used to determine the preset map as the target map when the deviation is less than a preset deviation threshold.

[0023] In one possible implementation, the processing module is specifically configured to determine the driving data, simulated chassis frame data, and laser point cloud convex hull data of adjacent vehicles based on visual sensor data and lidar sensor data. The driving data includes at least one of the following: driving direction, position information, vehicle offset speed, and offset distance. The processing module is specifically configured to determine the first line-crossing amount of adjacent vehicles based on visual sensor data and target lane lines in the target map, where the target lane lines are the lane lines between the target vehicle and the adjacent vehicles. The processing module is specifically configured to determine the second line-crossing amount of adjacent vehicles based on the laser point cloud convex hull data and target lane lines in the target map. The processing module is specifically configured to determine the third line-crossing amount of adjacent vehicles based on the driving data, simulated chassis frame data, and target lane lines in the target map.

[0024] In one possible implementation, the processing module is specifically used to determine the position information of the four vertices of the adjacent vehicles based on the driving data of the adjacent vehicles and the simulated chassis frame data; the processing module is specifically used to determine the target distance between the adjacent vehicles and the target lane line based on the position information of the four vertices and the target lane line in the target map, and to determine the target distance as the third lane crossing amount.

[0025] In one possible implementation, the processing module is specifically used to input the first line crossing amount, the second line crossing amount, and the third line crossing amount into a preset model, determine multiple confidence levels corresponding to any one of the first line crossing amount, the second line crossing amount, and the third line crossing amount, and determine a target confidence level corresponding to any one of the multiple confidence levels; the processing module is specifically used to determine the line crossing amount corresponding to the largest target confidence level among the first target confidence level, the second target confidence level, and the third target confidence level as the lane offset distance of the adjacent vehicle, based on the first target confidence level corresponding to the first line crossing amount, the second target confidence level corresponding to the second line crossing amount, and the third target confidence level corresponding to the third line crossing amount.

[0026] According to a third aspect provided in this application, an electronic device is provided, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to execute instructions to implement the method of the first aspect described above and any possible implementation thereof.

[0027] According to a fourth aspect provided in this application, a computer-readable storage medium is provided that, when the instructions in the computer-readable storage medium are executed by a processor of an electronic device, enables the electronic device to perform the methods described in the first aspect and any possible implementation thereof.

[0028] According to the fifth aspect provided in this application, a vehicle is provided, including: a lane departure distance determination device for implementing the first aspect described above and any possible implementation thereof.

[0029] According to the sixth aspect provided in this application, a computer program product is provided, the computer program product including computer instructions, which, when executed on an electronic device, cause the electronic device to perform the method described in the first aspect and any possible implementation thereof.

[0030] Therefore, the above-mentioned technical features of this application have the following beneficial effects:

[0031] (1) First, acquire the visual sensor data, lidar sensor data, positioning information, and preset map of the target vehicle. Then, generate a visual map from the visual sensor data. Based on the positioning information, determine the target map from the visual map and the preset map. Then, determine the first line crossing amount of the adjacent vehicle corresponding to the target vehicle based on the visual sensor data, the second line crossing amount of the adjacent vehicle based on the lidar sensor data, and the third line crossing amount of the adjacent vehicle based on the visual sensor data and the lidar sensor data. Further, determine the lane offset distance from the first, second, and third line crossing amounts using a preset model. Through the above method, the preset map and the visual map determined by the visual sensor can be compared and judged to select the target map, thereby obtaining the accurate lane offset distance of the adjacent vehicle. From multiple line crossing amounts, based on various judgment criteria of the preset model, determine one of the line crossing amounts with higher accuracy as the lane offset distance, improving the accuracy of judging whether the driving position of the adjacent vehicle is crossed, and obtaining the accurate lane offset distance of the adjacent vehicle.

[0032] (2) By determining the first lane line point set data corresponding to the visual map and the second lane line point set data corresponding to the preset map, it is determined whether the amount of data included in the point set data is greater than the preset amount of data, and the target map is determined from the visual map and the preset map. This can effectively avoid the point set data being too small, which would prevent the accurate lane lines and the amount of line crossing from being obtained, and improve the accuracy of determining whether adjacent vehicles are crossing the line and the lane offset distance.

[0033] (3) If the amount of data included in the point set data is less than the preset amount of data, it means that there is less data representing lane lines in the map and the lane lines are not accurate enough. Therefore, the map cannot be used as the target map. Conversely, if the amount of data included in the point set data is greater than or equal to the preset amount of data, it means that there is more data representing lane lines in the map and the lane lines are more accurate. Therefore, the map can be used as the target map, and then it can be determined whether adjacent vehicles are crossing the lines.

[0034] (4) If the amount of data included in the lane line point set data of both the visual map and the preset map is greater than or equal to the preset data amount, it means that both the visual map and the preset map can be used as target maps. Further, by determining the deviation between the lane lines in the visual map and the lane lines in the preset map, if the deviation is less than the preset deviation threshold, the preset map is determined as the target map. This can effectively combine the positioning function of the preset map with the target vehicle and improve the accuracy of determining the lane offset distance.

[0035] (5) Multiple lane departure measurements are determined by using visual sensor data and lidar sensor data. These measurements are then compared to determine the lane departure distance with the highest confidence level, thereby improving the accuracy of lane departure distance determination.

[0036] (6) In addition to using visual sensor data to determine the first line crossing amount and using lidar sensor data to determine the second line crossing amount, the visual sensor data and lidar sensor data can be combined to determine the driving data of adjacent vehicles and the simulated chassis frame data, and then determine the position information of the four vertices of the adjacent vehicles. Thus, based on the position information of the four vertices and the target lane line in the target map, the maximum lateral difference between the adjacent vehicles and the target lane line can be determined as the target distance and determined as the third line crossing amount. Furthermore, the line crossing amount with the highest confidence among the three line crossing amounts can be determined as the lane offset distance.

[0037] (7) By inputting the first line crossing amount, the second line crossing amount and the third line crossing amount into a preset model including a sensor module, a variance module, a motion state module, a scene module, a deviation module and a multi-source data comparison module, the target confidence level corresponding to any line crossing amount is determined, and the line crossing amount with the largest target confidence level is taken as the lane offset distance of the adjacent vehicle.

[0038] It should be noted that the technical effects of any of the implementation methods in aspects two through six can be found in the technical effects of the corresponding implementation methods in aspect one, and will not be repeated here.

[0039] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0040] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application, and do not constitute an undue limitation of this application.

[0041] Figure 1 This is a schematic diagram illustrating the structure of a lane departure distance determination system according to an exemplary embodiment;

[0042] Figure 2 This is a flowchart illustrating a method for determining lane departure distance according to an exemplary embodiment;

[0043] Figure 3 This is a flowchart illustrating yet another method for determining lane departure distance according to an exemplary embodiment;

[0044] Figure 4 This is a block diagram illustrating a lane departure distance determination device according to an exemplary embodiment;

[0045] Figure 5 This is a block diagram illustrating an electronic device according to an exemplary embodiment. Detailed Implementation

[0046] The embodiments of the present invention will be described below with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention and not for limiting the scope of protection of the present invention.

[0047] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0048] In road traffic, many drivers disregard traffic rules, particularly by changing lanes arbitrarily or crossing lane lines, which severely impacts the normal operation of autonomous vehicles. Therefore, determining whether vehicles in adjacent lanes are crossing lane lines, quantifying the degree of line crossing, and thus anticipating vehicle intentions and developing appropriate response strategies in advance are pressing issues in the field of autonomous driving.

[0049] Currently, determining whether a vehicle has crossed the lane lines relies on visual images obtained from vision sensors. This method is highly accurate only under favorable weather conditions where lane lines are clearly visible. However, in extreme weather conditions such as heavy rain or fog, or when lane lines are severely worn, the accuracy drops, resulting in lower conclusions about whether a vehicle has crossed the lines. Furthermore, it only judges whether the target vehicle has crossed the lines without quantifying the extent of the crossing, making it impossible to accurately predict the vehicle's intent based on the degree of crossing.

[0050] For ease of understanding, the method for determining lane offset distance provided in this application will be described in detail below with reference to the accompanying drawings.

[0051] The method for determining lane offset distance provided in this application embodiment can be applied to lane offset distance determination systems. Figure 1 This is a schematic diagram illustrating the structure of a lane departure distance determination system according to an exemplary embodiment. Figure 1 As shown, the lane departure distance determination system 10 includes: multiple sensor devices 11 and a vehicle controller 12.

[0052] Among them, multiple sensor devices 11 include vehicle-mounted sensors such as vision sensors and lidar sensors, which can be used to acquire environmental information around the target vehicle. The vision sensor is an onboard camera, which can be a monocular camera, a binocular camera, a fisheye camera, etc., and the lidar sensor can be a mechanical lidar, a phased array lidar, a floodlight array lidar, etc.

[0053] The vehicle controller 12 first acquires visual sensor data, lidar sensor data, and positioning information of the target vehicle collected by multiple sensor devices 11, as well as a preset map. It then generates a visual map based on the visual sensor data and determines the target map based on the positioning information, the preset map, and the visual map. Next, based on the visual sensor data, lidar sensor data, and the target map, it determines the first, second, and third lane-crossing distances of adjacent vehicles corresponding to the target vehicle. Using a preset model, it determines the lane offset distance of the adjacent vehicles from these three lane-crossing distances.

[0054] Figure 2 This is a flowchart illustrating a method for determining lane departure distance according to an exemplary embodiment, such as... Figure 2 As shown, the method for determining the lane offset distance includes the following steps:

[0055] S201. Obtain sensor data and positioning information of the target vehicle, as well as a preset map of the target vehicle in the target area.

[0056] The sensor data includes visual sensor data and lidar sensor data, and the preset map includes lane line data.

[0057] For example, sensor data can be collected using visual sensors and LiDAR sensors. The visual sensor is an onboard camera, which can be a front-view camera, specifically a monocular camera, a binocular camera, a fisheye camera, etc. The LiDAR sensor can be a mechanical LiDAR, a phased array LiDAR, a floodlight array LiDAR, etc.

[0058] It is understandable that using lane line data included in the preset map as the data source for the target lane line can effectively avoid the problem of lane lines being worn or not being accurately obtained under extreme weather conditions, which would lead to an inability to accurately determine whether adjacent vehicles have crossed the line and the extent of the crossing.

[0059] S202. Generate a visual map based on visual sensor data.

[0060] Specifically, the visual sensor data acquired by the visual sensor can effectively represent the environmental information of the target vehicle, and a visual map of the target vehicle can be constructed based on the environmental information of the target vehicle.

[0061] Optionally, visual sensor data, including multiple visual images or video data, can be obtained from the visual sensor, and a visual map can be constructed based on the pixel information in the multiple visual images or video data.

[0062] S203. Determine the target map from the visual map and the preset map based on the positioning information.

[0063] The target map includes lane line data information for the target area where the target vehicle is located.

[0064] Specifically, firstly, the data volume of lane line point sets included in the visual map and the preset map is compared with the preset data volume. If the data volume of lane line point sets in only one map is greater than the preset data volume, then that map is taken as the target map. If the data volume of lane line point sets included in both the visual map and the preset map is greater than the preset data volume, the deviation between the lane lines in the visual map and the lane lines in the preset map is further determined. If the deviation is less than the preset deviation threshold, the preset map can be determined as the target map.

[0065] It is understandable that by judging the deviation between the lane lines in the visual map and the lane lines in the preset map, the visual map and the preset map are compared to ensure that the target map is closer to the vehicle lines on the real road within the driving space of the target vehicle, thus ensuring the accuracy of whether adjacent vehicles cross the line and the amount of line crossing.

[0066] S204. Determine the first, second, and third line-crossing amounts of the adjacent vehicles corresponding to the target vehicle.

[0067] The first line crossing amount is determined based on visual sensor data and lane line data information in the target map; the second line crossing amount is determined based on lidar sensor data and lane line data information in the target map; and the third line crossing amount is determined based on visual sensor data, lidar sensor data, and lane line data information in the target map.

[0068] Specifically, the visual sensor data and lidar sensor data are first processed to determine the driving data of adjacent vehicles, the simulated chassis frame data, and the lidar point cloud convex hull data. Then, based on the visual sensor data and the target lane lines in the target map, the first line crossing amount of adjacent vehicles is determined. Based on the lidar point cloud convex hull data and the target lane lines in the target map, the second line crossing amount of adjacent vehicles is determined. Based on the driving data of adjacent vehicles, the simulated chassis frame data, and the target lane lines in the target map, the third line crossing amount of adjacent vehicles is determined.

[0069] Optionally, during the driving of the target vehicle, at each fixed time period, the first, second, and third line-crossing amounts for that time period are calculated.

[0070] S205. Based on the first line crossing amount, the second line crossing amount, and the third line crossing amount, determine the lane offset distance of adjacent vehicles.

[0071] Specifically, the first, second, and third line crossing amounts are input into a preset model to determine multiple confidence levels corresponding to each line crossing amount. These multiple confidence levels are then summed to obtain the target confidence level for that line crossing amount. The target confidence levels for each line crossing amount are further compared, and the line crossing amount with the highest target confidence level is taken as the lane departure distance, which is the lane departure distance between adjacent vehicles.

[0072] In some embodiments, the method in step S203 above specifically includes the following steps S301-S302:

[0073] S301. Determine the first lane line point set data corresponding to the visual map and the second lane line point set data corresponding to the preset map.

[0074] Specifically, based on the point set data constituting the visual map, the first lane line point set data constituting the lane lines in the visual map is determined; and based on the point set data constituting the preset map, the second lane line point set data constituting the lane lines in the preset map is determined.

[0075] It is understandable that by using the visual map and the preset map, the first lane line point set data corresponding to the visual map and the second lane line point set data corresponding to the preset map are determined. These data are then used to determine the amount of subsequent lane line point set data, thereby determining the target map.

[0076] S302. Based on the amount of data included in the first lane line point set data and the amount of data included in the second lane line point set data, determine the target map from the visual map and the preset map.

[0077] Specifically, the data volume of the first lane line point set and the data volume of the second lane line point set are compared with the preset data volume to determine whether the lane lines of the visual map and the lane lines of the preset map are valid, that is, whether the lane line point set data can form accurate lane lines, and the target map is determined from the visual map and the preset map.

[0078] In some embodiments, the method in step S302 above specifically includes the following step S401:

[0079] S401. If it is determined that the amount of data included in the first target point set is less than the preset amount of data, and the amount of data included in the second target point set is greater than or equal to the preset amount of data, the map corresponding to the second target point set is determined as the target map.

[0080] The first target point set data is either the first lane line point set data or the second lane line point set data, and the second target point set data is either the first lane line point set data or the second lane line point set data. The first target point set data and the second target point set data are different point set data.

[0081] For example, if the amount of data in the first lane line point set data corresponding to the visual map is less than the preset amount of data, and the amount of data in the second lane line point set data corresponding to the preset map is greater than or equal to the preset amount of data, it indicates that the lane lines of the visual map are invalid, while the lane lines of the preset map are valid, and the preset map is determined as the target map.

[0082] For example, if the amount of data in the second lane line point set data corresponding to the preset map is less than the preset amount of data, and the amount of data in the first lane line point set data corresponding to the visual map is greater than or equal to the preset amount of data, it indicates that the lane lines of the preset map are invalid, while the lane lines of the visual map are valid, and the visual map is determined as the target map.

[0083] Optionally, if the amount of data in the first lane line point set corresponding to the visual map and the amount of data in the second lane line point set corresponding to the preset map are both less than the preset amount of data, it means that the lane lines in the preset map and the lane lines in the visual map are both invalid. In this case, the lane lines cannot be determined, and it is impossible to determine whether the target vehicle has crossed the line or the amount of crossing. The determination of the lane offset distance ends here.

[0084] In some embodiments, the method in step S302 above specifically includes the following steps S501-S502:

[0085] S501. If the amount of data included in the first lane line point set data and the amount of data included in the second lane line point set data are both greater than or equal to the preset amount of data, determine the deviation between the lane lines in the visual map and the lane lines in the preset map.

[0086] Specifically, if the amount of data included in the first lane line point set and the amount of data included in the second lane line point set are both greater than or equal to the preset amount of data, it indicates that the lane lines of the preset map and the lane lines of the visual map are both valid. Furthermore, within a certain vertical range, the lateral deviation between the lane lines in the visual map and the lane lines in the preset map is determined, thereby judging the magnitude of the lateral deviation and the preset deviation threshold, and determining whether the preset map is relatively close to the visual map representing the real road conditions.

[0087] For example, if lane lines in the visual map and lane lines in the preset map are both valid, both lane lines in the visual map and lane lines in the preset map can be used as candidate lane lines. Further, based on road information, visual sensor data, and multiple preset road parameters, the longitudinal range `max_distance` to be tested is determined using the following formula:

[0088]

[0089] Among them, |X road | represents the absolute value of the road curvature. Y is the absolute value of the rate of change of road curvature. range Here, d is the spatial range adjustment coefficient, d is the farthest distance from the target vehicle collected by the vision sensor, max_dist is the preset longitudinal range parameter, kSlope is the preset slope parameter, and min(d,max_dist) is used to determine the smaller value between d and max_dist.

[0090] Furthermore, within the longitudinal range max_distance to be tested, multiple lateral deviations Δposy between the lane lines in the visual map and the lane lines in the preset map are calculated.

[0091] For example, the preset longitudinal range parameter max_dist can be 20 meters, and the preset slope parameter kSlope can be 0.2.

[0092] S502. If the deviation is less than the preset deviation threshold, the preset map is determined as the target map.

[0093] Specifically, if the calculated deviation is less than the preset deviation threshold, it means that the lane lines in the visual map are relatively close to the lane lines in the preset map, and the preset map can be used as the target map.

[0094] Optionally, if the calculated deviation is greater than or equal to the preset deviation threshold, it indicates that the lane lines in the visual map and the lane lines in the preset map have a large deviation. Therefore, the preset map cannot be used as the target map, and the visual map is used instead.

[0095] For example, based on the calculated multiple lateral deviations Δposy, the root mean square error posy is determined using the following formula two. mse :

[0096]

[0097] Furthermore, will put posy mse Compare with a preset deviation threshold, and in the positive mse If the deviation is less than the preset deviation threshold, the preset map is determined as the target map.

[0098] In some embodiments, the method in step S204 above specifically includes the following steps S601-S604:

[0099] S601. Based on visual sensor data and lidar sensor data, determine the driving data of adjacent vehicles, simulated chassis frame data, and lidar point cloud convex hull data.

[0100] The driving data includes at least one of the following: driving direction, location information, vehicle offset speed, and offset distance.

[0101] Optionally, the simulated chassis frame data of the adjacent vehicles may include the length and width of the chassis frame.

[0102] It should be noted that the laser point cloud convex hull data is obtained by fitting multiple laser point cloud convex hulls from the laser point cloud dataset of adjacent vehicles obtained by the lidar sensor through scanning.

[0103] Optionally, various algorithms can be used to obtain the convex hull data of the laser point cloud based on the lidar sensor data. Specifically, these algorithms can include exhaustive search, divide and conquer, convolute wrapping (Jarvis step method), Graham scanning method, and fast polygon convex hull algorithm (Melkman algorithm).

[0104] S602. Based on visual sensor data and target lane lines in the target map, determine the first lane crossing amount of adjacent vehicles.

[0105] The target lane line is the lane line between the target vehicle and the adjacent vehicle.

[0106] Optionally, it can be determined first whether adjacent vehicles are within the effective range of the vision sensor. If the vertical coordinate of the location information is between the near and far effective distances of the vision sensor, and the horizontal coordinate of the location information is within a fixed lane width (e.g., 0.5 to 1.5 times the lane width), then the adjacent vehicles are determined to be within the effective range of the vision sensor.

[0107] Optionally, the location of adjacent vehicles can be determined based on the x-coordinate of their position information. If the x-coordinate of the adjacent vehicle's position information is positive, it means that the adjacent vehicle is to the left of the target vehicle; if the x-coordinate of the adjacent vehicle's position information is negative, it means that the adjacent vehicle is to the right of the target vehicle.

[0108] Specifically, based on the target lane lines in the target map and visual sensor data, it is determined whether the visual sensor data of adjacent vehicles is greater than the data representing the target lane lines. If the visual sensor data is less than the data representing the target lane lines, it means that the adjacent vehicles are not crossing the lines, and the lateral difference between the visual sensor data and the data representing the target lane lines is negative. If the visual sensor data is equal to the data representing the target lane lines, it means that the adjacent vehicles are crossing the lines, and the lateral difference between the visual sensor data and the data representing the target lane lines is 0. If the visual sensor data is greater than the data representing the target lane lines, it means that the adjacent vehicles are crossing the lines, and the lateral difference between the visual sensor data and the data representing the target lane lines is positive. Furthermore, based on the value of the lateral difference between the visual sensor data and the data representing the target lane lines, the largest lateral difference is determined as the first crossing amount of the adjacent vehicles.

[0109] Optionally, a range of values ​​for the first line pressing quantity can be set (e.g., the absolute value is less than or equal to 1.5 meters), which can reduce the amount of data to be calculated for the first line pressing quantity and improve the efficiency of determining the first line pressing quantity.

[0110] Optionally, the first pressure point of the previous period is obtained and compared with the first pressure point calculated in the current period. If the difference between the first pressure point of the previous period and the first pressure point of the current period is less than the preset pressure point threshold, it indicates that the first pressure point of the current period has not changed and is valid data. Therefore, the valid flag of the first pressure point of the current period is set to 1. If the difference between the first pressure point of the previous period and the first pressure point of the current period is greater than or equal to the preset pressure point threshold, it indicates that the first pressure point of the current period is invalid and the valid flag of the first pressure point of the current period is set to 0.

[0111] For example, when the over-limit amount of the previous period and the over-limit amount of the current period are both positive or negative, the preset over-limit amount threshold is 0.5; when the over-limit amount of the previous period and the over-limit amount of the current period are one positive and one negative, the preset over-limit amount threshold is 0.3.

[0112] S603. Based on the laser point cloud convex hull data and the target lane line in the target map, determine the second lane crossing amount of adjacent vehicles.

[0113] Optionally, it can be determined first whether adjacent vehicles are within the effective range of the lidar sensor. If the ordinate of the adjacent vehicle's position information is within the effective distance of the lidar sensor (generally 0 to 80 meters), and the lidar sensor of the target vehicle can scan a preset number (e.g., 3) of the laser point cloud convex hulls, then the adjacent vehicle is determined to be within the effective range of the lidar sensor.

[0114] Optionally, the location of adjacent vehicles can be determined based on the x-coordinate of the location information. If the x-coordinate value is positive, it means that the adjacent vehicle is to the left of the target vehicle; if the x-coordinate value is negative, it means that the adjacent vehicle is to the right of the target vehicle.

[0115] Specifically, based on the laser point cloud convex hull data and the target lane lines in the target map, it is determined whether the laser point cloud convex hull data is greater than the data representing the target lane lines, and the lateral difference between all laser point cloud convex hulls and the target lane lines in the target map is determined. If the laser point cloud convex hull data is less than the data representing the target lane lines, it means that the adjacent vehicles are not crossing the lines, and the lateral difference between the laser point cloud convex hull data and the data representing the target lane lines is negative; if the laser point cloud convex hull data is equal to the data representing the target lane lines, it means that the adjacent vehicles are crossing the lines, and the lateral difference between the laser point cloud convex hull data and the data representing the target lane lines is 0; if the laser point cloud convex hull data is greater than the data representing the target lane lines, it means that the adjacent vehicles are crossing the lines, and the lateral difference between the laser point cloud convex hull data and the data representing the target lane lines is positive. Furthermore, based on the value of the lateral difference between the laser point cloud convex hull data and the data representing the target lane lines, the largest lateral difference is determined as the second crossing amount of the adjacent vehicle.

[0116] Optionally, the second pressure point of the previous period is obtained and compared with the second pressure point calculated in the current period. If the difference between the second pressure point of the previous period and the second pressure point of the current period is less than the preset pressure point threshold, it indicates that the second pressure point of the current period has not changed and is valid data. Therefore, the valid flag of the second pressure point of the current period is set to 1. If the difference between the second pressure point of the previous period and the second pressure point of the current period is greater than or equal to the preset pressure point threshold, it indicates that the second pressure point of the current period is invalid and the valid flag of the second pressure point of the current period is set to 0.

[0117] S604. Based on driving data, simulated chassis frame data, and target lane lines in the target map, determine the third lane crossing amount of adjacent vehicles.

[0118] Specifically, based on the driving data of adjacent vehicles and the simulated chassis frame data, the position information of the four vertices of the adjacent vehicles is first determined. Then, based on the position information of the four vertices and the data representing the target lane line, it is determined whether the position information of the four vertices is greater than the data representing the target lane line, and the lateral difference between the four vertices and the target lane line is determined. If the position information of the four vertices is less than the data representing the target lane line, it means the adjacent vehicle is not crossing the line, and the lateral difference between the position information of the four vertices and the data representing the target lane line is negative. If the position information of the four vertices is equal to the data representing the target lane line, it means the adjacent vehicle is crossing the line, and the lateral difference between the position information of the four vertices and the data representing the target lane line is 0. If the position information of the four vertices is greater than the data representing the target lane line, it means the adjacent vehicle is crossing the line, and the lateral difference between the position information of the four vertices and the data representing the target lane line is positive. Finally, based on the value of the lateral difference between the position information of the four vertices and the data representing the target lane line, the largest lateral difference is determined as the third crossing amount of the adjacent vehicle.

[0119] Optionally, the third pressure line quantity of the previous period is obtained and compared with the third pressure line quantity calculated in the current period. If the difference between the third pressure line quantity of the previous period and the third pressure line quantity of the current period is less than the preset pressure line quantity threshold, it indicates that the third pressure line quantity of the current period has not changed and is valid data. Therefore, the valid flag bit of the third pressure line quantity of the current period is set to 1. If the difference between the third pressure line quantity of the previous period and the third pressure line quantity of the current period is greater than or equal to the preset pressure line quantity threshold, it indicates that the third pressure line quantity of the current period is invalid and the valid flag bit of the third pressure line quantity of the current period is set to 0.

[0120] In some embodiments, the method in step S604 above specifically includes the following steps S701-S702:

[0121] S701. Based on the driving data of adjacent vehicles and the simulated chassis frame data, determine the position information of the four vertices of the adjacent vehicles.

[0122] Optionally, it can be determined first whether adjacent vehicles are within the effective range of the forward-facing camera and LiDAR sensor. This can be done by first determining the coordinates of the furthest point from the target lane line to the target vehicle based on the target map, and then obtaining the distance range of adjacent vehicles as collected by the fusion of the forward-facing camera and LiDAR sensor data. Based on the driving data of adjacent vehicles, it can be determined whether they are within the effective range of the fusion of the forward-facing camera and LiDAR sensor data.

[0123] For example, if the selected target map is a preset map, determine the coordinates (map_posx, map_posy) of the furthest point from the target vehicle in the preset map. Then, based on the chassis frame information of adjacent vehicles, the following formula can be used to determine whether the adjacent vehicles are within the effective range of the fusion acquisition of the forward-looking camera and the LiDAR sensor:

[0124]

[0125] Where length is the length of adjacent vehicles, near_distance is the minimum value of the effective range, and far_distance is the maximum value of the effective range.

[0126] Furthermore, based on the vertical coordinates of the adjacent vehicle's position information, if they are within the range of near_distance and far_distance, it indicates that the adjacent vehicle is within the effective range of the fusion acquisition by the forward-looking camera and the lidar sensor, thereby determining the third line-crossing amount of the adjacent vehicle.

[0127] Optionally, the location of adjacent vehicles can be determined based on the x-coordinate of the location information. If the x-coordinate value is positive, it means that the adjacent vehicle is to the left of the target vehicle; if the x-coordinate value is negative, it means that the adjacent vehicle is to the right of the target vehicle.

[0128] Specifically, based on the driving direction (heading), position information (posx, posy), chassis frame length, and chassis frame width of adjacent vehicles, the position information of the four vertices of adjacent vehicles is determined using the following formula four:

[0129]

[0130] The position information of the front left vertex is (top_left_x, top_left_y), the position information of the front right vertex is (top_right_x, top_right_y), the position information of the back left vertex is (bottom_left_x, bottom_left_y), and the position information of the back right vertex is (bottom_right_x, bottom_right_y).

[0131] S702. Based on the position information of the four vertices and the target lane line in the target map, determine the target distance between adjacent vehicles and the target lane line, and determine the target distance as the third line crossing amount.

[0132] Specifically, based on the position information of the four vertices and the target lane line in the target map, the lateral difference between the position information of the four vertices of adjacent vehicles and the data representing the target lane line is determined, and the largest lateral difference is determined as the target distance, thus determining the third line crossing amount.

[0133] Optionally, based on the location of adjacent vehicles relative to the target vehicle, two vertices can be selected from the position information of the four vertices to determine the lateral difference between the vertex and the target lane line. If the adjacent vehicle is to the left of the target vehicle, the vertices on the front and rear sides to the right of the adjacent vehicle can be selected to calculate the lateral difference between the vertex and the target lane line, thereby reducing the amount of calculation and improving the efficiency of determining the third lane crossing amount.

[0134] Figure 3 This is a flowchart illustrating yet another method for determining lane departure distance according to an exemplary embodiment, such as... Figure 3 As shown, the method in step S205 above specifically includes the following steps S801-S802:

[0135] S801. Input the first pressure line amount, the second pressure line amount, and the third pressure line amount into the preset model, determine multiple confidence levels corresponding to any one of the first pressure line amount, the second pressure line amount, and the third pressure line amount, and determine the target confidence level corresponding to any one of the pressure line amounts based on the multiple confidence levels.

[0136] Optionally, the preset model may include: a sensor module, a variance module, a motion state module, a scene module, a deviation module, and a multi-source data comparison module.

[0137] Optionally, based on the first, second, and third over-the-line amounts obtained in the current period, and the first, second, and third over-the-line amounts of a fixed number of historical periods (e.g., 15 historical periods), outliers are removed through smoothing to obtain the corresponding first, second, and third average over-the-line amounts.

[0138] For example, first obtain the first cross-value of the current period (vision_cross_value), the validity flag (vision_valid), and the data structure (vision_org_value_tracker) that stores the first cross-value and validity flag of up to 15 historical periods.

[0139] Furthermore, the number of elements in `vision_org_value_tracker` is checked. If the number of elements is 15, the oldest data set from the current period (the first cross-value and validity flag of the 15th historical period) is deleted. If the number of elements is less than 15, no operation is performed. Then, the first cross-value (`vision_cross_value`) and the validity flag (`vision_valid`) of the current period are inserted at the end of `vision_org_value_tracker`.

[0140] Furthermore, iterate through all elements in vision_org_value_tracker from the beginning, record the number of valid data with the valid flag vision_valid set to 1, denoted as valid_counter, and accumulate the vision_cross_value corresponding to the valid data, denoted as valid_cross_value.

[0141] Furthermore, if valid_counter is greater than 0, it means that at least one vision_cross_value is valid within the 15 periods. The valid_cross_value is then divided by valid_counter to obtain the smoothed first average crossing value, vision_avg_cross_value. Similarly, the smoothed second average crossing value, lidar_avg_cross_value, and the smoothed third average crossing value, fusion_avg_cross_value, can be obtained.

[0142] Optionally, the variances of the first, second, and third over-the-line amounts, with the valid flag set to 1, can be determined using the first, second, and third over-the-line amounts in the current period, as well as the first, second, and third over-the-line amounts in a fixed number of historical periods (e.g., 15 historical periods).

[0143] Specifically, the sensor module works as follows: First, the first, second, and third line-crossing measurements are input into the sensor module to determine the target vehicle's acquisition status of adjacent vehicles over a period of time. This determines the confidence coefficient corresponding to any line-crossing measurement, and based on the confidence weight of the sensor module, the sensor confidence level corresponding to any line-crossing measurement is determined. If the target vehicle's forward-facing camera and LiDAR sensor are both visible, the confidence coefficients for the three line-crossing measurements are relatively high (e.g., 1). If the target vehicle is only visible through the forward-facing camera and not through the LiDAR sensor, the confidence coefficient for the second line-crossing measurement is set to 0, and the confidence coefficients corresponding to the first and third line-crossing measurements are relatively high (e.g., 1). If the target vehicle is only visible through the LiDAR sensor and not through the forward-facing camera, the confidence coefficient for the first line-crossing measurement is set to 0. If the selected target map is a preset map, the confidence coefficients corresponding to the second and third line-crossing measurements are relatively high (e.g., 1). If neither the target vehicle's forward-facing camera nor the LiDAR sensor is visible, the confidence coefficients for the three line-crossing measurements are 0.

[0144] Furthermore, the variance module works as follows: The first, second, and third pressure points are input into the variance module. Based on the magnitude of the variances of the three pressure points in the current period, a confidence coefficient is determined for each pressure point. This coefficient is then multiplied by the confidence weight of the variance module to determine the variance confidence level for each pressure point. Specifically, the pressure point with the largest variance indicates the least stable pressure point data, thus receiving a smaller confidence coefficient (e.g., 0.6); the pressure point with the smallest variance indicates the most stable pressure point data, thus receiving a larger confidence level (e.g., 1); and pressure points with intermediate variances receive a moderate confidence coefficient (e.g., 0.8).

[0145] Furthermore, the motion state module specifically works as follows: For adjacent vehicles in motion, based on their driving direction, offset speed, and offset distance, it determines three dimensions: the driving direction, offset speed direction, and offset distance direction of the adjacent vehicles, and verifies the direction of change of the line crossing amount. If all three dimensions are consistent with the direction of change of the line crossing amount, then each line crossing amount is multiplied by a large confidence coefficient (e.g., 1) and the confidence weight of the motion state model; if only two dimensions are consistent with the direction of change of the line crossing amount, then the motion state confidence of the corresponding line crossing amount is multiplied by a medium confidence coefficient (e.g., 0.8) and the confidence weight of the motion state model; if only one dimension is consistent with the direction of change of the line crossing amount, then the motion state confidence of the corresponding line crossing amount is multiplied by a small confidence coefficient (e.g., 0.4) and the confidence weight of the motion state model; if any dimension is inconsistent with the direction of change of the line crossing amount, then the confidence coefficient of the motion state module is 0, and the confidence of the corresponding motion state model is 0.

[0146] Furthermore, the scenario module specifically involves setting different confidence coefficients for different scenarios of the target vehicle and adjacent vehicles. When multiple scenarios are superimposed, the different confidence coefficients are multiplied together to obtain the final confidence coefficient, which is then multiplied by the preset confidence weight of the scenario module to obtain the confidence of the scenario module for any line crossing amount.

[0147] Furthermore, the deviation module works as follows: Based on the difference between the current and previous periods' values ​​of any line-crossing quantity, if the difference is less than a preset line-crossing threshold, the current period's line-crossing quantity is considered valid, and the valid flag is set to 1. The differences between the three line-crossing quantity values ​​are then sorted. The smallest difference indicates less data fluctuation, resulting in a higher confidence coefficient for the corresponding deviation model (e.g., 1). A medium difference indicates moderate data fluctuation, also resulting in a medium confidence coefficient for the corresponding deviation model (e.g., 0.8). The largest difference indicates greater data fluctuation, resulting in a lower confidence coefficient for the corresponding deviation model (e.g., 0.6).

[0148] Furthermore, the multi-source data comparison module works as follows: Since the first line-crossing measurement is directly fitted by the forward-facing camera using a deep learning algorithm, while the second and third line-crossing measurements are calculated based on the target lane lines in the target map, the confidence level of the first line-crossing measurement is relatively high. When the second and third line-crossing measurements differ significantly from the first line-crossing measurement, it may be due to inaccurate target lane lines. Therefore, when the confidence level of the first line-crossing measurement is greater than or equal to a preset confidence threshold, the confidence coefficient of the multi-source data comparison module for the first line-crossing measurement is relatively high (e.g., 1). Then, it is determined whether the difference between the second and third average line-crossing measurements and the first average line-crossing measurements is less than a preset comparison threshold (e.g., 0.3). If the difference between the average line-crossing measurement and the first average line-crossing measurement is less than the preset comparison threshold, the confidence coefficient corresponding to that average line-crossing measurement is 0.5; if the difference between the average line-crossing measurement and the first average line-crossing measurement is greater than or equal to the preset comparison threshold, the confidence coefficient corresponding to that average line-crossing measurement is 0.

[0149] Furthermore, the confidence scores of each module in the preset model for any line pressing amount are summed to obtain the target confidence score corresponding to any line pressing amount.

[0150] S802. Based on the first target confidence level corresponding to the first line crossing amount, the second target confidence level corresponding to the second line crossing amount, and the third target confidence level corresponding to the third line crossing amount, the line crossing amount corresponding to the highest target confidence level among the first target confidence level, the second target confidence level, and the third target confidence level is determined as the lane offset distance of the adjacent vehicle.

[0151] Specifically, the first target confidence level of the first line crossing amount, the second target confidence level of the second line crossing amount, and the third target confidence level of the third line crossing amount are judged to determine the effective flag bit as 1. The line crossing amount corresponding to the target confidence level with the largest value is the lane deviation distance, indicating that the line crossing amount data is more reliable and accurate. This improves the accuracy of the line crossing amount of adjacent vehicles corresponding to the target vehicle, and provides accurate data support for subsequent operations such as predicting vehicle intentions and formulating response strategies.

[0152] It should be noted that if the confidence values ​​of the first target, the second target, and the third target are the same, the second line crossing amount with higher data accuracy is selected first as the lane departure distance, followed by the third line crossing amount, and finally the first line crossing amount.

[0153] For example, the confidence weights of the six modules are first defined. The confidence weight of the sensor module (detect_sensor_weight) is 1, the confidence weight of the variance module (variance_weight) is 0.8, the confidence weight of the motion state module (orient_weight) is 1, the confidence weight of the scene module (range_weight) is 1.2, the confidence weight of the bias module (diff_weight) is 0.8, and the confidence weight of the multi-source data comparison module (compare_weight) is 1.

[0154] Furthermore, if the current acquisition status of the target vehicle's forward-facing camera and LiDAR sensor is visible, but the historical information of the sensors indicates that the acquisition status of the target vehicle's forward-facing camera was invisible during the historical period, while the acquisition status of the LiDAR sensor was always visible, then the confidence coefficients corresponding to the first and third line-crossing measurements acquired by the forward-facing camera are relatively small, while the confidence coefficient corresponding to the second line-crossing measurement acquired by the LiDAR sensor is relatively large. Therefore, the confidence coefficient for the first line-crossing measurement is 0.8, the confidence coefficient for the second line-crossing measurement is 1, and the confidence coefficient for the third line-crossing measurement is 0.8. Thus, the confidence level corresponding to any line-crossing measurement in the sensor module can be determined using the following formula five:

[0155]

[0156] Therefore, it can be determined that after passing through the sensor module, the confidence level of the first wire pressing quantity (vision_confidenc) is 0.8, the confidence level of the second wire pressing quantity (lidar_confidenc) is 1, and the confidence level of the third wire pressing quantity (fusion_confidenc) is 0.8.

[0157] Furthermore, comparing the variances of the three line-pressing amounts, we find that the variance of the third line-pressing amount is greater than that of the first line-pressing amount, and the variance of the first line-pressing amount is greater than that of the second line-pressing amount, i.e., fusion_variance > vision_variance > lidar_variance. Therefore, the confidence coefficient of the first line-pressing amount is 0.8, the confidence coefficient of the second line-pressing amount is 1, and the confidence coefficient of the third line-pressing amount is 0.6. Thus, the confidence level of any line-pressing amount after inputting the variance module can be determined using the following formula (Formula 6):

[0158]

[0159] Therefore, it can be determined that after passing through the variance module, the confidence level of the first line pressing quantity (vision_confidenc) is 1.44, the confidence level of the second line pressing quantity (lidar_confidenc) is 1.8, and the confidence level of the third line pressing quantity (fusion_confidenc) is 1.28.

[0160] Furthermore, based on the driving data of adjacent vehicles, the driving direction, vehicle offset speed direction, and offset distance direction of adjacent vehicles in the current cycle are determined. It is found that the driving direction and vehicle offset speed direction indicate that adjacent vehicles tend to move closer to the target vehicle, while the offset distance direction indicates a tendency to move away from the target vehicle. Then, the line-crossing amount in the current cycle is subtracted from the line-crossing amount in the previous cycle. If the difference between the first and second line-crossing amounts is positive, and the difference between the third line-crossing amounts is negative, it indicates that the first and second line-crossing amounts represent a tendency for adjacent vehicles to move closer to the target vehicle, and the corresponding confidence coefficient for the first and second line-crossing amounts is 0.8. The third line-crossing amount represents a tendency for adjacent vehicles to move away from the target vehicle, and the corresponding confidence coefficient for the third line-crossing amount is 0.4. Further, the difference in the third line-crossing amounts is verified to be within a tolerable fluctuation range. Therefore, the confidence level of any line-crossing amount after inputting into the motion state module can be determined using the following formula seven:

[0161]

[0162] Therefore, it can be determined that after passing through the motion state module, the confidence level of the first line pressing quantity (vision_confidenc) is 2.24, the confidence level of the second line pressing quantity (lidar_confidenc) is 2.6, and the confidence level of the third line pressing quantity (fusion_confidenc) is 1.68.

[0163] Furthermore, if the current driving mode is pilot mode, and the vehicle is traveling on a straight road, the target vehicle is not changing lanes, and an adjacent vehicle is accelerating to overtake the target vehicle, therefore the forward-facing camera cannot capture the complete adjacent vehicle. In this case, the confidence coefficient for the first line-crossing amount is 0.8, and the confidence coefficients for the second and third line-crossing amounts are 1. Therefore, the confidence level of any line-crossing amount after inputting the scene module can be determined using the following formula eight:

[0164]

[0165] Therefore, it can be determined that after passing the scene module, the confidence level of the first line-pressing quantity is 3.2 (vision_confidenc), the confidence level of the second line-pressing quantity is 3.8 (lidar_confidenc), and the confidence level of the third line-pressing quantity is 2.88 (fusion_confidenc).

[0166] Furthermore, after confirming that the difference between the average line-pressing values ​​of the three average line-pressing values ​​in the current period and the previous period does not exceed the preset line-pressing value threshold, the differences between the average line-pressing values ​​of the three average line-pressing values ​​in the current period and the previous period are compared. The order of the differences of the three line-pressing value values ​​is as follows: the difference of the third average line-pressing value value is greater than the difference of the first average line-pressing value value, and the difference of the first average line-pressing value value is greater than the difference of the second average line-pressing value value. Therefore, the confidence coefficient of the first line-pressing value is 0.8, the confidence coefficient of the second line-pressing value is 1, and the confidence coefficient of the third line-pressing value is 0.6. Therefore, the confidence of any line-pressing value after inputting the deviation module can be determined by the following formula nine:

[0167]

[0168] Therefore, it can be determined that after passing through the deviation module, the confidence level of the first creasing amount (vision_confidenc) is 3.84, the confidence level of the second creasing amount (lidar_confidenc) is 4.6, and the confidence level of the third creasing amount (fusion_confidenc) is 3.36.

[0169] Furthermore, since the confidence level of the first threshold quantity is 3.84, which equals the preset confidence threshold (the sum of the confidence weights of the first five modules is 4.8, the preset ratio is 80%, so the preset confidence threshold is 3.84), the confidence level of the first threshold quantity meets the preset confidence threshold. Calculating the current period, the difference between the second average threshold quantity and the first average threshold quantity is less than the preset comparison threshold, while the difference between the third average threshold quantity and the first average threshold quantity is greater than the preset comparison threshold. Therefore, the confidence coefficient of the first threshold quantity is 1, the confidence coefficient of the second threshold quantity is 0.5, and the confidence coefficient of the third threshold quantity is 0. Thus, the confidence level of any threshold quantity after inputting into the multi-source data comparison module can be determined using the following formula:

[0170]

[0171] Therefore, it can be determined that after passing through the multi-source data comparison module, that is, after finally passing through the preset model, the target confidence level of the first line-crossing quantity is 4.84, the target confidence level of the second line-crossing quantity is 5.1, and the target confidence level of the third line-crossing quantity is 3.36.

[0172] Furthermore, based on the target confidence scores of 4.84 for the first line crossing amount, 5.1 for the second line crossing amount, and 3.36 for the third line crossing amount, the line crossing amount corresponding to the highest target confidence score can be determined as the lane deviation distance, that is, the smoothed third average line crossing amount can be used as the lane deviation distance.

[0173] It should be noted that if the effective flag of a certain line crossing is 0, the confidence level is not calculated and cannot be used as the lane deviation distance; or the calculated confidence level is subtracted from the sum of the weights of all modules to ensure that the target confidence level is negative.

[0174] This application provides a method for determining lane departure distance. First, it acquires visual sensor data, lidar sensor data, positioning information, and a preset map of a target vehicle. Then, it generates a visual map from the visual sensor data and, based on the positioning information, determines the target map from the visual map and the preset map. Next, it determines the first line crossing amount of adjacent vehicles corresponding to the target vehicle based on the visual sensor data, the second line crossing amount based on the lidar sensor data, and the third line crossing amount based on the visual sensor data and the lidar sensor data. Further, a preset model is used to determine the lane departure distance from the first, second, and third line crossing amounts. This method compares and judges the preset map and the visual map determined by the visual sensors to select the target map, thereby obtaining an accurate lane departure distance for adjacent vehicles. Furthermore, from multiple line crossing amounts, based on various judgment criteria of the preset model, one line crossing amount with higher accuracy is determined as the lane departure distance, improving the accuracy of determining whether the driving position of adjacent vehicles is on the line and obtaining an accurate lane departure distance for adjacent vehicles.

[0175] The foregoing mainly describes the solutions provided by the embodiments of this application from a methodological perspective. To achieve the above functions, the lane departure distance determination device or electronic device includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0176] This application embodiment can, based on the above method, exemplarily divide the lane departure distance determination device or electronic device into functional modules. For example, the lane departure distance determination device or electronic device may include functional modules corresponding to each functional division, or two or more functions may be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module. It should be noted that the module division in this application embodiment is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0177] Figure 4 This is a block diagram illustrating a lane departure distance determination device according to an exemplary embodiment. (Refer to...) Figure 4The lane departure distance determination device 40 includes: an acquisition module 401 and a processing module 402; the acquisition module 401 is used to acquire sensor data and positioning information of the target vehicle, as well as a preset map of the target vehicle in the target area, the sensor data including: visual sensor data and lidar sensor data, the preset map including lane line data information; the processing module 402 is used to generate a visual map based on the visual sensor data, and determine the target map from the visual map and the preset map based on the positioning information, the target map including lane line data information of the target area where the target vehicle is located; the processing module 402 is also used to determine the first, second, and third lane crossing amounts of adjacent vehicles corresponding to the target vehicle, the first lane crossing amount is determined based on the visual sensor data and lane line data information in the target map, the second lane crossing amount is determined based on the lidar sensor data and lane line data information in the target map, and the third lane crossing amount is determined based on the visual sensor data, lidar sensor data, and lane line data information in the target map; the processing module 402 is also used to determine the lane departure distance of adjacent vehicles based on the first, second, and third lane crossing amounts using a preset model.

[0178] In one possible implementation, the processing module 402 is specifically used to determine the first lane line point set data corresponding to the visual map and the second lane line point set data corresponding to the preset map; the processing module 402 is specifically used to determine the target map from the visual map and the preset map based on the amount of data included in the first lane line point set data and the amount of data included in the second lane line point set data.

[0179] In one possible implementation, the processing module 402 is specifically used to determine the map corresponding to the second target point set data as the target map when it is determined that the amount of data included in the first target point set data is less than the preset amount of data and the amount of data included in the second target point set data is greater than or equal to the preset amount of data. The first target point set data is either the first lane line point set data or the second lane line point set data, and the second target point set data is either the first lane line point set data or the second lane line point set data. The first target point set data and the second target point set data are different point set data.

[0180] In one possible implementation, the processing module 402 is specifically used to determine the deviation between the lane lines in the visual map and the lane lines in the preset map when the amount of data included in the first lane line point set data and the amount of data included in the second lane line point set data are both greater than or equal to a preset amount of data; the processing module 402 is specifically used to determine the preset map as the target map when the deviation is less than a preset deviation threshold.

[0181] In one possible implementation, processing module 402 is specifically used to determine the driving data, simulated chassis frame data, and laser point cloud convex hull data of adjacent vehicles based on visual sensor data and lidar sensor data. The driving data includes at least one of the following: driving direction, position information, vehicle offset speed, and offset distance. Processing module 402 is specifically used to determine the first lane crossing amount of adjacent vehicles based on visual sensor data and target lane lines in the target map, where the target lane lines are the lane lines between the target vehicle and adjacent vehicles. Processing module 402 is specifically used to determine the second lane crossing amount of adjacent vehicles based on laser point cloud convex hull data and target lane lines in the target map. Processing module 402 is specifically used to determine the third lane crossing amount of adjacent vehicles based on driving data, simulated chassis frame data, and target lane lines in the target map.

[0182] In one possible implementation, the processing module 402 is specifically used to determine the position information of the four vertices of the adjacent vehicles based on the driving data of the adjacent vehicles and the simulated chassis frame data; the processing module 402 is specifically used to determine the target distance between the adjacent vehicles and the target lane line based on the position information of the four vertices and the target lane line in the target map, and to determine the target distance as the third line crossing amount.

[0183] In one possible implementation, the preset model includes at least one of the following: a sensor module, a variance module, a motion state module, a scene module, a deviation module, and a multi-source data comparison module; the processing module 402 is specifically used to input the first line crossing amount, the second line crossing amount, and the third line crossing amount into the preset model, determine multiple confidence levels corresponding to any one of the first line crossing amount, the second line crossing amount, and the third line crossing amount, and determine the target confidence level corresponding to any one of the multiple confidence levels; the processing module 402 is specifically used to determine the line crossing amount corresponding to the largest target confidence level among the first target confidence level, the second target confidence level, and the third target confidence level as the lane offset distance of the adjacent vehicle, based on the first target confidence level corresponding to the first line crossing amount, the second target confidence level corresponding to the second line crossing amount, and the third target confidence level corresponding to the third line crossing amount.

[0184] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0185] Figure 5 This is a block diagram illustrating an electronic device according to an exemplary embodiment. Figure 5 As shown, the electronic device 50 includes, but is not limited to, a processor 501 and a memory 502.

[0186] The memory 502 described above is used to store the executable instructions of the processor 501. It is understood that the processor 501 is configured to execute instructions to implement the lane departure distance determination method in the above embodiment.

[0187] It should be noted that those skilled in the art will understand that Figure 5 The electronic device structure shown does not constitute a limitation on the electronic device; the electronic device may include, but is not limited to, other electronic devices. Figure 5 This may indicate more or fewer components, or a combination of certain components, or a different arrangement of components.

[0188] Processor 501 is the control center of the electronic device. It connects various parts of the electronic device via various interfaces and lines. By running or executing software programs and / or modules stored in memory 502, and by calling data stored in memory 502, it performs various functions and processes data, thereby providing overall monitoring of the electronic device. Processor 501 may include one or more processing modules. Optionally, processor 501 may integrate an application processor and a modem processor. The application processor mainly handles the operating system, user interface, and applications, while the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into processor 501.

[0189] The memory 502 can be used to store software programs and various data. The memory 502 may primarily include a program storage area and a data storage area. The program storage area may store the operating system and application programs required by at least one functional module (such as an acquisition unit, a determination unit, a processing unit, etc.). Furthermore, the memory 502 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0190] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory 502 including instructions, which can be executed by a processor 501 of an electronic device 50 to implement the lane offset distance determination method in the above embodiments.

[0191] In actual implementation, Figure 4 The functions of the acquisition module 401 and the processing module 402 can both be provided by Figure 5 The processor 501 calls the computer program stored in the memory 502 to implement the process. The specific execution process can be found in the description of the lane offset distance determination method in the previous embodiment, and will not be repeated here.

[0192] Optionally, the computer-readable storage medium may be a non-transitory computer-readable storage medium, such as a read-only memory (ROM), random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device.

[0193] In an exemplary embodiment, a vehicle including a lane departure distance determination device is also provided, which can perform the lane departure distance determination method in the above embodiments by means of the lane departure distance determination device.

[0194] In an exemplary embodiment, this application also provides a computer program product including one or more instructions, which can be executed by a processor 501 of an electronic device to complete the method for determining lane deviation distance in the above embodiments.

[0195] It should be noted that when one or more instructions in the computer-readable storage medium or computer program product are executed by the processor of an electronic device, they implement the various processes of the above-described method for determining lane offset distance and achieve the same technical effect as the above-described method for determining lane offset distance. To avoid repetition, they will not be described again here.

[0196] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0197] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another apparatus, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0198] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the classified units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0199] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0200] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, essentially, or the part that contributes to the prior art, or a complete or partial classification of the technical solution, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0201] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for determining lane departure distance, characterized in that, The method includes: Acquire sensor data and positioning information of the target vehicle, as well as a preset map of the target vehicle in the target area. The sensor data includes visual sensor data and lidar sensor data, and the preset map includes lane line data information. A visual map is generated based on the visual sensor data, and a target map is determined from the visual map and the preset map based on the positioning information. The target map includes lane line data information of the target area where the target vehicle is located. The first, second, and third lane crossing amounts of adjacent vehicles corresponding to the target vehicle are determined. The first lane crossing amount is determined based on visual sensor data and lane line data information in the target map. The second lane crossing amount is determined based on lidar sensor data and lane line data information in the target map. The third lane crossing amount is determined based on visual sensor data, lidar sensor data, and lane line data information in the target map. Based on the first line crossing amount, the second line crossing amount, and the third line crossing amount, the lane offset distance of the adjacent vehicles is determined.

2. The method according to claim 1, characterized in that, Determining the target map from the visual map and the preset map based on the positioning information includes: Determine the first lane line point set data corresponding to the visual map and the second lane line point set data corresponding to the preset map; The target map is determined from the visual map and the preset map based on the amount of data included in the first lane line point set data and the amount of data included in the second lane line point set data.

3. The method according to claim 2, characterized in that, Determining the target map based on the amount of data included in the first lane line point set data and the amount of data included in the second lane line point set data includes: If the amount of data included in the first target point set is less than the preset amount of data, and the amount of data included in the second target point set is greater than or equal to the preset amount of data, the map corresponding to the second target point set is determined as the target map. The first target point set is either the first lane line point set or the second lane line point set, and the second target point set is either the first lane line point set or the second lane line point set. The first target point set and the second target point set are different point sets.

4. The method according to claim 2, characterized in that, Determining the target map based on the amount of data included in the first lane line point set data and the amount of data included in the second lane line point set data includes: If the amount of data included in the first lane line point set data and the amount of data included in the second lane line point set data are both greater than or equal to a preset amount of data, the deviation between the lane lines in the visual map and the lane lines in the preset map is determined. If the deviation is less than a preset deviation threshold, the preset map is determined as the target map.

5. The method according to claim 1, characterized in that, Determining the first, second, and third line-crossing amounts of adjacent vehicles corresponding to the target vehicle includes: Based on the visual sensor data and the lidar sensor data, the driving data, simulated chassis frame data and lidar point cloud convex hull data of the adjacent vehicles are determined. The driving data includes at least one of the following: driving direction, position information, vehicle offset speed and offset distance. Based on the visual sensor data and the target lane line in the target map, the first line crossing amount of the adjacent vehicle is determined, where the target lane line is the lane line between the target vehicle and the adjacent vehicle; Based on the laser point cloud convex hull data and the target lane lines in the target map, the second line crossing amount of the adjacent vehicles is determined; Based on the driving data, the simulated chassis frame data, and the target lane lines in the target map, the third lane crossing amount of the adjacent vehicles is determined.

6. The method according to claim 5, characterized in that, The step of determining the third lane overlap of the adjacent vehicles based on the driving data, the simulated chassis frame data, and the target lane lines in the target map includes: Based on the driving data of the adjacent vehicles and the simulated chassis frame data, the position information of the four vertices of the adjacent vehicles is determined; Based on the position information of the four vertices and the target lane line in the target map, the target distance between the adjacent vehicle and the target lane line is determined, and the target distance is determined as the third lane crossing amount.

7. The method according to any one of claims 1-6, characterized in that, Determining the lane departure distance of the adjacent vehicles based on the first lane departure amount, the second lane departure amount, and the third lane departure amount includes: The first pressure line amount, the second pressure line amount, and the third pressure line amount are input into a preset model to determine multiple confidence levels corresponding to any one of the first pressure line amount, the second pressure line amount, and the third pressure line amount, and the target confidence level corresponding to any one of the pressure line amounts is determined based on the multiple confidence levels. Based on the first target confidence level corresponding to the first line crossing amount, the second target confidence level corresponding to the second line crossing amount, and the third target confidence level corresponding to the third line crossing amount, the line crossing amount corresponding to the largest target confidence level among the first target confidence level, the second target confidence level, and the third target confidence level is determined as the lane departure distance of the adjacent vehicle.

8. A device for determining lane departure distance, characterized in that, The lane departure distance device includes an acquisition module and a processing module; The acquisition module is used to acquire sensor data and positioning information of the target vehicle, as well as a preset map of the target vehicle in the target area. The sensor data includes visual sensor data and lidar sensor data, and the preset map includes lane line data information. The processing module is used to generate a visual map based on the visual sensor data, and to determine a target map from the visual map and the preset map based on the positioning information. The target map includes lane line data information of the target area where the target vehicle is located. The processing module is further configured to determine a first line crossing amount, a second line crossing amount, and a third line crossing amount for adjacent vehicles corresponding to the target vehicle. The first line crossing amount is determined based on visual sensor data and lane line data information in the target map. The second line crossing amount is determined based on lidar sensor data and lane line data information in the target map. The third line crossing amount is determined based on visual sensor data, lidar sensor data, and lane line data information in the target map. The processing module is further configured to determine the lane offset distance of the adjacent vehicle based on the first line-crossing amount, the second line-crossing amount, and the third line-crossing amount.

9. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, When the computer-executable instructions stored in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device is capable of performing the method as described in any one of claims 1 to 7.

11. A vehicle, characterized in that, The vehicle includes the lane departure distance determination device as described in claim 8, and the vehicle is used to implement the method as described in any one of claims 1 to 7.

Images