Visibility determination method, apparatus, vehicle, and computer-readable storage medium
By identifying lane lines in road images and ignoring data obstructed by obstacles, visibility is assessed using the farthest endpoint of the lane lines, thus solving the problem of strong randomness in visibility assessment in existing technologies and achieving high-precision assessment on various roads.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- ZHEJIANG ZEEKR INTELLIGENT TECH CO LTD
- Filing Date
- 2026-04-28
- Publication Date
- 2026-07-28
AI Technical Summary
In existing technologies, visibility assessments relying on preset reference targets are highly random, especially in rural roads without fixed targets or urban roads where signs are obscured, making the assessment difficult and inaccurate.
Using lane lines as the core reference target, visibility is assessed by identifying the position of lane lines in road images and ignoring data obscured by obstacles, re-acquiring images until all lane lines are unobstructed, and determining the position of the farthest endpoint.
It reduces the difficulty of visibility assessment and ensures the accuracy of the assessment, especially on roads without fixed destinations, thus improving the accuracy of visibility assessment.
Smart Images
Figure CN122473764A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of visibility assessment, and more particularly to a method, apparatus, vehicle, and computer-readable storage medium for determining visibility. Background Technology
[0002] Related technologies often rely on some preset reference targets on the road (such as vehicles, traffic signs, 100-meter markers, street lamp poles, etc.) to assess visibility. By calculating the distance of the preset reference targets relative to the vehicle itself, the visibility is determined by the distance of the farthest identifiable target.
[0003] However, the distribution of the aforementioned preset reference targets is highly random (e.g., some rural roads have no traffic signs, and signs on urban roads are easily obscured by trees and buildings). If the preset reference targets do not appear on the road, it is difficult to assess visibility and the accuracy is low. Summary of the Invention
[0004] To address the aforementioned technical problems, this application provides a visibility determination method, a visibility determination device, a vehicle, and a computer-readable storage medium. The technical solution is as follows: According to a first aspect of this application, a visibility determination method is provided, the method comprising: Acquire a road image, the road image containing at least one lane line; Based on the acquired road images, the location of at least one lane line is identified, and the location of obstacles on the road is also identified; If at least a portion of the position of at least one lane line is covered by the position of the obstacle, ignore the data corresponding to the positions of each lane line contained in the road and re-acquire the road image until the positions of each lane line contained in the road are not covered by the position of the obstacle. If the positions of all lane lines on the road are not covered by the positions of the obstacles, determine the position of the farthest endpoint of each lane line in the road image, and determine the visibility based on the position of the farthest endpoint.
[0005] According to a second aspect of this application, a visibility determining device is provided, the device comprising: An acquisition unit is used to acquire a road image, wherein the road image contains at least one lane line; The identification unit is used to identify the position of at least one lane line based on the acquired road image, and to identify the position of obstacles on the road; The acquisition unit is further configured to, when at least a portion of the position of the at least one lane line is covered by the position of the obstacle, ignore the data corresponding to the position of each lane line contained in the road and re-acquire the road image until the position of each lane line contained in the road is not covered by the position of the obstacle. The determining unit is configured to determine the position of the farthest endpoint of each lane line in the road image when the positions of each lane line contained in the road are not covered by the position of the obstacle, and to determine the visibility based on the position of the farthest endpoint.
[0006] According to a third aspect of this application, a vehicle is provided, the vehicle comprising: processor; Memory used to store processor-executable instructions; The processor is configured to implement the method as described in the first aspect.
[0007] According to a fourth aspect of this application, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the method as described in the first aspect.
[0008] The technical solution provided in this application uses lane lines as the core reference target. Lane lines are an essential element of road infrastructure, covering all levels of roads (including rural roads without fixed targets), and have the characteristics of strong continuity and high visibility, thus avoiding the problem of "no reference target available" in visibility assessment as much as possible. Visibility is assessed by utilizing the position of the farthest visible endpoint of the lane line in the image, reducing the difficulty of assessment while ensuring its accuracy.
[0009] While identifying lane line positions using the acquired road images, obstacles on the road are also identified. If an obstacle covers (i.e. occludes) any lane line, the identification result for that lane line is considered inaccurate, and the visibility assessment based on the farthest endpoint of the lane line at that time is considered to be of low accuracy (not meeting the requirements). In this case, the data corresponding to the positions of each lane line on the road are ignored, and road images are re-acquired until the positions of each lane line on the road are not covered by obstacles.
[0010] Only when the positions of all lane lines on the road are not covered by obstacles is it considered that the visibility assessment based on the position of the farthest endpoint of the lane line on the road at this time is accurate enough. In this case, determining the position of the farthest endpoint of each lane line in the road image and determining the visibility based on the position of the farthest endpoint can further ensure the accuracy of the visibility assessment.
[0011] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description
[0012] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings.
[0013] Figure 1 This is a schematic diagram of a visibility assessment scenario in related technologies; Figure 2 This is a flowchart illustrating a visibility determination method according to an embodiment of this application; Figure 3 This is a schematic diagram of the structure of a BEVFormer model according to an embodiment of this application; Figure 4 This is a schematic diagram of a visibility determination scenario according to an embodiment of this application; Figure 5 This is a schematic diagram of a visibility determination scenario according to another embodiment of this application; Figure 6 This is a schematic diagram of a visibility determination scenario according to another embodiment of this application; Figure 7 This is a schematic diagram of a visibility determination scenario according to another embodiment of this application; Figure 8 This is a schematic diagram of the visibility determination device according to an embodiment of this application; Figure 9 This is a schematic diagram of the structure of a vehicle according to an embodiment of this application. Detailed Implementation
[0014] To enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art should fall within the scope of protection of this application.
[0015] Visibility, as a core indicator of atmospheric transparency, directly determines the visual perception range of road traffic and is a key environmental factor affecting traffic safety and efficiency.
[0016] In low-visibility scenarios such as fog, rain, dust storms, and low light at night, drivers' visibility is limited, vehicle braking distance is increased, and the incidence of traffic accidents such as rear-end collisions and side collisions rises significantly. Therefore, achieving real-time, accurate, and low-cost visibility estimation in road scenarios is of great practical significance for ensuring traffic safety and improving the operational efficiency of the transportation system.
[0017] Traditional visibility monitoring technologies mainly rely on specialized meteorological instruments, such as transilluminators and scatterometers. While these devices offer high accuracy, they are expensive, difficult to deploy, have limited coverage, and poor environmental adaptability, making them unsuitable for continuous, real-time visibility monitoring of large-scale road networks (such as highway networks and urban arterial road networks).
[0018] See Figure 1 As shown, even visibility assessment methods in related technologies that do not rely on professional meteorological instruments often depend on some pre-set reference targets on the road (such as other vehicles besides the vehicle itself, traffic signs, 100-meter markers, streetlights, etc.) for visibility assessment. Visibility is determined by calculating the distance of the pre-set reference targets relative to the vehicle, and then using the distance of the farthest identifiable target to determine the visibility, thereby reducing the cost of visibility assessment. Specifically, by using the known ratio between the actual size of the pre-set reference targets and the pixel size in the image, combined with a perspective model, the target distance is calculated, and then the distance of the farthest identifiable target among the pre-set reference targets is used as the visibility.
[0019] However, the distribution of the aforementioned preset reference targets is highly random (e.g., some rural roads have no traffic signs, and signs on urban roads are easily obscured by trees and buildings). If the preset reference targets do not appear on the road, it is difficult to assess visibility and the accuracy is low.
[0020] It is worth noting that the above description of visibility assessment scenarios in related technologies is only an illustrative example. In practical applications, other assessment scenarios may exist, and no specific limitations are made for them.
[0021] To address the aforementioned problems, this application provides a visibility determination method applicable to vehicles, which can reduce the difficulty of visibility assessment while ensuring assessment accuracy. Figure 2 As shown, the method includes the following steps: S201. Acquire the collected road images.
[0022] The road image contains at least one lane line.
[0023] S202. Based on the acquired road images, identify the position of at least one lane line and the position of obstacles on the road.
[0024] S203. If at least a portion of the position of at least one lane line is covered by the position of the obstacle, ignore the data corresponding to the positions of each lane line contained in the road and re-acquire the road image until the positions of each lane line contained in the road are not covered by the position of the obstacle.
[0025] S204. If the positions of all lane lines on the road are not covered by the positions of the obstacles, determine the position of the farthest endpoint of each lane line in the road image, and determine the visibility based on the position of the farthest endpoint.
[0026] The technical solution provided in this application uses lane lines as the core reference target. Lane lines are an essential element of road infrastructure, covering all levels of roads (including rural roads without fixed targets), and have the characteristics of strong continuity and high visibility, thus avoiding the problem of "no reference target available" in visibility assessment as much as possible. Visibility is assessed by using the position of the farthest visible endpoint of the lane line in the image, reducing the difficulty of assessment while ensuring its accuracy.
[0027] While identifying lane line positions using the acquired road images, obstacles on the road are also identified. If an obstacle covers (i.e. occludes) any lane line, the identification result for that lane line is considered inaccurate, and the visibility assessment based on the farthest endpoint of the lane line at that time is considered to be of low accuracy (not meeting the requirements). In this case, the data corresponding to the positions of each lane line on the road are ignored, and road images are re-acquired until the positions of each lane line on the road are not covered by obstacles.
[0028] Only when the positions of all lane lines on the road are not covered by obstacles is it considered that the visibility assessment based on the position of the farthest endpoint of the lane line on the road at this time is accurate enough. In this case, determining the position of the farthest endpoint of each lane line in the road image and determining the visibility based on the position of the farthest endpoint can further ensure the accuracy of the visibility assessment.
[0029] The road image containing at least one lane line can be acquired in a variety of ways. As an example, the road image can be acquired using a camera configured on the exterior of the vehicle.
[0030] There are various specific implementations for the arrangement of cameras for capturing vehicle images. As an example, the number of cameras configured on the exterior of the vehicle can be at least 6, including at least: a front-view camera, a front-left camera, a front-right camera, a rear-left camera, a rear-right camera, and a rear-view camera.
[0031] As an example, each of the six cameras corresponds to a different coverage area. For instance, the front camera covers the area directly in front of the vehicle at medium to long distances, monitoring vehicles, pedestrians, traffic signs, etc. on the main road; the left-side front camera covers the area on the left front of the vehicle, covering blind spots when turning left or changing lanes, compensating for the insufficient field of view of the front camera; the right-side front camera covers the area on the right front of the vehicle, with the same function as the left-side front camera, covering right turns and the right-side blind spot; the left-side rear camera covers the area on the left rear of the vehicle, monitoring vehicles approaching from the left rear and assisting with lane changes; the right-side rear camera covers the area on the right rear of the vehicle, monitoring the situation on the right rear; and the rear camera covers the area directly behind the vehicle, responsible for scenarios such as reversing and following other vehicles.
[0032] In this embodiment, by arranging at least six cameras, seamless 360-degree surround view coverage around the vehicle can be achieved. Each camera is responsible for one sector, and a certain degree of overlap in the field of view ensures that there are no blind spots. All the image data collected is then provided to the subsequent lane line and obstacle recognition steps.
[0033] It is worth noting that the above description of the camera arrangement for capturing vehicle images is merely an illustrative example. In practical applications, other arrangement scenarios are possible. For instance, in addition to the aforementioned six cameras, a narrow-angle forward-looking telephoto lens or an in-vehicle lens could be added. Therefore, the specific camera arrangement for capturing vehicle images is not limited.
[0034] There are several ways to identify the location of at least one lane line and the location of obstacles on the road based on the acquired road images. As an example, the acquired road images can be input into a visual perception model, which can then use the images to identify the locations of lane lines and obstacles on the road. The visual perception model can then output the locations of the lane lines and obstacles based on the input road images. It is worth noting that the above description of methods for identifying lane lines and obstacles is merely an illustrative example; in practical applications, other identification methods may exist, and no specific limitations are imposed on these methods.
[0035] The aforementioned visual perception model can be pre-built, and it can be constructed in various ways. As an example, a set of sample images can be acquired, which consists of road images containing at least one lane line and obstacles. This sample image set is used as the training dataset to train the visual perception model. The model is trained using the error between the predicted positions of the lane lines and obstacles and their actual positions as the loss function. The actual positions of the lane lines and obstacles are determined using the perception results from a LiDAR system deployed on the vehicle. It is worth noting that the above description of the visual perception model construction method is merely an illustrative example. In practical applications, other construction methods may exist, and no specific limitation is made.
[0036] As an example, the true positions of lane lines and obstacles in the sample image set can be labeled using the aforementioned LiDAR. Specifically, at least six cameras configured outside the vehicle can be used to collect road image data under different visibility conditions. Simultaneously, the LiDAR configured on the vehicle can be used to perceive the true positions (true distances) of lane lines and obstacles in these road images (the LiDAR is only used during the sample data labeling stage; after training the visual perception model, inference can be performed using only the road images collected by the at least six cameras). The true positions of lane lines and obstacles are labeled using the LiDAR perception results. In some embodiments, obstacles (e.g., other vehicles besides this vehicle) in the sample image set can be labeled using 3D target detection, and lane lines can be labeled with coordinate point sets. In other embodiments, the labeling of lane lines can extend to the limit visible in the image (i.e., the edge of visibility, i.e., the farthest identifiable point). In other embodiments, if lane lines in the sample image set are obscured by obstacles (such as vehicles), the obscured lane lines can also be marked. At the same time, obstacles on the road that may obscure lane lines, such as other vehicles besides this vehicle, such as cars and trucks, as well as pedestrians, cyclists, cones, and guardrails, are also marked.
[0037] The aforementioned visual perception model can have various specific implementations. As an example, the visual perception model can be an autonomous driving visual perception (Bird's Eye View Former, BEVFormer) model based on the Transformer architecture. Understandably, the core objective of the BEVFormer model is to generate bird's-eye view features from multi-camera images. Having BEV spatial features makes it easier to extend downstream tasks.
[0038] In some embodiments, the BEVFormer model can be used to extract features from images for lane line and obstacle detection. During the model training phase, sample image sets collected by at least six cameras can be input into the BEVFormer model. The BEVFormer model performs feature extraction to generate BEV spatial features. After extracting the BEV spatial features, the 3D object detection head of the BEVFormer model is used to detect obstacles, and the lane line detection head is used to detect lane lines. In other embodiments, the training data labels used are the 3D obstacle coordinates and lane line coordinates manually labeled using LiDAR in the visual perception model construction method described above.
[0039] The following is combined with Figure 3 The BEVFormer model of the embodiments of this application will be described by way of example: The BEVFormer model mainly includes three core modules: BEV Queries, Spatial Cross-Attention, and Temporal Self-Attention.
[0040] The core function of the BEV Queries module is to serve as the core parameter for learning the BEVFormer model, used to construct a feature grid in the BEV space. An exemplary implementation could be a set of learnable grid-like parameters (e.g., 200x200), with each query point representing a fixed area in the real world (e.g., a 100x100 meter area).
[0041] The core function of the Spatial Cross-Attention module is to aggregate spatial information from multi-camera images. An exemplary implementation can be that each BEV query is projected onto a 2D image through camera parameters, and features are sampled locally in the relevant image region using a deformable attention mechanism to achieve efficient and accurate feature extraction.
[0042] The core function of the Temporal Self-Attention module is to aggregate temporal information from historical BEV features. An exemplary implementation could be aligning the BEV features from the previous moment with the current moment based on vehicle motion. The current BEV query samples from historical features through deformable attention, dynamically fusing historical information, which helps determine the speed of moving objects and identify occluded objects.
[0043] In some embodiments, the BEVFormer model first extracts multi-scale features from multi-camera images through a backbone network. Then, a set of BEV queries repeatedly interacts with a spatial cross-attention layer (extracting current image features) and a temporal self-attention layer (extracting historical BEV features). After iterations through multiple encoder layers, these queries ultimately form a unified BEV feature map containing rich spatiotemporal information. This feature map can be directly input into various task heads, such as 3D detection heads or map segmentation heads (e.g., lane line detection heads and obstacle detection heads in this application), to complete multi-task perception.
[0044] Whether a lane line's position is obscured by an obstacle can be determined in various ways. As an example, based on the acquired road image, a set of coordinate points describing the position of at least one lane line and a set of coordinate points describing the position of an obstacle on the road can be identified. The set of coordinate points describing the position of the at least one lane line is then compared with the set of coordinate points describing the position of the obstacle to determine whether the lane line's position is obscured by the obstacle. Correspondingly, there are also various ways to determine the position of the farthest endpoint of each lane line in the road image and to determine visibility based on the position of the farthest endpoint. As an example, the coordinate points describing the farthest endpoint of each lane line in the road image can be identified from the set of coordinate points describing each lane line, and the distance between the farthest endpoint and the vehicle can be determined based on these coordinate points. Visibility is then determined based on the distance between the farthest endpoint and the vehicle. It is worth noting that the above methods for determining whether a lane line's position is obscured by an obstacle and for determining visibility are merely illustrative examples. In practical applications, other methods may exist, and no specific limitations are imposed on these methods.
[0045] It is understandable that the farthest endpoint of each lane line in the road image can refer to the fact that each lane line has a corresponding farthest endpoint (this point is the farthest point of that lane line). When multiple lane lines are identified, there are multiple farthest endpoints. Among these multiple farthest endpoints, the farthest point is determined, and this point is the farthest endpoint of each lane line in the road image.
[0046] As an example, the acquired road image can be input into the aforementioned visual perception model, so that the visual perception model can use the acquired road image to identify a set of coordinate points describing the position of at least one of the lane lines and a set of coordinate points describing the position of obstacles on the road; the visual perception model can output a set of coordinate points describing the position of lane lines and a set of coordinate points describing the position of obstacles based on the input road image.
[0047] As an example, it's understandable that there are typically at least four lane markings on a regular road. The two lane markings closest to the vehicle's current position can be used as reference points for visibility assessment (i.e., one lane marking to the left and one lane marking to the right of the vehicle). By comparing the 3D coordinates of obstacles with the coordinate points of the lane markings, it's determined whether the current lane marking is obscured by an obstacle. If the obstacle is above the lane marking, it means the obscured lane marking was not predicted (or the prediction result is inaccurate), and the data for that lane marking cannot be used as the basis for visibility assessment.
[0048] Considering that lane-based visibility assessment is most accurate when driving on straight roads, it can be affected when vehicles are traveling on concave or convex surfaces. For example, when crossing an arched bridge, the visible road surface area in the image decreases, and the farthest endpoint of the lane line often fails to reflect the true visibility, potentially leading to an underestimation of visibility. Furthermore, in uphill and downhill scenarios, lane detection issues caused by changes in vehicle posture manifest as perspective distortion: when going uphill, lane lines appear as an inverted trapezoid (narrower at the top and wider at the bottom), and when going downhill, they appear as a trapezoid (wider at the top and narrower at the bottom).
[0049] Based on this, as an example, the aforementioned vehicle can be equipped with an attitude sensor to detect the vehicle's attitude; the attitude data of the vehicle collected by the attitude sensor can be acquired; based on the acquired attitude data, it can be determined whether the vehicle is traveling on a straight road; if it is determined that the vehicle is not traveling on a straight road, the data corresponding to the positions of each lane line on the road can be ignored and the road image can be re-acquired until it is determined that the vehicle is traveling on a straight road; only when it is determined that the positions of each lane line on the road are not covered by the positions of obstacles, and it is determined that the vehicle is traveling on a straight road, can the position of the farthest endpoint of each lane line in the road image be determined, and the visibility can be determined based on the position of the farthest endpoint.
[0050] Understandably, the attitude sensor at this time is used as a road condition sensor to assess the current vehicle condition, thereby excluding lane line data corresponding to uneven road conditions.
[0051] The aforementioned attitude sensor can be implemented in various ways. As an example, considering that the center of gravity of a vehicle often changes when going uphill or downhill, the attitude sensor can include sensors such as an inertial measurement unit (IMU). These sensors can acquire the vehicle's pitch angle and roll angle in real time to determine whether the vehicle is going uphill or downhill, or other non-straight road scenarios. This allows the system to determine whether it is necessary to ignore the data corresponding to the positions of the lane lines on the road and re-acquire road images until it is determined that the vehicle is traveling on a straight road.
[0052] Considering that lane markings on roads generally include solid lane markings and dashed lane markings, and since dashed lane markings are discontinuous, when using dashed lane markings to assess visibility, if the farthest identifiable point on the image happens to be at the "dash-sand boundary", the visibility assessment based on the farthest identifiable point may not be as accurate as that based on solid lane markings.
[0053] Based on this, as an example, when determining the position of the farthest endpoint of each lane line in the road image, if both solid and dashed lane lines exist in each lane line, the dashed lane lines are excluded, and the position of the farthest endpoint is determined by the solid lane lines.
[0054] As another example, when determining the position of the farthest endpoint of each lane line in a road image, if only dashed lane lines exist among the lane lines, then in order to ensure the output of the visibility assessment results, accuracy can be sacrificed, and the position of the farthest endpoint can still be determined by the only dashed lane lines.
[0055] In this embodiment, when both solid and dashed lane lines exist on the road, the visibility assessment accuracy of solid lane lines may be higher, so solid lane lines are preferred as the basis for assessing visibility.
[0056] The following is combined with Figure 4 , Figure 5 , Figure 6 , Figure 7 The following is an exemplary description of a specific application scenario of one embodiment of this application: See Figure 4 When the vehicle is traveling in a lane, it can acquire road images from at least six cameras mounted on the vehicle. These road images include the lane lines corresponding to the left and right sides of the vehicle. The acquired road images are input into the BEVFormer model, which identifies the coordinate point sets describing the positions of the lane lines corresponding to the left and right sides of the vehicle. This is done assuming that the positions of the lane lines on both sides of the vehicle are not obscured by road obstacles. The model then determines the position of the farthest endpoint of each lane line in the road image and uses this position to determine visibility. Specifically, the farthest endpoint is considered the farthest identifiable target, and the distance between it and the vehicle is determined. This distance is then used to determine visibility.
[0057] See Figure 5 The BEVFormer model identifies the coordinate point sets describing the positions of the lane lines on the left and right sides of the vehicle. It also uses the BEVFormer model to identify obstacles on the road (such as…). Figure 5 The location of other vehicles (as shown) is identified, and at least a portion of at least one lane line is covered by an obstacle (such as...). Figure 5If other vehicles are currently covering one of the lane lines, ignore the data corresponding to the positions of each lane line on the road and re-acquire road images until the positions of each lane line on the road are not covered by the positions of obstacles.
[0058] See Figure 6 When determining the position of the farthest endpoint of each lane line in a road image, if both solid and dashed lane lines exist, considering that dashed lane lines are discontinuous, the visibility assessment based on their farthest identifiable point in the image might be less accurate than that based on solid lane lines if that point is located at the "dash-vane boundary." Therefore, when both solid and dashed lane lines exist, dashed lane lines are excluded, and the position of the farthest endpoint is determined primarily through solid lane lines. This determines the distance between the farthest identifiable target and the vehicle, and visibility is then determined based on this distance.
[0059] See Figure 7 As another example, when determining the position of the farthest endpoint of each lane line in a road image, if only dashed lane lines exist and none of the lane lines are obscured by obstacles, then since there are no solid lane lines in the image as substitutes, if all the existing dashed lane lines are excluded, visibility assessment cannot be performed based on the lane lines, and no visibility assessment result can be obtained. If a new road image needs to be acquired, due to the continuity of lane lines, vehicles may need to travel for a long time before solid lane lines appear, resulting in an excessively long assessment time. In this case, to ensure the timely output of visibility assessment results, some assessment accuracy can be sacrificed, and the position of the farthest endpoint can still be determined using the only dashed lane lines.
[0060] Understandably, since even a small deviation in the farthest endpoint of a lane line can compromise the accuracy of visibility assessment, if any lane line is obscured by an obstacle, the identification of its farthest endpoint cannot be guaranteed to be accurate. Consequently, it cannot be assured whether the identified farthest endpoint of that lane line is the true farthest endpoint relative to the farthest endpoints of other unobscured lane lines, thus compromising the accuracy of the visibility assessment. Therefore, in such cases, the data corresponding to the positions of all lane lines on the road are ignored, and road images are re-acquired until the positions of all lane lines on the road are not obscured by obstacles. Only then can the accuracy of determining the position of the farthest endpoint be maximized, thereby ensuring the accuracy of the visibility assessment results.
[0061] The types of obstacles mentioned above can be implemented in various ways. As an example, obstacles can be other vehicles besides the vehicle itself, such as small vehicles like cars or large vehicles like trucks; pedestrians, electric bicycle riders, and motorcyclists; as well as cones, guardrails, etc. Obstacles can also be construction sites on the road, such as construction teams occupying a lane for construction, maintenance, or emergency repairs. In this case, the construction site may cover the lane line. For construction sites, construction workers usually erect obvious construction signs in front of the construction site. By using the BEVFormer model to identify construction signs contained in the road image, the presence of a construction site obstacle on the road can be determined. The types of obstacles are not limited to these; they can also be any other obstacles that may obstruct the lane line.
[0062] Corresponding to the above method embodiments, this application also provides a visibility determination device, such as... Figure 8 As shown, the device includes: Acquisition unit 801 is used to acquire a road image, wherein the road image contains at least one lane line; The identification unit 802 is used to identify the position of at least one lane line based on the acquired road image, and to identify the position of obstacles on the road; The acquisition unit 801 is further configured to, when at least a portion of the position of the at least one lane line is covered by the position of the obstacle, ignore the data corresponding to the position of each lane line contained in the road and re-acquire the road image until the position of each lane line contained in the road is not covered by the position of the obstacle. The determining unit 803 is used to determine the position of the farthest endpoint of each lane line in the road image when the positions of each lane line included on the road are not covered by the position of the obstacle, and to determine the visibility based on the position of the farthest endpoint.
[0063] The technical solution provided in this application uses lane lines as the core reference target. Lane lines are an essential element of road infrastructure, covering all levels of roads (including rural roads without fixed targets), and have the characteristics of strong continuity and high visibility, thus avoiding the problem of "no reference target available" in visibility assessment as much as possible. Visibility is assessed by using the position of the farthest visible endpoint of the lane line in the image, reducing the difficulty of assessment while ensuring its accuracy.
[0064] While identifying lane line positions using the acquired road images, obstacles on the road are also identified. If an obstacle covers (i.e. occludes) any lane line, the identification result for that lane line is considered inaccurate, and the visibility assessment based on the farthest endpoint of the lane line at that time is considered to be of low accuracy (not meeting the requirements). In this case, the data corresponding to the positions of each lane line on the road are ignored, and road images are re-acquired until the positions of each lane line on the road are not covered by obstacles.
[0065] Only when the positions of all lane lines on the road are not covered by obstacles is it considered that the visibility assessment based on the position of the farthest endpoint of the lane line on the road at this time is accurate enough. In this case, determining the position of the farthest endpoint of each lane line in the road image and determining the visibility based on the position of the farthest endpoint can further ensure the accuracy of the visibility assessment.
[0066] As an example, the identification unit 802 is specifically used to identify a set of coordinate points describing the position of at least one lane line based on the acquired road image, and to identify a set of coordinate points describing the position of the obstacle; the determination unit 803 is specifically used to compare the set of coordinate points describing the position of the lane line with the set of coordinate points describing the position of the obstacle to determine whether the position of the lane line is covered by the position of the obstacle; the determination unit 803 is specifically used to determine the coordinate points describing the position of the farthest endpoint in the set of coordinate points describing each lane line, and to determine the distance between the farthest endpoint and the vehicle based on the coordinate points of the farthest endpoint; and to determine visibility based on the distance.
[0067] As an example, the vehicle is equipped with an attitude sensor to detect the vehicle's attitude; the determination unit 803 is also used to acquire the vehicle's attitude data collected by the attitude sensor; determine whether the vehicle is traveling on a straight road based on the attitude data; if it is determined that the vehicle is not traveling on a straight road, ignore the data corresponding to the positions of each lane line contained in the road and re-acquire the road image until it is determined that the vehicle is traveling on a straight road; if it is determined that the positions of each lane line contained in the road are not covered by the positions of the obstacles, and it is determined that the vehicle is traveling on a straight road, determine the position of the farthest endpoint of each lane line in the road image, and determine the visibility based on the position of the farthest endpoint.
[0068] As an example, the recognition unit 802 is specifically used to input the acquired road image into the visual perception model, so that the visual perception model can use the road image to identify the position of the lane line and the position of the obstacle; the visual perception model can output the position of the lane line and the position of the obstacle based on the input road image.
[0069] As an example, the visual perception model is constructed as follows: acquiring a set of sample images, wherein the images in the sample image set are road images containing at least one lane line and an obstacle; using the sample image set as a training dataset to train the visual perception model, wherein the visual perception model is trained using the error between the predicted positions of the lane lines and the obstacles and the actual positions of the lane lines and the obstacles as a loss function; wherein the actual positions of the lane lines and the actual positions of the obstacles are determined using the perception results of a LiDAR deployed on the vehicle.
[0070] As an example, the determining unit 803 is specifically used to exclude the dashed lane lines if both solid and dashed lane lines exist in each lane line, and to determine the position of the farthest endpoint by using the solid lane lines.
[0071] As an example, the road images are captured using cameras configured on the exterior of the vehicle; the number of cameras is at least six, including at least: a front-view camera, a left-side front-view camera, a right-side front-view camera, a left-side rear-view camera, a right-side rear-view camera, and a rear-view camera.
[0072] This application also provides a vehicle, such as Figure 9 As shown, the vehicle includes: Processor 901; Memory 902 is used to store processor-executable instructions; The processor 901 is configured to implement the visibility determination method described in any of the embodiments above.
[0073] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the visibility determination method described in any of the embodiments above.
[0074] The above description is only a specific embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.
Claims
1. A visibility determination method, applied to vehicles, characterized in that, The method includes: Acquire a road image, the road image containing at least one lane line; Based on the acquired road images, the location of at least one lane line is identified, and the location of obstacles on the road is also identified; If at least a portion of the position of at least one lane line is covered by the position of the obstacle, ignore the data corresponding to the positions of each lane line contained in the road and re-acquire the road image until the positions of each lane line contained in the road are not covered by the position of the obstacle. If the positions of all lane lines on the road are not covered by the positions of the obstacles, determine the position of the farthest endpoint of each lane line in the road image, and determine the visibility based on the position of the farthest endpoint.
2. The method according to claim 1, characterized in that, The process of identifying the position of at least one lane line based on the acquired road images, and identifying the position of obstacles on the road, includes: Based on the acquired road images, a set of coordinate points describing the position of at least one lane line is identified, and a set of coordinate points describing the position of the obstacle is also identified. Whether the position of the lane line is covered by the position of the obstacle is determined by the following method: The set of coordinate points describing the position of the lane line is compared with the set of coordinate points describing the position of the obstacle to determine whether the position of the lane line is covered by the position of the obstacle. Determining the position of the farthest endpoint of each lane line in the road image, and determining visibility based on the position of the farthest endpoint, includes: In the set of coordinate points describing each lane line, determine the coordinate point describing the position of the farthest endpoint, and determine the distance between the farthest endpoint and the vehicle based on the coordinate point of the farthest endpoint. Visibility is determined based on the distance.
3. The method according to claim 1, characterized in that, The vehicle is equipped with an attitude sensor, which is used to detect the attitude of the vehicle; The method further includes: Acquire the attitude data of the vehicle collected by the attitude sensor; Based on the attitude data, determine whether the vehicle is traveling on a straight road surface; If it is determined that the vehicle is not traveling on a straight road, the data corresponding to the positions of each lane line contained on the road are ignored and the road image is re-acquired until it is determined that the vehicle is traveling on a straight road. Determining visibility based on the location of the farthest endpoint includes: If it is determined that the positions of all lane lines on the road are not covered by the positions of the obstacles, and it is determined that the vehicle is traveling on a straight road surface, the position of the farthest endpoint of each lane line in the road image is determined, and the visibility is determined based on the position of the farthest endpoint.
4. The method according to claim 1, characterized in that, The process of identifying the position of at least one lane line based on the acquired road images, and identifying the position of obstacles on the road, includes: The acquired road image is input into the visual perception model, so that the visual perception model can use the road image to identify the position of the lane line and the position of the obstacle; the visual perception model can output the position of the lane line and the position of the obstacle based on the input road image.
5. The method according to claim 4, characterized in that, The visual perception model is constructed in the following way: Acquire a set of sample images, wherein the images in the sample image set are road images containing at least one lane line and obstacles; The visual perception model is trained using the sample image set as the training dataset. The visual perception model is trained using the error between the predicted positions of the lane lines and obstacles and the actual positions of the lane lines and obstacles as the loss function. The actual positions of the lane lines and the obstacles are determined using the perception results of a lidar sensor mounted on the vehicle.
6. The method according to claim 1, characterized in that, Determining the position of the farthest endpoint of each lane line in the road image includes: If both solid and dashed lane lines exist in each lane, the dashed lane lines are excluded, and the position of the farthest endpoint is determined by the solid lane lines.
7. The method according to claim 1, characterized in that, The road images were captured using a camera located on the exterior of the vehicle; The number of cameras is at least 6, including at least: a front-view camera, a left-side front-view camera, a right-side front-view camera, a left-side rear-view camera, a right-side rear-view camera, and a rear-view camera.
8. A visibility determination device, characterized in that, The device includes: An acquisition unit is used to acquire a road image, wherein the road image contains at least one lane line; The identification unit is used to identify the position of at least one lane line based on the acquired road image, and to identify the position of obstacles on the road; The acquisition unit is further configured to, when at least a portion of the position of the at least one lane line is covered by the position of the obstacle, ignore the data corresponding to the position of each lane line contained in the road and re-acquire the road image until the position of each lane line contained in the road is not covered by the position of the obstacle. The determining unit is configured to determine the position of the farthest endpoint of each lane line in the road image when the positions of each lane line contained in the road are not covered by the position of the obstacle, and to determine the visibility based on the position of the farthest endpoint.
9. A vehicle, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to implement the method according to any one of claims 1 to 7.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.