A method for road information occlusion completion, storage medium and electronic device

Abstract

This application discloses a method, storage medium, and electronic device for road information occlusion completion. The method includes: acquiring a perceived image of the area in front of the vehicle while it is in motion; identifying occlusions in the perceived image and analyzing the occlusion area corresponding to the occlusion; the occlusion area is a three-dimensional visual cone formed by extending from the center point of the image acquisition sensor as its vertex, constrained by the imaging boundary along the edge of the occlusion, towards the far end in front of the vehicle, with its extension distance less than a set threshold; acquiring environmental information corresponding to the occlusion area and determining whether the environmental information contains key information; if the environmental information contains key information, generating a virtual image of the key information corresponding to the key information and determining the projection area of ​​the key information on the perceived image; and displaying the virtual image of the key information on the projection area. This application enables the driver to obtain occluded key road information, effectively eliminating blind spots.

Description

Technical Field

[0001] This application relates to the field of intelligent driving technology, specifically to a method for road information occlusion completion, a storage medium, and an electronic device. Background Technology

[0002] In complex road environments, drivers' visibility is often obstructed by large vehicles (such as trucks and buses), temporary construction sites, or terrain, making it impossible to directly observe key information in the obstructed areas, such as traffic lights, road signs, lane markings, pedestrian crossings, or unusual obstacles. Existing technologies use augmented reality navigation to overlay navigation arrows and lane markings onto head-up displays or central control screens. However, this approach often involves continuous information overlay, and its display logic is unrelated to the actual obstructed driver's view, failing to address the problem of localized blind spots caused by specific obstacles. Furthermore, when the overlaid information does not match the actual location of the obstructed object, it can actually cause interference. Summary of the Invention

[0003] The technical problem to be solved by this application is that existing road information occlusion completion schemes are difficult to match visual blind spots caused by obstacles, and thus provide a road information occlusion completion method, storage medium and electronic device.

[0004] Firstly, the technical solution of this application provides a method for road information occlusion completion, including: While the vehicle is in motion, a perceived image of the area in front of the vehicle is acquired; Identify occlusions in the perceived image and analyze the occlusion area corresponding to the occlusions; the occlusion area is a three-dimensional visual cone formed by extending from the center point of the image acquisition sensor as the vertex, along the imaging boundary constraint of the occlusion edge, to the far end in front of the vehicle, and its extension distance is less than a set threshold. Obtain environmental information corresponding to the obstructed area, and determine whether the environmental information contains key information; the key information includes at least one of traffic lights, road signs, lane lines, pedestrian crossings, or abnormal obstacles; If the environmental information contains key information, a virtual image of the key information corresponding to the key information is generated, and the projection area of ​​the key information on the perceived image is determined. The key information is displayed as a virtual image on the projection area.

[0005] Some road information occlusion completion methods include the following steps: obtaining environmental information corresponding to the occluded area and determining whether the environmental information contains key information. Obtain map data corresponding to the obscured area, and determine whether the map data contains key information.

[0006] Some solutions describe road information occlusion completion methods, including obtaining map data corresponding to the occluded area and determining whether the map data contains key information, such as: The first mapping relationship between the image coordinate system and the sensor coordinate system is determined based on the parameters of the image acquisition sensor; Based on the relative positional relationship between the image acquisition sensor and the vehicle, and the first mapping relationship, a second mapping relationship between the image coordinate system and the geographical coordinate system where the vehicle is located is determined; Obtain vehicle positioning information and the set of projected contour coordinates of the occluded area in the image coordinate system, as the first coordinate set; Based on the second mapping relationship and the extension distance of the occluded area, determine the three-dimensional spatial range under the geographic coordinate system corresponding to the first coordinate set; The target data corresponding to the three-dimensional spatial range is queried from the high-precision map database and used as the map data.

[0007] In some road information occlusion completion methods, the step of generating a virtual image of the key information corresponding to the key information and determining the projection area of ​​the key information on the perceived image if the environmental information contains key information further includes: Based on the vehicle positioning information and the parameters of the image acquisition sensor, the three-dimensional geographic coordinates of the key information are projected onto the image coordinate system to obtain the projection area.

[0008] In some solutions, the method for completing road information occlusion includes the step of displaying the key information virtual image on the projection area: The virtual image of the key information is displayed using a differentiated visual style that differs from the real scene. The differentiated visual style includes at least one of semi-transparent display, luminous outline display, or dashed line filling display.

[0009] In some road information occlusion completion methods described in the schemes, the step of displaying the key information virtual image on the projection area further includes: Obstructions in the perceived image are blurred.

[0010] In some road information occlusion completion methods, the step of blurring occluded objects in the perceived image includes: The blurring process includes at least one of making the obscuring object semi-transparent, blurring it, or reducing its saturation.

[0011] Secondly, the present application provides a computer-readable storage medium storing program information, wherein a computer reads the program information and executes the steps of the road information occlusion completion method described in any of the first aspects.

[0012] Thirdly, the present application provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the road information occlusion completion method described in any of the first aspects.

[0013] Fourthly, the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the road information occlusion completion method described in any of the first aspects.

[0014] The technical solution provided in this application has the following technical effects compared with the prior art: The road information occlusion completion method, storage medium, and electronic device provided in this application acquire a perceived image in front of the vehicle, identify occluders, and analyze the occluded area. The occluded area is a three-dimensional visual cone formed by extending from the center point of the image acquisition sensor as the vertex, along the imaging boundary constraint of the occluder's edge, and towards the far end in front of the vehicle. Its extension distance is less than a set threshold, thereby determining the range of the visual blind spot obscured by the occluded object. The environmental information corresponding to the occluded area is acquired. When it is determined that the environmental information contains key information such as traffic lights, road signs, lane lines, or pedestrian crossings, a corresponding virtual image of the key information is generated. After determining the projection area of ​​the virtual image of the key information on the perceived image, the virtual image of the key information is displayed in the projection area, thereby enabling the driver to obtain the obscured key road information and effectively eliminating blind spots. Attached Figure Description

[0015] Figure 1 This is a flowchart of a road information occlusion completion method according to an embodiment of this application; Figure 2 This is a schematic diagram of the system architecture related to the execution of the road information occlusion completion method in the vehicle of this application; Figure 3 This is a flowchart illustrating the steps for determining map data corresponding to an obscured area according to one embodiment of this application; Figure 4 This is a flowchart illustrating the processing steps for key information virtual images and perceived image occlusions according to one embodiment of this application; Figure 5 This is a schematic diagram illustrating the processing of the display method for the truck and the traffic light when the truck is used as an obstruction and the traffic light is used as key information, according to one embodiment of this application. Figure 6This is a schematic diagram of the hardware connections of an electronic device that performs the road information occlusion completion method according to an embodiment of this application. Detailed Implementation

[0016] The specific embodiments of this application will be further described below with reference to the accompanying drawings.

[0017] It is readily understood that, based on the technical solution of this application, various structural and implementation methods can be interchanged by those skilled in the art without altering the essential spirit of this application. Therefore, the following detailed embodiments and accompanying drawings are merely illustrative examples of the technical solution of this application and should not be considered as the entirety of this application or as limitations or restrictions on the technical solution of the application.

[0018] This application aims to address the problem that drivers are unable to obtain critical road information (such as traffic lights, road signs, lane markings, pedestrian crossings, or abnormal obstacles) during vehicle operation due to obstructions caused by large vehicles (such as trucks and buses), temporary construction facilities, or terrain features. The proposed method utilizes a technical path encompassing image acquisition, obstruction identification and obstruction area analysis, environmental information acquisition and critical information judgment, and virtual image generation and display to complete the obstructed road information. This solution enables drivers to promptly obtain obscured critical road information even in the presence of obstructions, thus improving driving safety. The following section will elaborate on this solution in detail with specific implementation steps and application scenarios (such as following a van at an urban intersection).

[0019] This embodiment provides a method for road information occlusion completion, applied in a vehicle controller, such as... Figure 1 As shown, it includes the following steps: S100: Acquire a perceived image of the area in front of the vehicle while it is in motion.

[0020] In this step, while the vehicle is in motion, sensing data from the area in front of the vehicle can be acquired in real time via a sensing module installed at the front of the vehicle. For example... Figure 2As shown, the vehicle system architecture includes a vehicle perception and localization layer, a cloud / local data service layer, an onboard controller computing and output layer, and a human-machine interaction layer. In this step, the perception module may include an onboard forward-facing image acquisition sensor, such as a camera. Cameras are typically mounted inside the windshield or in the front grille, with a field of view typically ranging from 30° to 120°, covering a certain range of road scenes in front of the vehicle. Existing vehicles are usually equipped with high-definition cameras, which can provide rich texture and color information. The acquired perception images are transmitted to the controller for analysis and processing in subsequent steps. Furthermore, the perception images can undergo preprocessing operations such as noise reduction, white balance adjustment, and contrast enhancement according to pre-set processing methods to improve image quality. The perception images in this step contain occlusions and road environment information beyond the occlusions, providing a data foundation for subsequent occlusion recognition and occlusion information completion. It can be understood that the occlusions in the perception images are the projection results of actual spatial occlusions onto the perception images.

[0021] S200: Identify occlusions in the perceived image and analyze the occlusion region corresponding to the occlusion. The occlusion region is a three-dimensional visual cone formed by extending from the center point of the image acquisition sensor as its vertex, along the imaging boundary constraint of the occlusion edge, towards the far end in front of the vehicle, and its extension distance is less than a set threshold.

[0022] In this step, after acquiring the perceived image, occlusion identification and analysis are performed. Occlusions in the perceived image are projections of actual occlusions, which are objects within the vehicle's field of vision that obstruct the driver's observation of road information. Typical actual occlusions include: large freight vehicles (such as vans, container trucks, and tankers), public transportation vehicles (such as buses and articulated buses), construction site barriers or barriers, and temporarily parked large vehicles. In practical implementation, the onboard controller's computation and output layers can use pre-trained deep learning-based image semantic segmentation algorithms or object detection algorithms for occlusion identification. The deep learning model is trained on large-scale labeled datasets, such as the Cityscapes dataset, the BDD100K dataset, or self-collected and labeled datasets of various occlusions. During model training, the training data includes occlusion samples of various types, angles, and lighting conditions. For large vehicles, the rear view samples of large vehicles should be emphasized, as drivers most frequently encounter occlusions from the rear of vehicles in front while driving.

[0023] After identifying occlusions in the perceived image, the occlusion area corresponding to the occlusion is analyzed. In real-world scenarios, the actual occlusion is relatively close to the image acquisition sensor, while the area further in front of the vehicle, behind the actual occlusion, is a blind spot. This solution does not need to acquire and analyze all spatial data behind the actual occlusion; instead, it selects and analyzes spatial data within a set threshold range based on driving requirements, avoiding excessive data processing. The threshold can be dynamically determined based on the effective sensing distance of the image acquisition sensor (e.g., 300 meters), the vehicle's current speed (the threshold increases at high speeds and decreases on urban roads), or the road type (300 meters for high speeds and 100 meters for urban roads). For the specific algorithm for generating the occlusion frustum based on the camera's optical center and the obstacle contour during camera capture, existing algorithms can be used. In practical applications, the frustum is constructed as follows: Vertex: The starting point of all rays, i.e., the optical center of the image acquisition sensor (camera); Near plane: The front section of the view frustum, the plane position where the actual occluder is located, that is, the imaging boundary of the actual occluder edge in three-dimensional space; Far plane: The rear cross section of the view frustum, the plane at a set threshold distance; The side view is a sloping surface that starts from the vertex, passes through the edge of the near plane, and extends to the far plane. The shape of the sloping surface is determined according to the shape of the actual edge of the occluder. Essentially, it is a collection of ray beams that start from the optical center of the camera, pass through the outline edge of the occluder, and extend outward.

[0024] The concept and acquisition of view frustum have been described in the prior art (e.g., the multimodal fusion target detection algorithm based on view frustum disclosed in Metrology and Testing Technology [November 2024]), indicating that view frustum generation algorithms have been applied, and will not be repeated in this application.

[0025] S300: Obtain environmental information corresponding to the obstructed area, and determine whether the environmental information contains key information; the key information includes at least one of traffic lights, road signs, lane lines, pedestrian crossings, or abnormal obstacles.

[0026] After identifying the occlusion area, this step obtains the corresponding environmental information to determine which road facilities exist within the occluded 3D space. A high-precision map database is stored in the cloud / local data service layer, and environmental information is primarily obtained from this high-precision map. First, the vehicle's real-time location information (latitude, longitude, elevation, heading angle, etc.) is acquired. Then, the 3D view frustum is transformed to a geographic coordinate system, and road elements falling within this 3D spatial range are queried in the high-precision map.

[0027] In this plan, key information is defined as road elements that directly impact driving safety and compliance, including: Traffic lights: red, green, yellow lights, countdown displays, and arrow indicators. Road signs: speed limit signs, directional signs, lane guidance signs, distance warning signs, and construction warning signs. Lane markings: solid lines, dashed lines, double yellow lines, guide lines, stop lines, and pedestrian crossing warning markings. Pedestrian crossings: the location and extent of zebra crossings. Other abnormal obstacles such as cargo dropped from the vehicle in front, construction debris, equipment left behind by disabled or accident vehicles, and obstacles caused by inclement weather.

[0028] For routine, critical information, it is obtained through high-precision map queries. For unusual obstacles, it can be obtained through V2V communication to receive information from the vehicle ahead or through roadside unit broadcasts.

[0029] S400: If the environmental information contains key information, then generate a virtual image of the key information and determine the projection area.

[0030] This step introduces an information missing assessment mechanism to determine whether a completion step needs to be triggered. The vehicle-mounted controller's calculation and output layer analyzes the road environment of the obscured area, combining contextual information from high-precision maps or the vehicle navigation system (such as whether the current road type is urban or highway, how far is the upcoming intersection from the next one, and the precise coordinates of known road signs, traffic light poles, and road signs) to assess whether the obscured area poses a risk of missing critical information that could affect driving decisions. If at least one of the following is present: traffic lights, road signs, lane markings, pedestrian crossings, or abnormal obstacles, this information missing may pose a safety hazard, and a completion step will be triggered. For example, if the obscured area blocks a traffic light 50 meters from the next intersection, the traffic light is critical information and a subsequent completion step needs to be triggered. If the obscured area only blocks a green belt, the green belt is not critical information, and a completion step does not need to be triggered. This on-demand triggering mechanism avoids continuous virtual rendering from interfering with the driver, reflecting scene-driven proactive intelligence.

[0031] After identifying key information, corresponding virtual images are generated based on the type and attributes of the key information. The generation of virtual images can employ computer graphics techniques, such as real-time rendering engines based on OpenGL ES or Vulkan. For example, traffic lights can be generated as circular illuminated images, with the color determined according to the real-time status, and countdown numbers can be displayed in the center; road signs can be generated as road sign images of corresponding shapes, with text content rendered; pedestrian crossings can be generated as a set of white strip-shaped virtual images to simulate zebra crossings; and lane lines can be generated as virtual lane line images of corresponding colors and line types. Next, the projection area of ​​the key information on the perceived image is determined. Specifically, based on vehicle positioning information and the parameters of the image acquisition sensors, the three-dimensional geographic coordinates of the key information are projected onto the image coordinate system to obtain its projection position and range on the image. This projection area is typically located inside or near the outline of occluded objects in the perceived image.

[0032] S500: Display the key information virtual image on the projection area.

[0033] In this step, the perceived image, overlaid with virtual images of key information, is ultimately displayed on the in-vehicle display device in the human-computer interaction layer. Specifically, it can be displayed through an augmented reality head-up display, projecting the overlaid image onto the windshield so that the driver can see the supplementary information without shifting their gaze.

[0034] The solution provided in this embodiment acquires a perceived image of the area in front of the vehicle, identifies obstructions, analyzes the obstructed areas, and obtains environmental information corresponding to those areas. When the environmental information is determined to contain key information such as traffic lights, road signs, lane lines, or pedestrian crossings, a corresponding virtual image of the key information is generated and displayed in a calculated projection area. This allows the driver to access the obstructed key road information, effectively eliminating blind spots. Furthermore, this solution only generates and displays virtual images when key information exists within the obstructed area, avoiding continuous display of invalid information that could cause visual interference to the driver. Moreover, the real scene in the unobstructed areas remains unchanged, enabling the driver to clearly distinguish between real road facilities and the generated virtual information.

[0035] Preferably, in the above scheme, the step of obtaining environmental information corresponding to the occluded area and determining whether the environmental information contains key information in S300 includes: obtaining map data corresponding to the occluded area and determining whether the map data contains key information. In this step, the occluded area is a three-dimensional space, which is actually a set of coordinates. The map data is preferably a high-precision map. The high-precision map not only contains the geometric shape and topological relationship of the road, but also contains rich semantic information, such as the precise location, type, and color of lane lines; the precise location, type, speed limit, and text content of traffic signs; the precise location, pole height, light group type, orientation, and timing scheme of traffic lights; the precise location range of pedestrian crossings; and the precise location of stop lines. In specific implementation, the three-dimensional view cone occluded area is used as a query window. Road elements located within the three-dimensional space range are searched in the high-precision map database, and it is determined whether the map data contains key information. If at least one matching item exists, it is determined that key information is contained, and completion needs to be triggered. For example, if a traffic light is found within the map coordinate range corresponding to the occluded area, it is determined that key information is contained. If the map data contains only non-critical information, it is determined that it does not contain critical information, and no completion is required.

[0036] Furthermore, such as Figure 3 As shown, step S300, which involves obtaining map data corresponding to the occluded area and determining whether the map data contains key information, includes: S301: Determine the first mapping relationship between the image coordinate system and the sensor coordinate system based on the parameters of the image acquisition sensor.

[0037] In this step, taking a camera as an example, the image acquisition sensor uses a two-dimensional coordinate system with pixels as the unit, and the origin is usually located at the upper left corner of the image. The sensor coordinate system is a three-dimensional Cartesian coordinate system established with the optical center of the camera as the origin, where the Z-axis is along the optical axis (pointing forward of the vehicle), the X-axis is horizontal to the right, and the Y-axis is vertically downward. The first mapping relationship describes how points in three-dimensional space are imaged on a two-dimensional image; it is a perspective projection model. This mapping relationship is determined by the camera's intrinsic parameters, including focal length, principal point coordinates, distortion coefficients, etc. These intrinsic parameters can be obtained by offline calibration of the camera. This perspective projection process can be implemented using existing projection calculation methods.

[0038] S302: Based on the relative positional relationship between the image acquisition sensor and the vehicle, and the first mapping relationship, determine the second mapping relationship between the image coordinate system and the geographical coordinate system where the vehicle is located.

[0039] After establishing the initial mapping relationship, this step introduces the relative positional relationship between the image acquisition sensor and the vehicle. Cameras are typically mounted in fixed positions on the vehicle (e.g., inside the windshield), therefore the camera coordinate system has a fixed transformation relationship relative to the vehicle's geographic coordinate system. In practice, a vehicle coordinate system can also be introduced. First, the mapping relationship between the camera coordinate system and the vehicle coordinate system is obtained, then the mapping relationship between the vehicle coordinate system and the geographic coordinate system is obtained, and finally, the mapping relationship between the camera coordinate system and the geographic coordinate system is determined. The geographic coordinate system is an absolute world coordinate system, usually represented by latitude, longitude, and elevation. The relative positional relationship between the camera and the vehicle includes translation vectors and rotation matrices (corresponding to pitch, yaw, and roll angles), which can be obtained through extrinsic parameter calibration. The vehicle's positioning information provides the position (latitude, longitude, and elevation) of the vehicle coordinate system origin in the geographic coordinate system and the vehicle's heading angle (yaw angle), thus achieving the transformation from the vehicle coordinate system to the geographic coordinate system.

[0040] Therefore, there is a defined transformation chain from image coordinate system to geographic coordinate system, which allows any pixel in the image to be converted to geographic coordinates in the real environment.

[0041] S303: Obtain vehicle positioning information and the set of projected contour coordinates of the occluded area in the image coordinate system, as the first coordinate set.

[0042] In this step, vehicle positioning information is obtained through an onboard positioning module, which can be a Global Positioning System (GPS), BeiDou Navigation Satellite System (BDS), inertial measurement unit (INS), etc. Vehicle positioning information includes the vehicle's current latitude and longitude coordinates, elevation, heading angle, pitch angle, roll angle, speed, and acceleration.

[0043] The set of projection contour coordinates of the occluded area in the image coordinate system is the image coordinate of the occluded object contour in the perceived image, which can be a set of contour points or the four vertices of the smallest bounding rectangle.

[0044] S304: Based on the second mapping relationship and the extension distance of the occluded area, determine the three-dimensional spatial range under the geographic coordinate system corresponding to the first coordinate set.

[0045] In this step, each pixel coordinate in the first coordinate set is mapped to the coordinates of the actual edge of the occluder in the actual geographic coordinate system. The geographic coordinates of the camera's optical center are also determined. The ray drawn from the camera's optical center to the actual edge of the occluder can be determined. Combined with the extended set threshold, the three-dimensional spatial range, i.e., the three-dimensional view frustum, can be determined.

[0046] S305: Query the target data corresponding to the three-dimensional spatial range in the high-precision map database as map data.

[0047] This step uses the 3D spatial range obtained in the previous steps as a query window to retrieve all road elements falling within this range from the high-precision map database. The query results are the target map data. For example, the query might find a traffic light, a speed limit sign, and a pedestrian crossing within the 3D spatial range. This data is extracted and used as the basis for generating the virtual image. If the query results contain only non-critical information, the processing can be terminated early, eliminating the need for virtual rendering and saving computational resources.

[0048] Furthermore, S400 also includes: projecting the three-dimensional geographic coordinates of the key information onto the image coordinate system based on the vehicle positioning information and the parameters of the image acquisition sensor to obtain the projection area. Specifically, after retrieving the key information from map data, this key information has precise three-dimensional geographic coordinates, such as the coordinates of a traffic light (longitude, latitude, and elevation). To display the virtual image in the correct position, these three-dimensional geographic coordinates need to be projected back onto the image coordinate system. First, based on the vehicle's current positioning information (vehicle coordinates and heading, pitch, and roll angles), the geographic coordinates of the key information are transformed into the vehicle coordinate system. Then, the extrinsic parameters of the camera are used to transform the vehicle coordinate system into the camera coordinate system. Finally, the intrinsic parameters of the camera are used to project it onto the image coordinate system to obtain pixel coordinates. These pixel coordinates are the theoretical imaging position of the key information on the perceived image, which is the projection area where the virtual image should be displayed. If the key information is an object with a certain geometric scale (such as a road sign, 1.2 meters wide and 0.8 meters high), the projections of multiple feature points (such as the four corners) on its outline into the image are calculated to determine the size and shape of the projection area. This process utilizes the actual physical dimensions of the key information and the geometric relationship of perspective projection. The final projection area can be represented by a set of pixel coordinates. Obviously, the transformation chain in this process is the reverse of the transformation chain in step S302. This scheme ensures the precise spatial alignment between the completed virtual image of the key information and the occluded real key information by projecting the three-dimensional geographic coordinates of the key information back onto the image coordinate system.

[0049] Furthermore, in the above scheme, such as Figure 4 As shown, the S500 includes: S501: The key information virtual image is displayed using a differentiated visual style that distinguishes it from the real scene. This differentiated visual style includes at least one of semi-transparent display, luminous outline display, or dashed line fill display. This solution defines the visual presentation style of the virtual image, ensuring that the virtual image is visually distinct from road elements in the real scene, preventing drivers from mistaking the key information virtual image for real road facilities. Specifically, the differentiated visual styles include the following: Semi-transparent display: Virtual images are overlaid with a certain level of transparency (such as 50% or 70%), allowing the driver to see the obstruction through the virtual image. The semi-transparent effect can be achieved by setting the Alpha blending mode during rendering.

[0050] The glowing outline is displayed as follows: the boundaries of the virtual image have a soft halo or highlighted outline, making it look like a holographic projection. The glowing effect can be achieved by applying Gaussian blur to the edges of the virtual image and overlaying it during the post-processing stage.

[0051] Dashed fill display: Virtual images are presented as dashed lines or dot grids, rather than solid fills. For example, virtual lane lines are drawn as dashed lines with alternating solid and blank sections, and virtual pedestrian crossings are drawn as dotted strips.

[0052] The aforementioned differentiated visual styles can be used individually or in combination. For example, virtual traffic lights can be displayed with a glowing outline and semi-transparent fill. Different types of key information can use different styles: traffic lights use a glowing outline to emphasize their warning attribute, lane lines use dashed fill to distinguish them from real solid lines, and road signs use semi-transparent display to indicate that they are supplementary information.

[0053] This solution employs differentiated visual styles such as semi-transparency, luminous outlines, and dashed line filling to make the virtual completed image visually distinct from road elements in the real scene, avoiding confusion between virtual information and the real environment for users and reducing driving risks caused by misunderstanding.

[0054] More preferably, such as Figure 4 As shown, S500 also includes: S502: Obstructions in the perceived image are blurred. In specific implementations, obstructions (such as a truck in front) occupy a certain pixel area in the perceived image. Blurring this area reduces the visual salience of the obstruction, making it blurred and faded, while the key information virtual image located above (or overlaid on) it becomes more prominent. It should be noted that the blurring process targets the two-dimensional imaging area of ​​the obstruction in the perceived image (i.e., the area covered by the pixel mask or bounding box of the obstruction). After blurring, the texture details of the obstruction (such as text or advertising patterns on the vehicle) are weakened, no longer attracting the driver's visual attention, making the key information virtual image more obvious and improving information transmission efficiency. The intensity of the blurring process can be dynamically adjusted according to the type, size, and distance of the obstruction from the vehicle. For example, the closer the obstruction, the higher the degree of blurring; the farther the obstruction, the lower the degree of blurring can be.

[0055] Furthermore, the blurring process includes at least one of the following: making the obscuring object semi-transparent, blurring it, or reducing its saturation. Wherein: Transparency processing: By adjusting the alpha channel value (transparency) of the pixels in the occluded area, the occluded object appears semi-transparent.

[0056] Blur Processing: Gaussian blur and other convolutional filtering algorithms are used to blur the pixels in the occluded area, eliminating texture details. Gaussian blur is implemented as follows: for each pixel within the region, an N×N neighborhood is taken centered on that pixel, and the weighted average of all pixels in the neighborhood is calculated as the new value of that pixel. The weights conform to a Gaussian distribution. The degree of blurring can be controlled by adjusting the standard deviation of the normal distribution; the larger the standard deviation, the higher the degree of blurring. After blurring, the texture details of the occluded object (such as text on a car body or advertising patterns) are smoothly eliminated.

[0057] Reduce saturation processing: Reduce the color saturation of pixels in the occluded area to make it appear closer to grayscale. For example, in the RGB color space, first convert RGB to the HSV color space, then multiply the S channel value by a coefficient less than 1, and finally convert it back to the RGB color space.

[0058] The three methods described above can be used individually or in combination. For example, combining semi-transparency with desaturation creates a grayscale semi-transparent effect for the obscured object; combining blurring with desaturation makes the obscured object both blurred and faded. The specific choice can be flexibly adjusted according to the type of obscured object and the complexity of the virtual information: if the obscured object is a billboard with a lot of text, blurring is preferred; if the obscured object is a brightly colored vehicle, desaturation is preferred. This solution reduces the visual salience of obscured objects through different technical means, such as semi-transparency, blurring, or desaturation. The combination of multiple methods can adapt to the needs of different types of obscured objects and different driving scenarios, enabling the system to maintain excellent visual effects under various complex road conditions.

[0059] by Figure 5 Taking the scenario shown as an example, a vehicle is following a van on a city road. The van is large and completely obstructs the driver's forward view. There is an intersection a short distance ahead of the van, with traffic lights installed above it. Due to the van's obstruction, the driver cannot directly see the current status of the traffic light (red, green, or yellow), posing a serious safety risk of running a red light or misjudging the situation. It should be noted that... Figure 5Instead of using the driver's perspective, the viewpoint of the truck's front is used to illustrate the relationship between the traffic lights and the truck. Using the aforementioned embodiment of this application, a camera installed inside the vehicle's windshield acquires real-time images of the area in front of the vehicle. These images are transmitted to the controller via an in-vehicle Ethernet network. The controller processes the images, identifies the truck as an obstruction, and outputs a pixel-level mask of the truck as its projected outline in the image. Using the center point of the image acquisition sensor as the vertex and the truck's projected outline in the image as the cross section, a three-dimensional view cone space is constructed forward of the vehicle as the obstruction area. A threshold of 100 meters is set based on the current road type (urban road), which covers the distance to the traffic lights at the intersection ahead. The vehicle's current positioning information (GNSS / RTK positioning coordinates, accuracy ±10cm) and attitude information (heading angle, pitch angle, and roll angle provided by the IMU) are acquired. Based on the second mapping relationship and the extension distance of the obstruction area, the three-dimensional view cone obstruction area is converted to a geographic coordinate system to obtain the corresponding three-dimensional spatial range. The system queries a high-precision map database for road elements falling within the specified 3D spatial range. The query results indicate a crossroads 50 meters ahead, with a traffic light whose 3D geographic coordinates are (longitude Lon, latitude Lat, elevation Alt), pole height 5.5 meters, and light type a round-headed three-lamp system. Since the query results include a traffic light, the system determines that the environmental information contains key information, triggering completion. A virtual image corresponding to the traffic light is generated. The current traffic light status is determined to be red via V2X communication, thus generating a red circular glowing image. Based on the vehicle's current location information and camera parameters, the 3D geographic coordinates of the traffic light are projected onto the image coordinate system, obtaining the projection area of ​​the traffic light on the perceived image. This projection area is located inside the truck's projection outline. The rendering output module displays the generated virtual traffic light image on the projection area. Figure 5 As shown, to make the virtual traffic light more prominent, the truck in the perceived image is blurred. Specifically, the pixel area where the truck is located is semi-transparent. Ultimately, on the vehicle screen, the driver sees a semi-transparent, low-saturation visual effect of the truck ahead, with a red, glowing virtual traffic light precisely suspended behind the truck's rear. As the vehicle moves forward, the display position and size of the virtual traffic light are updated in real time, maintaining perspective alignment with the real scene. Based on this, the driver can accurately determine that the light is red and thus slow down to prepare to stop. This application achieves accurate completion of obscured traffic lights when following a large truck, effectively eliminating blind spots for the driver and improving driving safety.

[0060] This application also provides a computer-readable storage medium storing program information. After reading the program information, the computer executes the steps of the road information occlusion completion method described in any one of the method embodiments.

[0061] This application also provides a computer program product, including a computer program / instructions, which, when executed by a processor, implement the steps of the road information occlusion completion method described in any of the method embodiments.

[0062] This application also provides an electronic device, such as... Figure 6 As shown, the electronic device includes at least one processor 61 and at least one memory 62. The at least one memory 62 stores program information. After reading the program information, the at least one processor 61 executes the road information occlusion completion method described in any of the above method embodiments. The device may further include an input device 63 and an output device 64. The processor 61, memory 62, input device 63, and output device 64 can be communicatively connected. The memory 62, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. The processor 61 executes various functional applications and data processing by running the non-volatile software programs, instructions, and modules stored in the memory 62, thereby implementing the road information occlusion completion method provided in any of the above embodiments. The memory 62 may include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a function; the data storage area may store data created based on the use of the road information occlusion completion method, etc. Furthermore, memory 62 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, memory 62 may optionally include memory remotely located relative to processor 61, and these remote memories may be connected via a network to the apparatus performing the road information occlusion completion method. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof. Input device 63 may receive user clicks and generate signal inputs related to user settings and function control of the road information occlusion completion method. Output device 64 may include a display device such as a display screen. When the one or more modules are stored in memory 62 and are run by the one or more processors 61, the road information occlusion completion method in any of the above method embodiments is executed.

[0063] As needed, the above technical solutions can be combined to achieve the best technical effect.

[0064] The above are merely the principles and preferred embodiments of this application. It should be noted that, for those skilled in the art, several other modifications can be made based on the principles of this application, and these modifications should also be considered within the scope of protection of this application.

Claims

1. A method for completing road information occlusion, characterized in that, include: While the vehicle is in motion, a perceived image of the area in front of the vehicle is acquired; Identify occluders in the perceived image and analyze the occlusion area corresponding to the occluder; The occlusion area is a three-dimensional visual cone formed by extending from the center point of the image acquisition sensor as the vertex, constrained by the imaging boundary along the edge of the occlusion object, and extending to the far front of the vehicle. Its extension distance is less than a set threshold. Obtain environmental information corresponding to the obstructed area, and determine whether the environmental information contains key information; the key information includes at least one of traffic lights, road signs, lane lines, pedestrian crossings, or abnormal obstacles; If the environmental information contains key information, a virtual image of the key information corresponding to the key information is generated, and the projection area of ​​the key information on the perceived image is determined. The key information is displayed as a virtual image on the projection area.

2. The road information occlusion completion method according to claim 1, characterized in that, The step of obtaining environmental information corresponding to the occluded area and determining whether the environmental information contains key information includes: Obtain map data corresponding to the obscured area, and determine whether the map data contains key information.

3. The road information occlusion completion method according to claim 2, characterized in that, The step of acquiring map data corresponding to the obscured area and determining whether the map data contains key information includes: The first mapping relationship between the image coordinate system and the sensor coordinate system is determined based on the parameters of the image acquisition sensor; Based on the relative positional relationship between the image acquisition sensor and the vehicle, and the first mapping relationship, a second mapping relationship between the image coordinate system and the geographical coordinate system where the vehicle is located is determined; Obtain vehicle positioning information and the set of projected contour coordinates of the occluded area in the image coordinate system, as the first coordinate set; Based on the second mapping relationship and the extension distance of the occluded area, determine the three-dimensional spatial range under the geographic coordinate system corresponding to the first coordinate set; The target data corresponding to the three-dimensional spatial range is queried from the high-precision map database and used as the map data.

4. The road information occlusion completion method according to claim 3, characterized in that, The step of generating a virtual image of the key information corresponding to the key information and determining the projection area of ​​the key information on the perceived image if the environmental information contains key information further includes: Based on the vehicle positioning information and the parameters of the image acquisition sensor, the three-dimensional geographic coordinates of the key information are projected onto the image coordinate system to obtain the projection area.

5. The road information occlusion completion method according to claim 1, characterized in that, In the step of displaying the virtual image of the key information on the projection area: The virtual image of the key information is displayed using a differentiated visual style that differs from the real scene. The differentiated visual style includes at least one of semi-transparent display, luminous outline display, or dashed line filling display.

6. The road information occlusion completion method according to claim 5, characterized in that, The step of displaying the virtual image of the key information on the projection area further includes: Obstructions in the perceived image are blurred.

7. The road information occlusion completion method according to claim 6, characterized in that, In the step of blurring the occlusions in the perceived image: The blurring process includes at least one of making the obscuring object semi-transparent, blurring it, or reducing its saturation.

8. A computer-readable storage medium, characterized in that, The storage medium stores program information, and after the computer reads the program information, it executes the steps of the road information occlusion completion method according to any one of claims 1-7.

9. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instruction is executed by the processor, it implements the steps of the road information occlusion completion method according to any one of claims 1-7.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the road information occlusion completion method according to any one of claims 1-7.

Images