A vehicle front road condition perception method, image acquisition device and vehicle

Abstract

This application discloses a method for perceiving road conditions ahead of a vehicle, an image acquisition device, and a vehicle, applied in the field of vehicle perception technology. The method acquires road geometry information of a target road segment ahead, dynamically plans the scanning path of the image acquisition device based on this information, and generates corresponding control commands to drive the image acquisition device to perform directional scanning along the spatial shape of the target road segment to obtain road condition images. This approach enables the image acquisition device to actively adapt to the actual road direction, thereby effectively acquiring road condition information in blind spots behind the target road segment and improving the vehicle's environmental perception capability in road scenarios with specific geometric shapes.

Description

Technical Field

[0001] This application relates to the field of vehicle perception technology, specifically to a method for perceiving road conditions ahead of a vehicle, an image acquisition device, and a vehicle. Background Technology

[0002] Highways, as a crucial component of modern transportation, place extremely high demands on driving safety. To achieve advanced warnings of road conditions ahead, relevant technical solutions typically rely on vehicle-mounted sensing devices with long-range detection capabilities. These devices utilize high resolution and narrow field-of-view optical characteristics to continuously observe distant areas directly in front of the vehicle. However, when vehicles travel on roads with non-linear geometries such as curves and slopes, the fixed physical orientation of these sensing devices, or their ability to execute only preset simple scanning patterns, prevents their field of view from dynamically adapting to continuous changes in the road's spatial morphology. This results in blind spots in areas behind roads with specific geometric shapes. Therefore, adapting vehicle sensing capabilities to complex road geometries has become a pressing technical problem. Summary of the Invention

[0003] To address the aforementioned technical problems, this application is proposed. Embodiments of this application provide a method for perceiving road conditions ahead of a vehicle, an image acquisition device, and a vehicle. Utilizing road geometry information as guidance, the observation direction of the image acquisition device is adjusted, enabling it to actively track the road's direction. This extends the effective detection field of view of the image acquisition device beyond visual range along the target road segment, achieving image acquisition of road conditions behind the target road segment. This effectively solves the problem of blind spots beyond visual range in target road segment scenarios, improving driving perception and safety.

[0004] According to a first aspect of this application, a method for perceiving road conditions ahead of a vehicle is provided, comprising: acquiring road geometry information of a target road segment ahead of the vehicle; the road geometry information being used to characterize the spatial morphology of the target road segment ahead; planning a scanning path for image acquisition based on the road geometry information; wherein the scanning path is used to guide the pointing of an image acquisition device so that the pointing of the image acquisition device changes with the spatial morphology of the target road segment ahead; and generating a control command according to the scanning path; wherein the control command is used to control the image acquisition device to perform scanning according to the scanning path to acquire a road condition image of the target road segment ahead.

[0005] As one possible implementation, the step of planning a scanning path for image acquisition based on the road geometry information includes: determining a safety perception time based on the current vehicle speed; determining a look-ahead distance based on the current vehicle speed and the safety perception time; sampling virtual observation points along the centerline of the target road segment ahead based on the look-ahead distance and the road geometry information; and determining the scanning path based on the sampled virtual observation points.

[0006] By dynamically determining the forward distance to be scanned based on the current vehicle speed, the scanning range can be matched with the actual driving speed and safety requirements, ensuring coverage of relevant risk areas. This avoids the waste of resources caused by unnecessary long-distance scanning and prevents insufficient warning time that may result from close-range scanning.

[0007] As one possible implementation, generating control instructions based on the scanning path includes: generating a sequence of control instructions to control the image acquisition device to sequentially point to each virtual observation point based on multiple virtual observation points obtained from sampling, and a sequence of acquisition instructions to acquire images at each pointing direction.

[0008] By transforming continuous path planning into a discrete, precisely executable sequence of scanning points, point-by-point programmed control of the image acquisition device is achieved, ensuring that the scanning action can systematically and completely cover the planned target road segment trajectory, avoiding scanning omissions or repetitions.

[0009] As one possible implementation, the step of generating a sequence of control commands to control the image acquisition device to point sequentially at each virtual observation point based on multiple virtual observation points obtained from sampling includes: acquiring real-time pose data of the vehicle; calculating the target azimuth and target pitch angles of the image acquisition device pointing at the virtual observation points based on the road geometry information corresponding to the virtual observation points and the real-time pose data of the vehicle; and generating the sequence of control commands based on the target azimuth and target pitch angles.

[0010] By combining the real-time pose of the vehicle with road information for joint calculation, it is possible to compensate for the deviation caused by the vehicle's own movement in the calculation of the aiming point, thereby improving the angle control accuracy of the image acquisition equipment pointing to the virtual observation point of the target and ensuring that the camera equipment accurately aims at the center line position of the target road segment.

[0011] As one possible implementation, the real-time pose data includes the vehicle's real-time position and real-time heading angle, and the road geometry information includes the road heading angle and road slope; wherein, calculating the target azimuth and target pitch angles of the image acquisition device pointing to the virtual observation point includes: determining the target azimuth angle based on the road heading angle at the virtual observation point and the vehicle's real-time heading angle; determining the target pitch angle based on the spatial relationship between the virtual observation point and the vehicle, and the road slope of the virtual observation point; the spatial relationship is determined according to the vehicle's real-time position.

[0012] By using the difference between the road heading angle and the vehicle heading angle to accurately calculate the horizontal aiming angle, and combining road slope information to correct the vertical pitch angle, the control commands can adapt to both the horizontal curvature and longitudinal slope changes of the road, ensuring that the line of sight can be accurately aligned with the center line of the target road segment under different road conditions.

[0013] As one possible implementation, after generating control commands based on the scanning path, the vehicle forward road condition perception method further includes: acquiring multiple frames of road condition images of the target road segment ahead, acquired by the image acquisition device; performing geometric correction on the multiple frames of road condition images; and stitching the corrected images together based on the road geometric information to generate a stitched road condition image covering the target road segment ahead.

[0014] By correcting and stitching discretely acquired multi-frame images based on road geometry, a continuous wide-view image covering the road segment behind the target road segment can be synthesized, thus providing drivers or subsequent processing systems with intuitive and complete overall situational information behind the target road segment, solving the problem that discrete frame images are difficult to understand intuitively.

[0015] As one possible implementation, the vehicle forward road condition perception method further includes: identifying risk targets based on the stitched road condition image; mapping the location information of the risk targets to the vehicle's display device; and providing graded warnings based on the location information of the risk targets.

[0016] Based on the obtained panoramic road condition images, further identification of risk targets and location mapping and graded warnings can transform perceived information into direct and easy-to-understand driving assistance prompts, improving the practicality of road condition perception results and the efficiency of supporting driving decisions.

[0017] As one possible implementation, the tiered early warning based on the location information of the risk target includes: when the distance between the risk target and the vehicle is less than a first distance threshold, displaying the location and distance information of the risk target and outputting a first-level voice prompt; when the distance between the risk target and the vehicle is less than a second distance threshold, increasing the visual warning intensity of the location and distance information and outputting a second-level voice prompt; wherein the volume of the second-level voice prompt is greater than the volume of the first-level voice prompt, and the second distance threshold is less than the first distance threshold; when the distance between the risk target and the vehicle is less than a third distance threshold, generating an emergency signal and sending it to the vehicle stability system and braking system; wherein the third distance threshold is less than the second distance threshold.

[0018] By setting multi-level distance thresholds and matching different intensity warning methods, refined warnings based on the degree of risk proximity are achieved. This not only provides sufficient early warnings but also triggers the pre-response of the proactive safety system in emergency situations, thereby improving the rationality and safety of the warning system.

[0019] According to a second aspect of this application, an image acquisition device for vehicle forward road condition perception is provided, comprising: a camera device; a driving pan-tilt unit for driving the camera device to rotate in azimuth and pitch directions; and a control interface configured to receive control commands in the vehicle forward road condition perception method as described in the first aspect or any implementation thereof, and to control the driving pan-tilt unit according to the control commands so that the driving pan-tilt unit drives the camera device to change its orientation along the spatial morphology of the target road segment ahead.

[0020] According to a third aspect of this application, a vehicle is provided, comprising: an image acquisition device as described in the second aspect; and a processing device configured to perform a vehicle forward road condition perception method as described in the first aspect or any implementation thereof, and communicatively connected to the image acquisition device.

[0021] According to a fourth aspect of this application, a vehicle forward road condition perception device is provided, comprising: an information acquisition module for acquiring road geometry information of a target road segment ahead of the vehicle, the road geometry information being used to characterize the spatial morphology of the target road segment ahead; a path planning module for planning a scanning path for image acquisition based on the road geometry information; wherein the scanning path is used to guide the pointing of the image acquisition device so that the pointing of the image acquisition device changes with the spatial morphology of the target road segment ahead; and a control command generation module for generating control commands according to the scanning path; wherein the control commands are used to control the image acquisition device to perform scanning according to the scanning path to acquire a road condition image of the target road segment ahead.

[0022] According to a fifth aspect of this application, a computer device is provided, the computer device comprising: one or more processors; a memory; and one or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the processor to implement the method as described in the first aspect or any implementation thereof.

[0023] According to a sixth aspect of this application, a computer-readable storage medium is provided, the storage medium storing a computer program for performing the method as described in the first aspect or any implementation thereof.

[0024] According to a seventh aspect of this application, an electronic device is provided, including a module for performing the method as described in the first aspect or any implementation thereof.

[0025] According to an eighth aspect of this application, a computer program product is provided, comprising program code for performing the method as described in the first aspect or any implementation thereof.

[0026] The vehicle forward road condition perception method, image acquisition device, and vehicle provided in this application provide a crucial data foundation for actively guiding the perception direction by introducing road geometric information that characterizes the spatial morphology of the target road segment ahead. Furthermore, by planning a scanning path based on the road geometric information, the pointing of the image acquisition device can dynamically follow changes in the morphology of the target road segment, achieving active matching between the perception direction and the road orientation. Control commands are generated and executed according to the scanning path, enabling the image acquisition device to flexibly acquire images along the trajectory of the target road segment. This overcomes the physical limitations of a fixed field of view, extending the effective image acquisition range from the straight line in front to a continuous area behind the target road segment. This achieves beyond-line-of-sight, continuous image information capture of the blind spot road conditions of the target road segment, solving the perception failure problem in target road segment scenarios, providing drivers with visual information behind the target road segment, increasing risk warning time, and contributing to improved driving safety. Attached Figure Description

[0027] The above and other objects, features, and advantages of this application will become more apparent from the more detailed description of the embodiments of this application in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain this application and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0028] Figure 1 This is a schematic diagram of the architecture of a vehicle forward road condition perception system provided in an exemplary embodiment of this application.

[0029] Figure 2 This is a flowchart illustrating a method for perceiving road conditions ahead of a vehicle, provided in an exemplary embodiment of this application.

[0030] Figure 3 This is a schematic diagram of the structure of a vehicle forward road condition sensing device provided in an exemplary embodiment of this application.

[0031] Figure 4 This is a structural diagram of an electronic device provided in an exemplary embodiment of this application. Detailed Implementation

[0032] Hereinafter, exemplary embodiments according to this application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments of this application. It should be understood that this application is not limited to the exemplary embodiments described herein.

[0033] Nighttime driving, especially on highways with complex lighting conditions, is a high-risk scenario for traffic accidents. Due to the unique driving environment at night, drivers face numerous challenges, and existing vehicle safety systems have significant shortcomings in addressing these challenges. First, there is a physical limitation to environmental perception, namely, limited visibility. The primary challenge for nighttime highway driving is insufficient lighting. Currently, the effective illumination distance of traditional passenger car active lighting systems is typically limited to 150 to 300 meters, still unable to achieve full illumination of the road ahead. This limited visibility directly prevents drivers from anticipating potential hazards within a range of 500 meters to 1 kilometer. These hazards include, but are not limited to: sudden traffic accident scenes, road obstacles caused by spilled cargo vehicles, illegally parked disabled vehicles, and the end of congested queues caused by traffic saturation. When a vehicle is cruising at a high speed of 120 km / h, these distant situations constitute blind spots, reducing driving behavior from proactive anticipation to reactive response. Furthermore, based on these limited visibility, when a dangerous target enters the headlight's illumination range, the driver has very little reaction time, easily leading to secondary accidents.

[0034] Currently, to address nighttime driving safety, existing vehicles are widely equipped with various environmental perception sensors, including millimeter-wave radar, lidar, and optical cameras. However, this combination of technologies still has shortcomings when dealing with nighttime highway conditions: millimeter-wave radar has good detection capabilities for moving targets, but it faces challenges in identifying stationary targets, often filtering them out as stationary clutter, leading to missed detections. While vehicle-mounted cameras have advantages in target classification, their performance is highly dependent on ambient light. In low-light conditions at night, the image signal-to-noise ratio decreases, and the focal length and field of view of ordinary vehicle-mounted cameras are mainly designed for short- to medium-range scenarios, making it difficult to capture subtle changes in light or small obstacles hundreds of meters away, thus failing to achieve effective beyond-line-of-sight perception. Traditional vehicles are often information silos, unable to acquire road hazard information beyond the field of view of their own sensors.

[0035] Therefore, how to overcome the physical and perceptual limits of traditional vehicle sensors in nighttime environments, provide drivers with beyond-line-of-sight warning capabilities, and inform them of potential risks within several kilometers ahead in advance has become a key technical problem that urgently needs to be solved to improve driving safety on highways at night.

[0036] To achieve beyond-line-of-sight road condition perception ahead of a vehicle, image acquisition devices equipped with telephoto lenses can be used. These devices combine high-sensitivity, long-focal-length optical lenses with image sensors to acquire clear images of distant areas directly in front of the vehicle or in a predetermined direction. The basic working principle is to leverage the narrow field of view and high resolution of the telephoto optical system to compensate for the limitations of the human eye and conventional vehicle cameras in long-distance detection. However, when this solution is applied to road sections with specific geometric shapes, the inherently narrow field-of-sight optical design of the image acquisition device, designed to optimize long-distance imaging resolution, inevitably impairs its ability to detect areas not aligned with its fixed visual axis. In other words, when the spatial shape of the road ahead of the vehicle does not match the device's fixed orientation, the fixed orientation of the telephoto lens deviates from the actual spatial direction of the road, and its visual axis cannot follow the changes in road geometry. This results in the area behind the road being completely within the device's blind spot due to physical obstruction, rendering the beyond-line-of-sight perception function essentially ineffective in such scenarios.

[0037] From a physical perspective of information acquisition, optical systems with narrow field of view effectively detect a narrow, approximately straight cone-shaped area that is relatively fixed in space. This inherently conflicts with the continuously changing shape of curved roads in three-dimensional space. From a control logic perspective, the pointing of image acquisition devices in related technical solutions is usually fixed or performs only simple, regular scanning. Their control logic is independent of the specific geometry of the external road environment and lacks a mechanism to allow the acquisition pointing to actively adapt to the road's orientation. These factors collectively lead to the limitations of existing sensing solutions in dealing with this common road morphology.

[0038] Therefore, to overcome the aforementioned technical limitations, this application proposes a method for perceiving road conditions ahead of a vehicle, an image acquisition device, and a vehicle. By introducing road geometric information representing the spatial morphology of the road ahead as prior knowledge, and based on this, planning the scanning path for image acquisition and generating corresponding control commands, the pointing control link of the image acquisition device is improved. Thus, without sacrificing the inherent advantage of long-distance imaging resolution, the effective detection range under specific road geometric morphologies is expanded, avoiding perception blind spots caused by the mismatch between the field of view and the road direction.

[0039] To enable vehicles to perceive road conditions ahead in advance. Figure 1 This is a schematic diagram of the architecture of a vehicle forward road condition perception system provided in an exemplary embodiment of this application. This system is integrated into the vehicle's electronic system and can be used to extend the vehicle's visual perception range, especially at night or in low visibility conditions, to detect blind spots with specific geometric shapes ahead of the vehicle beyond visual range and to provide graded warnings for the identified risks. Figure 1 As shown, the system adopts a layered modular architecture, including a perception layer 100, a data processing and decision-making layer 200, and an early warning and execution layer 300. Signals and data are transmitted between the layers via an internal vehicle bus (such as CAN bus or Ethernet) or a dedicated data channel.

[0040] The perception layer 100 is responsible for collecting raw data from the physical world and is the hardware foundation for the system to achieve beyond-line-of-sight perception. The perception layer 100 consists of multiple heterogeneous sensor units that work together to provide comprehensive and redundant environmental information. The perception layer 100 may include: The telephoto camera unit 110 is a camera module employing a high-sensitivity image sensor and a large-aperture optical lens. It can effectively capture clear images of targets within a distance of 500 to 1000 meters in front of the vehicle. To overcome the limitations of a fixed field of view, the telephoto camera unit 110 can be mounted on a two-axis drive gimbal. The drive gimbal can perform high-precision rotation scanning in both the horizontal (azimuth) and vertical (pitch) directions, thereby dynamically adjusting the pointing of the telephoto camera. Driven by control commands, the telephoto camera unit 110 can execute a preset scanning path, achieving directional observation of a distant area in a specific direction.

[0041] The wide-angle camera unit 120 has a wider field of view than the telephoto camera unit 110, and can be used to monitor the overall road conditions within a range of, for example, 150 to 300 meters in front of the vehicle. The wide-angle camera unit 120 provides a wide-field scene image, primarily used to assist vehicle positioning, perceive sudden situations at close range, and provide near-field reference information for the scanning of the telephoto camera unit 110. In some embodiments, the wide-angle camera unit 120 can be fixedly mounted or have limited rotation capability.

[0042] Other onboard sensor units 130 allow the system to integrate or connect to other existing vehicle sensors to fuse multi-source data and improve the reliability and accuracy of perception. These sensors may include millimeter-wave radar, lidar, and a global positioning system (GPS) and inertial measurement unit (INS). Millimeter-wave radar and lidar provide distance, speed, and angle information of targets, effectively supplementing visual perception, especially in adverse weather conditions, and are used to verify visual recognition results. The GPS and INS continuously acquire the vehicle's real-time pose data, including precise geographic coordinates, speed, heading angle, and pitch and roll angles. This real-time pose data is fundamental for calculating scanning direction, performing image stitching, and locating hazardous targets.

[0043] The data processing and decision-making layer 200 is the system's computing hub, responsible for processing the raw data uploaded from the perception layer, extracting valuable information, and making risk assessments and decisions. The data processing and decision-making layer 200 can be implemented using an in-vehicle computing platform or a high-performance domain controller. The data processing and decision-making layer 200 may include: The image fusion and enhancement module 210 receives raw image data from the telephoto camera unit 110 and the wide-angle camera unit 120. First, the images are preprocessed, including noise reduction and contrast enhancement. Particularly in night mode, a low-light image enhancement algorithm is employed to improve image usability. Then, the near-panoramic view from the wide-angle camera and the distant close-up view from the telephoto camera are aligned and fused to generate a comprehensive virtual telescope view, providing a more intuitive observation interface for the driver or subsequent algorithms.

[0044] The AI ​​risk recognition algorithm engine 220, built upon a deep learning model, performs real-time analysis of long-distance image sequences captured by the telephoto camera unit 110 or fused views generated by the image fusion and enhancement module 210. The AI ​​risk recognition algorithm engine 220 can embed a target detection and classification neural network to identify various types of risk targets, such as stationary or slow-moving vehicles, warning triangles, road debris, and construction zone signs. The AI ​​risk recognition algorithm engine 220 can output the identified target category and its bounding box information in the image.

[0045] The risk assessment and decision-making unit 230 integrates the output of the AI ​​risk recognition algorithm engine 220, the vehicle speed and pose data from the perception layer 100, and road geometry information (such as curvature) obtained from the navigation system or high-precision map. By continuously calculating the relative distance and relative speed between the risk target and the vehicle, and combining this with the road alignment to determine the collision risk, the risk assessment and decision-making unit 230 can calculate a quantified threat level. Based on the threat level, the risk assessment and decision-making unit 230 determines the warning level that should be triggered (such as Level 1 alert, Level 2 warning, Level 3 emergency preparedness) according to a preset strategy and generates corresponding response strategy instructions.

[0046] The warning and execution layer 300 is responsible for converting the decision results of the data processing and decision-making layer 200 into perceptible prompts for the driver, and, when necessary, for initial linkage with the vehicle control system. It serves as the interface for interaction between the system, the driver, and the vehicle. The warning and execution layer 300 includes: The graded warning and alert unit 310 executes human-machine interaction based on instructions issued by the risk assessment and decision-making unit 230. For example, it can be divided into three levels: Level 1 provides a warning, Level 2 provides a alert, and Level 3 directly prepares for emergency intervention by sending emergency signals to the vehicle stability control system and braking system via the vehicle bus. Emergency signals can be used to pre-tension seat belts, pre-establish brake line pressure, or adjust the response parameters of the electronic stability program and brake assist system, providing the driver with system response time for potential emergency avoidance or braking maneuvers.

[0047] The vehicle-to-everything (V2X) communication unit 320 supports the V2X communication protocol. The system can broadcast information about perceived risk targets (such as type, location, and speed) to surrounding vehicles via the V2X communication unit 320, enabling vehicle-to-vehicle cooperative early warning. Simultaneously, the V2X communication unit 320 can also receive early warning information from roadside facilities or other vehicles and incorporate it into its own risk assessment and decision-making processes, thereby expanding the perception boundary and achieving collaborative safety for the entire vehicle group.

[0048] The aforementioned system architecture provides the hardware and module foundation for realizing vehicle-ahead road condition perception methods. Based on the vehicle-ahead road condition advanced perception system, Figure 2 This is a flowchart illustrating a vehicle forward road condition perception method provided in an exemplary embodiment of this application. Figure 2 For example, firstly, obtain the road geometry information of the target road segment ahead of the vehicle (see...). Figure 2 (S21) Road geometry information is used to characterize the spatial morphology of the target road segment ahead. Then, based on the road geometry information, a scanning path for image acquisition is planned (see S21). Figure 2 (S22). The scan path guides the direction of the image acquisition device so that its direction changes with the spatial morphology of the target road segment ahead. Finally, control commands are generated based on the scan path (see S22). Figure 2 S23), the control command is used to control the image acquisition device to perform scanning according to the scanning path in order to obtain road condition images of the target road segment ahead.

[0049] The following text combines Figure 2 This application provides a more detailed description of the vehicle forward road condition perception method provided in the embodiments.

[0050] In S21, road geometry information of the target road segment ahead of the vehicle is obtained.

[0051] In some embodiments, a target road segment refers to a road area in front of the vehicle where, due to the spatial geometry of the road itself, the effective field of view of the image acquisition device installed on the vehicle cannot be fully covered in its current fixed pointing or conventional scanning mode, thus forming a perception blind spot. In other words, there is a continuous and non-negligible spatial deviation between the actual direction of the road segment and the default observation pointing of the original image acquisition device. The target road segment is not a fixed value but is dynamically determined based on the vehicle's current driving state and safety requirements. That is, the length of the target road segment can cover a continuous area in front of the vehicle's trajectory over a future period where perception may be limited due to road geometry. For example, firstly, a basic spatial range is calculated based on the vehicle's current speed and a preset safe time reference. The boundary of the spatial range is the farthest point that should be prioritized for exploration to meet safety response requirements, calculated forward along the road from the vehicle's current position. Subsequently, the system combines road geometry information to identify all continuous road segments within the spatial range that form potential blind spots due to mismatch with the default pointing of the image acquisition device, and defines these continuous road segments as the target road segment for the current perception cycle. Therefore, the length of the target road segment is determined by the current vehicle speed and the road's geometry, and is dynamically updated as the vehicle moves.

[0052] For target road segments, the following typical road conditions can be included: First, curves, i.e., road segments with significant curvature, whose centerline is a curve in planar or three-dimensional space. This is the most common situation causing a mismatch between the field of view of a fixed-pointing vision device and the road direction; Second, the crest of a slope, where the downhill section ahead is obscured by the slope itself as the vehicle approaches the crest, creating a blind spot; Third, special straight road segments with a central median or continuous roadside obstacles. Although the road alignment is straight, the high median or continuous obstacles will obstruct direct observation of adjacent lanes for a long time, requiring adjustment of the device's pointing direction to overcome the obstacles for observation; Fourth, combined road segments, such as road segments combining curves and slopes (slope curves), whose spatial morphology is more complex and the obstruction effect is stronger. Determining whether a road segment belongs to the target road segment requiring dynamic pointing adjustment involves comparing the road geometric information (such as curvature and slope changes) with the current or default field of view of the image acquisition device. If, based on road geometry information, a continuous portion of the road segment is located outside the current field of view of the device, then that road segment is identified as the target road segment.

[0053] In some embodiments, road geometry information is used to characterize the spatial morphology of the target road segment ahead, and may include, but is not limited to, precise geometric data such as road centerline, curvature, heading angle, slope, and elevation. The sources of road geometry information can be diverse. For example, one acquisition method is to query and extract the parametric geometric information of the required road segment from a high-precision map pre-stored in the vehicle or in the cloud, based on the vehicle's real-time location and heading; or to construct a geometric model of the road ahead in real time using the vehicle's own perception system combined with real-time localization and mapping algorithms. Road geometry information can provide basic data describing the spatial morphology of the road for subsequent scanning control.

[0054] For example, the system obtains the vehicle's current position P_v(x, y, z, yaw), where x, y, and z are the vehicle's coordinates in the spatial domain, and yaw is the real-time heading angle in the direction the vehicle is pointing. Simultaneously, it reads the parameterized centerline curve C(s) of the road ahead from a high-precision map, where s is the arc length along the centerline. Curve C(s) contains key geometric parameters such as curvature κ(s), heading angle ψ(s), and slope θ(s) at each arc length position s. This information collectively defines the three-dimensional spatial shape of the road ahead, especially curves.

[0055] In S22, a scanning path for image acquisition is planned based on road geometry information.

[0056] S22 transforms the abstract road shape into a concrete action blueprint that can be executed by the image acquisition device. The scanning path guides the direction of the image acquisition device, ensuring its orientation changes according to the spatial form of the target road segment ahead. The essence of planning is to calculate a series of control objectives that allow the image acquisition device's line of sight to extend forward, closely aligned with the road's centerline.

[0057] In some embodiments, the image acquisition device refers to a camera device with long-distance imaging capability, such as a telephoto camera unit including a telephoto lens and a high-sensitivity sensor, and can be configured with a drive gimbal to achieve pointing adjustment.

[0058] In some embodiments, to further improve the rationality and safety of scanning path planning, the implementation of scanning path planning based on road geometry information can be as follows: Based on the current vehicle speed, determine the safety perception time. Vehicle speed represents the vehicle's state, ensuring the system's perception range matches the vehicle's state and provides sufficient warning time. The safety perception time is a pre-set time threshold based on vehicle dynamics, system processing latency, and driver reaction models, such as 3 seconds, 5 seconds, or longer, representing the minimum reaction time window the system aims to provide for the driver. The safety perception time can be determined through direct lookup table mapping or as a function negatively correlated with vehicle speed; for example, the higher the vehicle speed, the longer the required safety perception time. Based on the current vehicle speed and the safety perception time, determine the lookahead distance. The lookahead distance refers to the length of road that needs to be prioritized for road condition perception and risk detection, calculated forward along the centerline of the target road segment from the vehicle's current position to allow the vehicle sufficient reaction and handling time. The lookahead distance is a dynamically calculated value. The lookahead distance S_lookahead defines the range of road arc lengths that need to be covered in this scan. The calculation method is: S_lookahead = v × T, where v is the current vehicle speed and T represents the safety perception time. That is, the lookahead distance is determined by both the current vehicle speed and the preset safety perception time. For example, if the vehicle speed v is 120 km / h (approximately 33.3 m / s) and the safety perception time T is 5 seconds, then the lookahead distance S_lookahead is approximately 166.5 meters. Setting a dynamically changing lookahead distance allows the scanning range to adapt dynamically to speed changes, seeing further at high speeds and focusing on closer areas at low speeds. Virtual observation points are sampled along the centerline of the target road segment ahead, based on the lookahead distance and road geometry information, and the scanning path is determined based on these sampled virtual observation points. Based on the lookahead distance, a target road segment of appropriate length is extracted from the road geometry information, and the scanning path is planned and a sequence of virtual observation points is generated. By dynamically matching the scanning coverage with speed-based safety requirements, the system ensures that it always focuses on the road area most relevant to current driving safety.

[0059] As one possible implementation, planning the scanning path can specifically involve generating a series of virtual observation points distributed along the road centerline based on road geometry information. These virtual observation point sequences constitute the scanning path. For example, starting from the projection point s0 of the vehicle's current position on the road centerline, sampling is performed along the centerline at a certain sampling interval Δs within the arc length of the look-ahead distance S_lookahead. The sampling interval Δs can be 10 meters. This generates a set of discrete virtual observation points P_i = C(s0 + i×Δs), i=1, 2, ..., n, until the cumulative arc length reaches or exceeds S_lookahead. This series of points {P_i} constitutes the path for this scan. Using this method, the scanning range is strongly correlated with driving safety, avoiding the resource waste caused by unnecessary long-distance scanning or the insufficient warning time caused by short-distance scanning.

[0060] Understandably, the sampling interval can be fixed or adaptive. For example, in curved sections of a road with significant curvature, a smaller sampling interval (e.g., 5 meters) can be used to obtain denser path points, thereby more accurately tracking the road curvature. In straight sections, a larger sampling interval (e.g., 20 meters) can be used to improve scanning efficiency.

[0061] In S23, control commands are generated based on the scan path.

[0062] In some embodiments, control commands are used to instruct the image acquisition device to perform scanning along a scanning path to obtain road condition images of the target road segment ahead. The content of the control commands directly depends on the form of the scanning path. If the scanning path is a sequence of virtual observation points, the control commands need to drive the image acquisition device to point to these points sequentially. If the scanning path is an angular curve, the control commands need to drive the pan-tilt unit to move according to a specific angular function.

[0063] In some embodiments, after determining the scanning path consisting of a sequence of virtual observation points, in order to convert it into a precise action sequence that can be executed by the image acquisition device, a control instruction sequence for controlling the image acquisition device to point to each virtual observation point in sequence, and an acquisition instruction sequence for acquiring images at each pointing direction can be generated based on the multiple virtual observation points obtained by sampling.

[0064] The control command sequence drives the pan-tilt-zoom (PTZ) movement, aligning the camera's optical axis sequentially with each virtual observation point P_i. The acquisition command sequence triggers the shutter after each point of contact stabilizes, capturing a frame of road condition image in that direction. These two sequences work in a staggered, sequential fashion to complete the scanning and acquisition process for mobile photography. Compared to controlling the PTN to rotate continuously at a constant speed and take continuous photos, this discrete point sequence control method offers advantages such as clear control logic, accurate target pointing, and stable image acquisition timing, facilitating subsequent precise geometric correlation processing of each frame.

[0065] Understandably, alternative control strategies to achieve the same purpose include: the gimbal scanning continuously at a slower speed while triggering capture at a preset angle position; or, controlling the gimbal to scan back and forth between the start and end points of the scanning path and acquiring images at a fixed frequency during the scanning process.

[0066] In some embodiments, to ensure that the angle parameters in the control command sequence accurately guide the image acquisition device to aim at a virtual observation point on the road, rather than pointing it into the air or on the ground, more comprehensive information is needed for precise calculation. To this end, real-time vehicle pose data can be acquired; based on the road geometry information corresponding to the virtual observation point and the vehicle's real-time pose data, the target azimuth and pitch angles of the image acquisition device pointing towards the virtual observation point can be calculated; and based on the target azimuth and pitch angles, a control command sequence can be generated.

[0067] For example, the vehicle's real-time pose data includes at least the vehicle's position (x_v, y_v, z_v) and real-time heading angle ψ_v. Road geometry information provides the position (x_i, y_i, z_i) of each virtual observation point P_i, the road's heading angle ψ_i, and the slope θ_i. Using these two sets of data, the azimuth angle α_i and pitch angle β_i required by the image acquisition device when viewed from the vehicle's current position towards the virtual observation point P_i can be accurately calculated. This conversion process allows the driving gimbal to compensate for changes in the vehicle's yaw, pitch, and roll, as well as changes in road slope, improving the accuracy of aiming at the center line and preventing target loss due to angular deviations.

[0068] During angle calculation, real-time pose data includes the vehicle's real-time position and real-time heading angle ψ_v, while road geometry information includes the road heading angle ψ_i and road slope θ_i at the virtual observation point. Therefore, the process of calculating the target azimuth and pitch angles of the image acquisition device pointing to the virtual observation point can be as follows: Determine the target azimuth angle based on the road heading angle at the virtual observation point and the vehicle's real-time heading angle; determine the target pitch angle based on the spatial relationship between the virtual observation point and the vehicle, and the road slope at the virtual observation point; the spatial relationship is determined based on the vehicle's real-time position.

[0069] For the target azimuth angle α_i, one possible calculation formula is: α_i = ψ_i―ψ_v, where ψ_v represents the vehicle's real-time heading angle, ψ_i represents the road heading angle, and the target azimuth angle is the offset angle of the road direction at the target point relative to the vehicle's heading direction. When the vehicle is traveling straight, if the road turns right ahead, ψ_i will be greater than ψ_v, and the calculated α_i will be positive, meaning the gimbal needs to rotate to the right by the corresponding angle. Calculating the target azimuth angle allows the optical axis of the image acquisition device to be aligned with the tangent direction of the road at the target point.

[0070] For the target pitch angle β_i, one possible formula is: β_i = arctan((z_i - z_v) / Δd) + θ_i, where (z_i - z_v) represents the height difference, Δd is the horizontal distance, and θ_i is the slope compensation term. If the road is uphill at point P_i (θ_i>0), the angle needs to be further raised to make the line of sight parallel to the slope surface, avoiding aiming above the top of the slope. This calculation formula ensures that the aiming line of the image acquisition device is aligned with the road slope surface, rather than a horizontal straight line, avoiding aiming at the sky or the ground.

[0071] After calculating the target azimuth and pitch angles, the system controls the gimbal to traverse the set of angles [α_1, β_1] to [α_n, β_n] at an extremely high speed (e.g., within 100 milliseconds). Once each angle stabilizes, the system triggers the camera to capture a high-resolution image I_i. This allows the image acquisition device to rapidly scan multiple key areas behind the target road segment in a very short time, similar to an electronic searchlight, thus forming a scanning strategy.

[0072] After acquiring multiple discrete road condition images of the target road segment ahead through dynamic scanning, an image post-processing step can be added to provide the driver or downstream autonomous driving module with a more intuitive and complete view of the situation behind the target road segment. For example, taking a curve as an example, the image acquisition device acquires multiple frames of road condition images of the curve ahead; the multiple frames of road condition images are geometrically corrected, and the corrected images are stitched together based on the road geometry information to generate a stitched road condition image covering the curve ahead.

[0073] Image post-processing can address the fragmentation of information in discrete frame images. Since each road condition image is captured from different orientations and elevation angles, it exhibits varying perspective distortions, making it difficult to form a coherent understanding of the road behind the target road segment upon direct viewing. By utilizing camera intrinsics and the actual elevation angle β_i at the time of capture, an inverse perspective transformation can be performed on each road condition image I_i, correcting it to appear as an orthographic view at point P_i, along the road tangent and parallel to the slope. Next, all corrected road condition images are projected onto a unified road surface model based on the precise location of their corresponding P_i points in the geodetic or vehicle coordinate system. This road surface model can be constructed from the centerline and elevation information in a high-precision map. This generates a wide-angle, stitched bird's-eye view covering the entire target road segment, without repetition and with correct geometric relationships. It clearly and continuously displays lane lines and potential obstacles behind the target road segment, improving the intuitiveness and comprehensibility of the information.

[0074] In some embodiments, safety warnings can be implemented based on the obtained stitched road condition images. For example, risk targets can be identified based on the stitched road condition images; the location information of the risk targets can be mapped to the vehicle's display device; and graded warnings can be issued based on the location information of the risk targets.

[0075] Implementing safety warnings allows vehicle-ahead road condition perception methods to complete a closed loop from perception to decision-making and then to human-machine interaction. Risk target identification based on stitched road condition images can utilize mature target detection algorithms (such as YOLO and SSD models based on convolutional neural networks). Target types identified can include stationary vehicles, debris, pedestrians, and warning triangles. After target identification, its pixel coordinates in the stitched image can be obtained, and then combined with road geometry information, its position (distance and orientation) in the real road coordinate system can be calculated. The risk target's location information is mapped to a display device, such as a head-up display or dashboard, projecting the risk target behind the target road segment onto the driver's actual field of vision or map interface in the form of icons, highlighted boxes, or augmented reality arrows, intuitively indicating that the target is approximately 150 meters behind the target road segment to the left. Tiered warnings calculate the risk level based on the target's distance, relative speed, etc., and trigger different levels of alerts. The entire process makes the previously invisible risks behind the target road segment visible, quantifiable, and predictable. For example, if the system detects a stationary vehicle 200 meters away in the stitched image, it can display a semi-transparent vehicle icon at the corresponding position on the AR-HUD.

[0076] As a possible tiered early warning implementation strategy, when the distance between the risk target and the vehicle is less than a first distance threshold, the location and distance information of the risk target are displayed, and a first-level voice prompt is output; when the distance between the risk target and the vehicle is less than a second distance threshold, the visual warning intensity of the location and distance information is enhanced, and a second-level voice prompt is output; wherein, the volume of the second-level voice prompt is greater than the volume of the first-level voice prompt, and the second distance threshold is less than the first distance threshold; when the distance between the risk target and the vehicle is less than a third distance threshold, an emergency signal is generated and sent to the vehicle stability system and braking system; wherein, the third distance threshold is less than the second distance threshold.

[0077] The first distance threshold (e.g., 500 meters) corresponds to the early perception and mild warning stage. At this time, the risk target icon and its distance are displayed as a semi-transparent icon on the HUD (Head-up Display) or central control screen, accompanied by a gentle voice prompt, "There may be an obstacle 1 kilometer after the curve ahead, please be careful," to attract the driver's attention without causing alarm. The second distance threshold (e.g., 200 meters) corresponds to the risk approach and explicit warning stage. At this time, the icon on the HUD may turn red and flash, while a louder, more urgent voice alarm is issued, and the seat or steering wheel may vibrate tactilely to prompt the driver to take immediate action. The third distance threshold (e.g., 100 meters) corresponds to the emergency preparation stage. At this time, the system determines that the collision risk is extremely high. In addition to the strongest audible and visual warnings, an emergency signal is sent to the vehicle stability system and electronic braking system via the CAN bus. The emergency signal can be used to pretension seat belts, pre-fill brake line pressure, or adjust the response characteristics of the brake assist system, providing the fastest system response support for any emergency braking the driver may take, and shortening the vehicle's braking reaction time. The three thresholds constitute a dynamic early warning space, enabling a smooth transition from information alerts to proactive intervention preparations.

[0078] Understandably, the specific values ​​of each distance threshold can be dynamically calibrated or determined by looking up tables based on factors such as vehicle type, vehicle speed, and road adhesion coefficient. For example, the first distance threshold can be associated with the lookahead distance S_lookahead; the second distance threshold can be calculated based on the braking distance at the current speed plus a safety margin; and the third distance threshold corresponds to more extreme operating conditions. Furthermore, it is not limited to dividing the vehicle into three distance thresholds; it can be divided into more or fewer warning levels according to actual needs.

[0079] Based on the same inventive concept, this application also provides an image acquisition device for vehicle forward road condition perception. The image acquisition device includes a camera device, a driving pan-tilt unit, and a control interface. The camera device can be a camera module with a long focal length, a high-sensitivity image sensor, and a large aperture lens, responsible for capturing clear images from a distance, such as a telephoto camera unit. The driving pan-tilt unit is used to drive the camera device to rotate in the azimuth (horizontal) and pitch (vertical) directions, and can be a two-axis servo pan-tilt unit, a stepper motor pan-tilt unit, or a voice coil motor, etc., a precision motion mechanism. The control interface is configured to receive control commands generated as in any of the method embodiments described above, and control the driving pan-tilt unit according to the control commands, so that the direction of the camera device driven by the driving pan-tilt unit changes along the spatial morphology of the target road segment ahead. For example, the control interface parses the received control commands and converts them into control signals for the drive motor, thereby precisely controlling the pan-tilt unit to rotate the camera to a specified angle, achieving dynamic scanning. The image acquisition device can be a physical entity executing the vehicle forward road condition perception method embodiments.

[0080] This application also provides a vehicle. The vehicle includes the image acquisition device as described above, and a processing unit. The processing unit is configured to execute any of the vehicle forward road condition perception methods described above and is communicatively connected to the image acquisition device. The processing unit can be specifically implemented as the aforementioned data processing and decision layer 200, or an on-board computing platform or domain controller integrating the functions of this layer, for acquiring information, executing algorithms, generating control commands, and controlling the actions of the image acquisition device, together constituting a complete vehicle forward target road segment beyond-line-of-sight perception system.

[0081] Below is an example application scenario. Suppose a vehicle equipped with this system is traveling at 100 km / h on a highway at night, about to enter a right-turn curve. When the processing device predicts the curve using high-precision map data or forward perception, the process flow is initiated. First, based on the current vehicle speed (approximately 27.8 m / s) and the safety perception time (set to 4 seconds), the forward look-ahead distance is calculated to be approximately 111 meters. The system samples along the centerline of the curve at 10-meter intervals, generating a sequence of approximately 11 virtual observation points. Next, the processing device combines the real-time pose (position, vehicle facing 10 degrees east of north) provided by the vehicle's native perception equipment with road information (position, road direction, slope) at each virtual observation point to calculate 11 sets of target azimuth and pitch angles. For example, for a virtual observation point, the calculated azimuth angle is 25 degrees to the right. Then, the processing device sends a sequence of rapid scanning commands to the driving gimbal of the image acquisition device via the control interface. Within less than one second, the gimbal rapidly rotated to 11 designated angles sequentially. After stabilizing at each angle, the telephoto camera captured a high-resolution image, completing a scan of an area 111 meters behind the curve. After the road condition image data was transmitted back, the processing unit performed geometric correction and stitching on the 11 frames to generate a stitched bird's-eye view covering the entire right curve. From this stitched image, the algorithm identified a stationary vehicle, partially obstructing the driving lane, approximately 80 meters from the end of the curve.

[0082] The system then begins tiered warnings. First, since the target distance (80 meters) is less than the first distance threshold (e.g., 150 meters), the system displays a semi-transparent red vehicle icon on the AR-HUD, corresponding to the direction of the curve exit in the driver's actual field of vision, with 80 meters marked, and simultaneously plays a voice message: "Vehicle ahead at the curve, please be careful." The driver is immediately alerted and focuses their attention on the curve exit. The vehicle continues to approach. When the distance shrinks to 60 meters (less than the second distance threshold, e.g., 70 meters), the icon on the HUD begins to flash frequently, its color changes to dark red, and the voice warning escalates: "Warning, stationary vehicle ahead." The steering wheel vibrates briefly. The driver receives a clear warning and begins to apply the brake pedal. When the distance further closes to 50 meters (less than the third distance threshold, e.g., 55 meters), the system provides the strongest audible and visual warning while sending a preload signal to the vehicle stability system, and the braking system pre-pressurizes. At this point, the braking force applied by the driver receives a faster and more complete response, and the vehicle comes to a smooth stop at a safe distance from the stationary vehicle.

[0083] In this scenario, the driver's perception distance for hidden risks behind the curve increased from 0 to 80 meters, providing additional warning and reaction time. Throughout the process, the system operates automatically, providing intuitive and appropriately tiered warnings, assisting the driver in preventing potential secondary rear-end collisions and improving nighttime curve driving safety.

[0084] Figure 3 This is a schematic diagram of the structure of a vehicle forward road condition sensing device provided in an exemplary embodiment of this application, as shown below. Figure 3 As shown, the vehicle forward road condition perception device 3 includes: an information acquisition module 31, used to acquire road geometry information of the target road segment ahead of the vehicle, the road geometry information being used to characterize the spatial form of the target road segment ahead; a path planning module 32, used to plan a scanning path for image acquisition based on the road geometry information; wherein, the scanning path is used to guide the direction of the image acquisition device so that the direction of the image acquisition device changes with the changes in the spatial form of the target road segment ahead; and a control command generation module 33, used to generate control commands according to the scanning path; wherein, the control commands are used to control the image acquisition device to perform scanning according to the scanning path to acquire road condition images of the target road segment ahead.

[0085] As one possible implementation, the path planning module 32 can be configured to: determine the safety perception time based on the current vehicle speed; determine the look-ahead distance based on the current vehicle speed and the safety perception time; sample virtual observation points along the centerline of the target road segment ahead based on the look-ahead distance and road geometry information, and determine the scanning path based on the sampled virtual observation points.

[0086] As one possible implementation, the control command generation module 33 can be configured to: generate a sequence of control commands for the image acquisition device to point sequentially to each virtual observation point, and a sequence of acquisition commands for image acquisition at each pointing direction, based on the multiple virtual observation points obtained from sampling.

[0087] As one possible implementation, the control command generation module 33 can be configured to: acquire real-time pose data of the vehicle; calculate the target azimuth and target pitch angles of the image acquisition device pointing to the virtual observation point based on the road geometry information corresponding to the virtual observation point and the real-time pose data of the vehicle; and generate a sequence of control commands based on the target azimuth and target pitch angles.

[0088] As one possible implementation, the real-time pose data includes the vehicle's real-time position and real-time heading angle, and the road geometry information includes the road heading angle and road slope. The control command generation module 33 can be configured to: determine the target azimuth angle based on the road heading angle at the virtual observation point and the vehicle's real-time heading angle; and determine the target pitch angle based on the spatial positional relationship between the virtual observation point and the vehicle, as well as the road slope of the virtual observation point. The spatial positional relationship is determined according to the vehicle's real-time position.

[0089] As one possible implementation, the vehicle's forward road condition perception device 3 can be configured to: acquire multiple frames of road condition images of the target road segment ahead using an image acquisition device; perform geometric correction on the multiple frames of road condition images; and stitch the corrected images together based on road geometric information to generate a stitched road condition image covering the target road segment ahead.

[0090] As one possible implementation, the vehicle's forward road condition perception device 3 can also be configured to: identify risk targets based on stitched road condition images; map the location information of the risk targets to the vehicle's display device; and provide graded warnings based on the location information of the risk targets.

[0091] As one possible implementation, the vehicle forward road condition perception device 3 can also be configured to: display the location and distance information of the risk target when the distance between the risk target and the vehicle is less than a first distance threshold, and output a first-level voice prompt; enhance the visual warning intensity of the location and distance information when the distance between the risk target and the vehicle is less than a second distance threshold, and output a second-level voice prompt; wherein the volume of the second-level voice prompt is greater than the volume of the first-level voice prompt, and the second distance threshold is less than the first distance threshold; generate an emergency signal and send it to the vehicle stability system and braking system when the distance between the risk target and the vehicle is less than a third distance threshold; wherein the third distance threshold is less than the second distance threshold.

[0092] An electronic device includes: a processor; a memory for storing processor-executable instructions; and a processor for executing the vehicle forward road condition perception method described in the embodiments of this application.

[0093] Below, for reference Figure 4 This application describes an electronic device according to embodiments thereof. The electronic device may be either or both of a first device and a second device, or a standalone device independent of them, which may communicate with the first device and the second device to receive acquired input signals from them.

[0094] Figure 4 A block diagram of an electronic device according to an embodiment of this application is illustrated.

[0095] like Figure 4As shown, the electronic device 40 includes one or more processors 41 and memory 42.

[0096] The processor 41 may be a central processing unit (CPU) or other form of processing unit with data processing and / or instruction execution capabilities, and may control other components in the electronic device 40 to perform desired functions.

[0097] The memory 42 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 41 may execute the program instructions to implement the vehicle forward road condition perception methods of the various embodiments of this application described above and / or other desired functions. Various contents such as input signals, signal components, and noise components may also be stored in the computer-readable storage medium.

[0098] In one example, the electronic device 40 may also include an input device 43 and an output device 44, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0099] When the electronic device is a standalone device, the input device 43 can be a communication network connector for receiving the collected input signals from the first device and the second device.

[0100] In addition, the input device 43 may also include, for example, a keyboard, a mouse, etc.

[0101] The output device 44 can output various information to the outside, including determined distance information, direction information, etc. The output device 44 may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0102] Of course, for the sake of simplicity, Figure 4 Only some of the components of the electronic device 40 relevant to this application are shown in this illustration; components such as buses, input / output interfaces, etc., are omitted. In addition, the electronic device 40 may include any other suitable components depending on the specific application.

[0103] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this application. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0104] A computer-readable storage medium stores a computer program for executing the vehicle forward road condition perception method described in the embodiments provided in this application.

[0105] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0106] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this application to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A method for sensing a road condition in front of a vehicle, characterized by, The method comprises: acquiring road geometry information of a target road segment ahead of the vehicle; the road geometry information is used to represent the spatial form of the target road segment ahead; based on the road geometry information, a scanning path for image acquisition is planned; the scanning path is used to guide the pointing direction of an image acquisition device, so that the pointing direction of the image acquisition device changes with the change of the spatial form of the target road segment ahead; based on the scanning path, control instructions are generated; the control instructions are used to control the image acquisition device to perform scanning according to the scanning path, so as to acquire road condition images of the target road segment ahead.

2. The method of claim 1, wherein, The method further comprises: based on the road geometry information, a scanning path for image acquisition is planned; the scanning path is used to guide the pointing direction of an image acquisition device, so that the pointing direction of the image acquisition device changes with the change of the spatial form of the target road segment ahead; based on the current vehicle speed and the safe perception time, a forward-looking distance is determined; along the center line of the target road segment ahead, virtual observation points are sampled based on the forward-looking distance and the road geometry information, and a scanning path is determined based on the sampled virtual observation points.

3. The method of claim 2, wherein, The method further comprises: based on the sampled virtual observation points, a control instruction sequence for controlling the image acquisition device to point to each virtual observation point in turn is generated, and a capture instruction sequence for image acquisition at each pointing direction is generated.

4. The method of claim 3, wherein, The method further comprises: real-time pose data of the vehicle is acquired; based on the road geometry information corresponding to the virtual observation points and the real-time pose data of the vehicle, a target azimuth angle and a target elevation angle of the image acquisition device pointing to the virtual observation points are calculated; based on the target azimuth angle and the target elevation angle, the control instruction sequence is generated.

5. The method of claim 4, wherein, The real-time pose data comprises real-time position and real-time heading angle of the vehicle, and the road geometry information comprises road heading angle and road slope; The method further comprises: based on the road heading angle at the virtual observation points and the real-time heading angle of the vehicle, the target azimuth angle is determined; based on the spatial position relationship between the virtual observation points and the vehicle and the road slope of the virtual observation points, the target elevation angle is determined; the spatial position relationship is determined according to the real-time position of the vehicle.

6. The method of claim 1, wherein, After the control instructions are generated based on the scanning path, the method further comprises: a plurality of frames of road condition images of the target road segment ahead are acquired by the image acquisition device; the plurality of frames of road condition images are geometrically corrected, and the corrected images are spliced based on the road geometry information to generate a spliced road condition image covering the target road segment ahead.

7. The method of claim 6, wherein, The method further comprises: a risk target is identified based on the spliced road condition image; position information of the risk target is mapped to a display device of the vehicle, and hierarchical early warning is performed based on the position information of the risk target.

8. The method of claim 7, wherein, The hierarchical early warning based on the position information of the risk target comprises: When the distance between the risk target and the vehicle is less than a first distance threshold, the location and distance information of the risk target are displayed, and a first-level voice prompt is output. When the distance between the risk target and the vehicle is less than a second distance threshold, the visual warning intensity of the location information and the distance information is enhanced, and a secondary voice prompt is output; wherein, the volume of the secondary voice prompt is greater than the volume of the primary voice prompt, and the second distance threshold is less than the first distance threshold; When the distance between the risk target and the vehicle is less than a third distance threshold, an emergency signal is generated and sent to the vehicle stability system and braking system; wherein the third distance threshold is less than the second distance threshold.

9. An image acquisition device for road ahead perception of a vehicle, characterized in that, include: Camera equipment; A gimbal is used to drive the camera device to rotate in the azimuth and pitch directions; The control interface is configured to receive control commands in the vehicle forward road condition perception method as described in any one of claims 1-8, and to control the drive pan-tilt unit according to the control commands, so that the drive pan-tilt unit drives the camera device to change its pointing direction along the spatial morphology of the target road segment ahead.

10. A vehicle characterized by comprising: include: The image acquisition device as described in claim 9; A processing device configured to perform the vehicle forward road condition perception method as described in any one of claims 1-8, and communicatively connected to the image acquisition device.

Images