Guidance control method for assisting driving, electronic device, and vehicle

Abstract

The application provides a guide control method for assisting driving, an electronic device and a vehicle. After determining the effective light spots corresponding to the guide lights in the front view image, the discrete effective light spots are restored to accurate compensation position points in the vehicle coordinate system, thereby solving the problem of curvature calculation distortion caused by the fact that the near is large and the far is small in the camera vision. Curve fitting is performed based on the compensated compensation position points, and a visual guide line is rendered in the virtual image coordinate system according to the path curve. In the extreme scene without GPS, high-precision map and lane line, a continuous, accurate and risk warning function driving guide line on a curved road can still be provided, thereby greatly improving the driving safety and comfort in the night and low-visibility environment.

Description

Technical Field

[0001] This application relates to the field of intelligent driving and technology, and in particular to a guidance and control method, electronic equipment and vehicle for assisted driving. Background Technology

[0002] To guide drivers safely through curves at night or in low-visibility conditions, highways typically install LED linear guide lights or roadside contour lights on guardrails or along the roadside. The LEDs in these guide lights create a discontinuous array of light points at night, outlining or indicating the general direction of the road ahead, thus improving the driver's route perception. However, the guide lights installed along the roadside and guardrails present a discrete array of lights, resulting in poor visibility. Especially in rainy weather or when there is glare from oncoming headlights, it becomes even more difficult for drivers to piece together these widely spaced light points into a continuous curve trajectory. Summary of the Invention

[0003] In view of this, the purpose of this application is to propose a guidance control method, electronic device and vehicle for assisted driving, which obtains a visual guide line for guiding the user's driving by curve fitting and visualization of the planar position points corresponding to the guide lights, and assists the driver in predicting the curve path.

[0004] To achieve the above objectives, this application provides a guidance and control method for assisted driving, comprising:

[0005] Identify the effective light spot corresponding to the guide light in the front view image; Based on the pixel position of the effective light spot in the front view image and the camera parameters, coordinate transformation is performed to obtain the planar position point of the guide light in the vehicle coordinate system; The planar position points are compensated to obtain compensated position points; curve fitting is performed based on the compensated position points to obtain the path curve; Based on the path curve, image rendering is performed in the virtual image coordinate system to obtain a visual guide line for assisted driving.

[0006] In one embodiment, determining the effective light spot corresponding to the guide light in the front view image includes: Based on the camera parameters and the current vehicle speed, a trapezoidal region of interest is determined in the forward view image; The effective light spot is determined in the trapezoidal region of interest based on the brightness threshold and the effective color temperature range.

[0007] In one embodiment, determining the trapezoidal region of interest in the forward-view image based on the camera parameters and the current vehicle speed includes: Based on the camera's internal parameters and camera pitch angle in the camera parameters, a candidate area for the road surface is determined in the front view image; The candidate road surface region is dynamically adjusted and distortion corrected based on the current vehicle speed and the front view image to obtain the trapezoidal region of interest.

[0008] It accurately eliminates non-road interference areas above the horizon, reducing the computational load for subsequent spot recognition and avoiding false spots caused by false detections in the sky area, thus significantly improving the system's real-time performance and detection accuracy.

[0009] In one embodiment, determining the effective light spot in the trapezoidal region of interest based on a brightness threshold and an effective color temperature range includes: The image region in the trapezoidal region of interest whose brightness value is greater than or equal to the brightness threshold is determined as the initial light spot; Determine the color temperature of the initial light spot, and identify the initial light spot whose color temperature exists within the effective color temperature range as the effective light spot.

[0010] By fusing multi-dimensional features, the accuracy of effective light spot recognition is improved. Even in rainy or foggy weather or under interference from oncoming high beams, the guide light features can still be stably locked, providing reliable input for subsequent coordinate transformation.

[0011] In one embodiment, the step of performing coordinate transformation based on the pixel position of the effective light spot in the front view image and camera parameters to obtain the planar position point of the guide light in the vehicle coordinate system includes: The difference height is determined based on the camera optical center height and guide light height in the camera parameters. The vertical pixel deviation and the horizontal pixel deviation are determined based on the position of the principal point in the image and the pixel position. Determine the ratio of the difference height to the vertical pixel deviation, and use the product of the ratio and the vertical focal length in the camera parameters as the initial longitudinal distance; The initial longitudinal distance is corrected based on the pitch angle and the difference height in the camera parameters to obtain the longitudinal distance; The product of the horizontal pixel deviation and the vertical distance is determined, and the ratio of the product to the horizontal focal length in the camera parameters is determined as the horizontal distance. The planar position point is determined based on the origin of the vehicle coordinate system, according to the longitudinal distance and the lateral distance. The origin of the vehicle coordinate system is the projection point of the camera's optical center onto the road surface.

[0012] By accurately mapping the image pixel coordinates to the actual road surface plane position points in the vehicle coordinate system, perspective distortion and installation errors are eliminated, and curvature errors in subsequent curve fitting are reduced.

[0013] In one embodiment, correcting the initial longitudinal distance based on the pitch angle and the difference height in the camera parameters to obtain the longitudinal distance includes: The product of the cosine of the pitch angle and the initial longitudinal distance is determined as the projected distance; The product of the sine of the pitch angle and the difference in height is determined as the compensation distance; The sum of the projected distance and the compensated distance is determined as the longitudinal distance.

[0014] By decomposing the data using trigonometric functions, the hypotenuse distance in the camera coordinate system is converted into the horizontal distance in the vehicle coordinate system. This eliminates the longitudinal distance deviation caused by pitch installation and ensures that the position of the guide light can still accurately correspond to the actual position on the road surface in slope scenarios such as uphill and downhill, avoiding curve fitting distortion caused by distance errors.

[0015] In one embodiment, compensating the planar position point to obtain a compensated position point includes: The grid boundary is determined based on the preset grid size and the forward longitudinal distance of the trapezoidal region of interest; Determine the set of planar position points located within each grid based on the grid boundary; For each set of points, a target grid corresponding to the set of points is determined. The average value of all lateral distances in the set of points is determined as the representative lateral distance of the target grid, and the longitudinal grid center distance of the target grid is determined as the representative longitudinal distance, thus obtaining the compensation position point.

[0016] By forcing uniform sampling, the weights of near and far light spots are balanced, which improves the sensitivity of the fitted curve to the features of distant curves and avoids the nearsightedness caused by dense near points.

[0017] In one embodiment, the step of rendering the image in a virtual coordinate system based on the path curve to obtain a visual guide line for assisted driving includes: The reciprocal of the real-time curvature of the path curve is determined as the radius of curvature; Lateral acceleration is determined based on the current vehicle speed and the radius of curvature; The path curve is transformed to the virtual image coordinate system to obtain the virtual image path curve, and the target segment curve corresponding to the real-time curvature is determined in the virtual image path curve; In response to the lateral acceleration being greater than or equal to a preset safety acceleration, the target segment curve is rendered as a preset first color in the virtual image coordinate system; In response to the lateral acceleration being less than a preset safe acceleration, the target segment curve is rendered as a preset second color in the virtual image coordinate system; The warning level of the first color is higher than that of the second color.

[0018] By transforming the abstract curvature of curves into a risk level that drivers can directly perceive, the accident rate on curves at night is reduced.

[0019] In one embodiment, determining the road surface candidate region in the forward view image based on the camera's internal parameters and camera pitch angle from the camera parameters includes: The absolute value of the product of the vertical focal length and the tangent of the pitch angle in the internal parameters of the camera is determined as the position offset; The difference between the vertical distance of the principal point in the image and the position offset is determined as the horizon coordinate; The area in the front view image located below the horizon coordinate is determined as the road surface candidate area.

[0020] Through rigorous geometric derivation, the positioning error of the horizon position is reduced, ensuring that the candidate area of ​​the road surface completely covers the effective road surface and does not include interference such as sky and trees.

[0021] Based on the same inventive concept, this application also provides a vehicle including an electronic device, the electronic device including a memory, a processor and a computer program stored in the memory and running on the processor, wherein the processor executes the program to implement the method described above.

[0022] As can be seen from the above, the assisted driving guidance control method, electronic equipment, and vehicle provided in this application can determine the effective light spot corresponding to the guide light in the forward view image; perform coordinate transformation based on the pixel position of the effective light spot in the forward view image and camera parameters to obtain the planar position point of the guide light in the vehicle coordinate system; compensate the planar position point to obtain the compensated position point; perform curve fitting based on the compensated position point to obtain the path curve; and perform image rendering in the virtual image coordinate system based on the path curve to obtain the visual guide line for assisted driving. After determining the effective light spot corresponding to the guide light in the forward view image, the discrete effective light spot is restored to a precise compensated position point in the vehicle coordinate system, solving the problem of curvature calculation distortion caused by near-large and far-small in camera vision; perform curve fitting based on the compensated position point, and render the visual guide line in the virtual image coordinate system based on the path curve. Even in extreme scenarios without GPS, high-precision maps, or missing lane lines, it can still provide continuous, accurate, and risk-warning curve driving guide lines, significantly improving driving safety and comfort at night and in low-visibility environments. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in this application or related technologies, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a flowchart of the assisted driving guidance and control method according to an embodiment of this application; Figure 2 This is a schematic diagram of the guidance and control device for assisted driving according to an embodiment of this application; Figure 3 This is a schematic diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with specific embodiments and the accompanying drawings.

[0026] It should be noted that, unless otherwise defined, the technical or scientific terms used in the embodiments of this application should have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms "first," "second," and similar terms used in the embodiments of this application do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed after the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are only used to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0027] In this article, it is important to understand that any number of elements in the accompanying figures is for illustrative purposes and not for limitation, and any naming is for distinction only and has no limiting meaning.

[0028] Based on the above background description, the following situations also exist in the related technologies: I. The Perceptual Discontinuity Problem of Discrete Guide Lights. Highway roadside guide lights are discretely arranged at fixed intervals (12-20 meters), and there is a lack of continuous visual correlation between the light beads. Drivers cannot piece together the discontinuous light spots into a complete curve trajectory with the naked eye while driving at high speeds. Rainy or foggy weather and interference from oncoming high beams further weaken the visibility of the light spots, resulting in a cognitive break of "visible lights but not visible road", which cannot support effective prediction of the curve direction.

[0029] II. The problem of positioning failure in scenarios without navigation. In related technologies, head-up displays (HUDs) rely heavily on navigation signals and high-precision map data. When signals are lost in tunnels, map updates are delayed (such as when new roads are built / rerouted), or the driver does not turn on navigation, the map-based curvature path display is completely interrupted, and no curve visualization assistance can be provided, resulting in the loss of key references for driving decisions.

[0030] III. Perception robustness issues of lane line detection failure. In nighttime / low-light environments, lane lines suffer from significantly reduced contrast or even complete disappearance due to insufficient light, worn lane markings, road surface obstruction, and other factors, causing curvature estimation systems that rely on lane lines to malfunction; adaptive cruise control, lane keeping, and other functions are limited or deactivated due to lack of input, and the vehicle loses its ability to autonomously perceive the curvature of curves.

[0031] IV. The lack of dynamic fitting capabilities in Augmented Reality Head-Up Display (AR-HUD) systems. Related technologies in AR-HUD can only statically overlay pre-stored navigation lines, failing to capture the dynamic characteristics of discrete roadside guide lights in real time. This lack of high-order curve fitting and dynamic rendering capabilities leads to a disconnect between the displayed content and the actual curvature of curves. Furthermore, it cannot adjust the curvature and warning level of the guide lines according to changes in road conditions, resulting in a "floating" effect and misleading information.

[0032] V. Coordinate Distortion Problem of Perspective Projection. When the camera captures the road surface from a top-down perspective, due to the perspective effect of "near objects appear larger and far objects appear smaller," LEDs with the same physical spacing exhibit non-linear compression in the image. Directly fitting the image plane will result in a curve shape that is seriously inconsistent with the actual curvature of the curve. The longitudinal / transverse slope of the road surface further introduces projection errors, exacerbating the coordinate deviation and causing the fitting results to lose physical meaning.

[0033] VI. Environmental adaptability and computing power balance issues. The stability of the vision solution drops sharply under extreme conditions such as rain, fog, and backlight, making it impossible to continuously and reliably detect features; the computing power consumption of full-image spot detection and complex fitting algorithms is too high, making it difficult to run in real time on automotive embedded platforms, resulting in a trade-off between technical feasibility and user experience.

[0034] 7. Lack of driver-oriented interaction. Although related technologies attempt to supplement lane line recognition with the curves of roadside lights, they only serve vehicle attitude adjustment (such as lane keeping) and do not provide drivers with graphical curve assistance displays. They cannot achieve a closed loop of "perception-decision-interaction", and driving safety still depends on the driver's subjective judgment and experience.

[0035] The assisted driving guidance control method, electronic equipment, and vehicle provided in this application can determine the effective light spot corresponding to the guide light in the forward view image; perform coordinate transformation based on the pixel position of the effective light spot in the forward view image and camera parameters to obtain the planar position point of the guide light in the vehicle coordinate system; compensate the planar position point to obtain the compensated position point; perform curve fitting based on the compensated position point to obtain the path curve; and perform image rendering in the virtual image coordinate system based on the path curve to obtain the visual guide line for assisted driving. After determining the effective light spot corresponding to the guide light in the forward view image, the discrete effective light spot is restored to a precise compensated position point in the vehicle coordinate system, solving the problem of curvature calculation distortion caused by near-to-far vision; perform curve fitting based on the compensated position point, and render the visual guide line in the virtual image coordinate system based on the path curve. Even in extreme scenarios without GPS, high-precision maps, or missing lane lines, it can still provide continuous, accurate, and risk-warning curve driving guide lines, significantly improving driving safety and comfort at night and in low-visibility environments.

[0036] The following describes in detail, with reference to the accompanying drawings, the assisted driving guidance and control method provided by the embodiments of this application.

[0037] In some embodiments, such as Figure 1 As shown, the guidance and control method for assisted driving includes steps 101-104.

[0038] Step 101: Identify the effective light spot corresponding to the guide light in the front view image.

[0039] In some embodiments, determining the effective light spot corresponding to the guide light in the front view image includes: Determine the trapezoidal region of interest in the forward view image based on camera parameters and current vehicle speed; The effective light spot is determined in the trapezoidal region of interest based on the brightness threshold and the effective color temperature range.

[0040] In practice, the vehicle-mounted forward-facing camera (monocular or multi-lens) is fixedly installed inside the windshield or at the front grille to capture forward-facing images. Camera parameters include internal parameters (focal length, principal point, distortion coefficients, etc.) and external parameters (mounting height, pitch angle, roll angle, yaw angle), which are pre-stored through offline calibration. Based on horizon constraints, the formula vhorizon=cy is used. The function `fy·tan(θpitch)` calculates the ordinate of the horizon in the front view image, where `vhorizon` is the ordinate of the horizon, representing the row index of the horizon in the front view image, `cy` is the ordinate of the principal point of the image, `fy` is the horizontal focal length, and `θpitch` is the camera pitch angle. Areas above the horizon in the front view image (sky, trees, etc.) are directly discarded, retaining only the road surface candidate area below the horizon.

[0041] Then, the longitudinal range of the trapezoidal region of interest (ROI) is dynamically adjusted based on the current vehicle speed. As the vehicle speed changes, the near end (the bottom of the trapezoidal ROI) remains unchanged, while the far end (the top of the trapezoidal ROI near the horizon) dynamically adjusts its longitudinal distance range according to the current vehicle speed and reaction time. For example, at a vehicle speed of 120 km / h, the trapezoidal ROI covers a range of 80 meters ahead; at a vehicle speed of 60 km / h, the trapezoidal ROI covers a range of 40 meters ahead, ensuring that both near and far guide lights are included in the trapezoidal ROI.

[0042] Finally, considering the change in driving direction in a curved scenario, the left and right boundaries of the trapezoidal region of interest (ROI) adaptively narrow or widen according to the vehicle's maximum yaw rate, initially corresponding to the actual road width of 30-50 meters ahead (e.g., lane width of 3.75 meters multiplied by an expansion factor of 1.2). To avoid wide-angle distortion, a distortion correction model is applied to the original front view image, correcting straight lines in the actual road to straight lines in the image, providing an accurate basis for subsequent spot localization. Furthermore, when the cluster of guide light dots detected in multiple consecutive frames converges on one side of the trapezoidal ROI, it is predicted that the vehicle is entering a curve. In the next frame, the trapezoidal ROI shifts to that side (horizontal translation), with the shift amount proportional to the initially estimated curvature, preventing the LEDs from moving out of the trapezoidal ROI range.

[0043] After determining the trapezoidal region of interest (ROI), high-speed spot extraction is performed on the image within the ROI. First, a brightness threshold is dynamically set based on the headlight beam intensity, ambient light attenuation coefficient, and weather conditions (e.g., rain, fog). Areas within the ROI with brightness values ​​greater than or equal to the brightness threshold are marked as initial spots. Since guide lights typically use yellow (color temperature approximately 2700K~3000K) or bluish-white (color temperature approximately 6000K~6500K) spectra, while ordinary streetlights or oncoming vehicle lights are mostly warm white (color temperature 4000K~5000K), the RGB values ​​of the initial spots are extracted and their color temperature is calculated using a color temperature sensor or image color channel analysis. Spots with color temperatures falling within the effective color temperature range (yellow or bluish-white spectrum) are identified as effective spots, eliminating interference from oncoming vehicle lights and road reflections. To improve robustness, effective spots that are stably tracked for more than three consecutive frames are fused using multi-frame fusion, and the average of their centroid coordinates is taken as the center of the effective spot, reducing the impact of single-frame noise.

[0044] Step 102: Perform coordinate transformation based on the pixel position of the effective light spot in the front view image and the camera parameters to obtain the planar position point of the guide light in the vehicle coordinate system.

[0045] In practice, the difference height ΔH = Hlight is first calculated based on the camera optical center height Hcam and the guide light height Hlight (known to be 0.8m ± 0.1m) in the camera parameters. Hcam. Then, using the image principal point (cx, cy) as a reference (the image principal point is the intersection of the camera's optical axis and the imaging plane, the reference origin connecting the "physical world" and the "digital image," and also the zero-point reference for all coordinate transformations), the vertical pixel deviation Δv = v is calculated based on the pixel position (u, v) of the effective light spot. cy and lateral pixel deviation Δu=u cx. Using the pinhole imaging principle, calculate the initial longitudinal distance (along the vehicle's direction of travel):

[0046] in, This is the initial longitudinal distance. Vertical focal length (in pixels). y is the ordinate of the pixel position. The vertical coordinate of the principal point in the image.

[0047] Because the camera has a pitch angle θpitch, the initial longitudinal distance needs to be corrected: the product of the cosine of the pitch angle cos(θpitch) and Xbase is used as the projected distance, and the product of the sine of the pitch angle sin(θpitch) and the difference height ΔH is used as the compensation distance. The sum of the two is then determined as the final longitudinal distance. X = Xbase·cos(θpitch) + (Hcam) Hlight)·sin(θpitch) Where X is the corrected true longitudinal distance, representing the physical distance between the guide light and the front of the vehicle in the vehicle coordinate system.

[0048] Optionally, to improve the robustness of distance estimation, multi-frame motion constraints are introduced. Vehicle displacement obtained from the vehicle's own wheel speed sensors is used, combined with Kalman filtering, to smooth the measured distance of the same guide light bulb in consecutive frames. Simultaneously, when the same guide light bulb is stably tracked for more than three consecutive frames, the instantaneous pitch angle of the camera can be optimized in reverse to adapt to changes in road slope.

[0049] Then, based on the vertical distance, calculate the horizontal distance (positive on the left, negative on the right):

[0050] Where Y is the actual lateral distance, representing the lateral distance by which the guide light deviates from the vehicle's longitudinal axis in the vehicle coordinate system, and fx is the horizontal focal length (in pixels). is the x-coordinate of the pixel position. is the x-coordinate of the principal point of the image.

[0051] Finally, using the origin of the vehicle coordinate system (e.g., the ground point directly below the camera) as a reference, and combining the longitudinal distance X and the lateral distance Y, the planar position point (Y,X) (lateral and longitudinal) of the guide light in the road surface plane is obtained. The calculation process of the planar position point coordinates eliminates the influence of perspective distortion and installation errors.

[0052] Step 103: Compensate the planar position points to obtain compensated position points; perform curve fitting based on the compensated position points to obtain the path curve.

[0053] In practice, the first step is to compensate for projection compression errors in the planar position points. The forward longitudinal distance D (e.g., 40-90 meters, positively correlated with the current vehicle speed) corresponding to the trapezoidal region of interest is divided into multiple grids of a preset grid size (e.g., 1 meter). Each grid corresponds to a longitudinal distance interval of [(k-1)·Δd, k·Δd) (k=1,...,N, Δd=1 meter, N=D / Δd). The set of planar position points within each grid is statistically analyzed, and the average lateral distance of all points in the set is calculated as the representative lateral distance of that grid. The longitudinal center distance of the grid (e.g., k·Δd-0.5Δd meters) is used as the representative longitudinal distance, thus obtaining equidistantly distributed compensated position points and eliminating fitting bias caused by uneven light spot density at different distances. Since the projection compression error compensation mainly involves lateral errors, the longitudinal distribution of points only needs to be reasonable; the accuracy of the lateral position is crucial. Therefore, the average longitudinal distance of all points in the set can be used as the representative longitudinal distance of that grid.

[0054] Then, a least-squares fit is performed on the compensation location point (Yi,Xi) using a polynomial of at least order three. The expression for the path curve is then Y = a0 + a1·X + a2·X 2 +a3·X 3 , where a0, a1, a2, and a3 are fitting coefficients.

[0055] Step 104: Render the image in the virtual coordinate system according to the path curve to obtain the visual guide line for assisted driving.

[0056] In practice, the derivative of the path curve is calculated, and the real-time curvature ρ = y′′ / [(1+(y′)] is determined. 2 ) 3 / 2The real-time curvature values ​​of adjacent frames are smoothed using Kalman filtering to avoid jitter. During the rendering stage, the path curve is transformed to the virtual image coordinate system to obtain the virtual image path curve. The target segment curve corresponding to the real-time curvature is then determined within the virtual image path curve. Finally, the radius of curvature R=1 / ρ is combined with the current vehicle speed V to calculate the lateral acceleration a=V. 2 / R. If a ≥ 0.3g (g is the acceleration due to gravity), the target segment curve is rendered in the first color (e.g., red, high-risk warning); if a < 0.3g, the target segment curve is rendered in the second color (e.g., green, safety warning). The visual guide line is superimposed on the driver's field of vision as a dynamic light strip, the width of which increases with vehicle speed to enhance immersion. In addition, if the number of effective light spots is insufficient (e.g., less than 6), the steering wheel angle or turn signal is introduced as an auxiliary interpolation source to ensure the continuity of the path curve.

[0057] The assisted driving guidance control method provided in this application can dynamically delineate an adaptive trapezoidal region of interest by fusing camera parameters and real-time vehicle speed, effectively eliminating interference from the sky, buildings, and non-road areas, and significantly reducing the computational load. It can accurately lock the guide lights in the forward view image by using brightness thresholds and effective color temperature ranges, overcoming the perception failure problem caused by lane line wear, backlight, and oncoming glare at night. Through coordinate transformation and compensation algorithms, it restores discrete effective light spots to accurate compensation position points in the real road plane, solving the problem of curvature calculation distortion caused by near-to-far vision. Finally, it performs curve fitting based on the compensation position points and dynamically renders visual guide lines by combining vehicle speed and real-time curvature. Even in extreme scenarios without GPS, high-precision maps, and missing lane lines, it can still provide continuous, accurate, and risk-warning curve driving guide lines, greatly improving driving safety and comfort at night and in low-visibility environments.

[0058] In some embodiments, determining a trapezoidal region of interest in the forward-view image based on camera parameters and the current vehicle speed includes: Based on the camera's internal parameters and camera pitch angle in the camera parameters, determine the road surface candidate area in the front view image; Based on the current vehicle speed and the front view image, the candidate area of ​​the road surface is dynamically adjusted and distortion corrected to obtain a trapezoidal region of interest.

[0059] In practice, the delineation of the Region of Interest (ROI) often fails to incorporate horizon constraints and frequently includes non-road areas such as the sky and trees in the detection range. This results in wasted computing power for subsequent spot recognition and makes the system susceptible to interference. When determining candidate road areas in the front view image based on the camera's internal parameters and pitch angle, the vertical focal length fy is first multiplied by the tangent of the pitch angle θpitch, tan(θpitch), and the absolute value is taken to obtain the position offset. Since the image coordinate system increases downwards with the top as the origin, the horizon position vhorizon is the principal point's ordinate cy minus this position offset, i.e., the horizon ordinate vhorizon = cy in the front view image. fy·tan(θpitch). This method identifies the area below the horizon in the front view image as a candidate road surface region, effectively eliminating non-road interference such as sky and trees, thus reducing subsequent computation. This calculation method accurately eliminates non-road interference areas above the horizon, reducing the computational load for subsequent spot recognition by more than 60%. It also avoids false spots caused by false detections in the sky area, significantly improving the system's real-time performance and detection accuracy, laying a reliable foundation for subsequent dynamic trapezoidal ROI adjustments.

[0060] In some embodiments, determining the effective light spot corresponding to the guide light in the trapezoidal region of interest based on a brightness threshold and an effective color temperature range includes: The image region with a brightness value greater than or equal to the brightness threshold in the trapezoidal region of interest is determined as the initial light spot; Determine the color temperature of the initial light spot, and identify the initial light spot whose color temperature falls within the effective color temperature range as the effective light spot.

[0061] In practice, to address the technical problem that spot recognition is easily interfered with by stray light such as oncoming vehicle lights, streetlights, and road surface reflections, resulting in poor robustness of guide light feature extraction, a two-stage filtering calculation method of "initial screening with brightness threshold + fine screening with effective color temperature range" is adopted to avoid interference issues.

[0062] First, the image is converted to grayscale within the trapezoidal region of interest. Connected components within the trapezoidal region of interest whose brightness values ​​are greater than or equal to a dynamically adjusted brightness threshold are marked as initial light spots. The brightness threshold is dynamically adjusted according to the ambient light intensity: 200 (8-bit grayscale value) when the ambient light is below 10 lux at night, and 150 when the ambient light is between 10 and 50 lux at dusk.

[0063] Then, the average RGB channel value of the initial light spot is extracted, and the color temperature T of the light spot is estimated by the formula T=(R+B) / G (R,G,B are the red, green and blue channel values). If the color temperature T of the light spot falls in the yellow color temperature range (1.8~2.2) or the blue-white color temperature range (0.8~1.2), the initial light spot is determined to be a valid light spot, and the interference from oncoming headlights in warm white (1.2~1.8) is eliminated.

[0064] By fusing multi-dimensional features, the accuracy of effective light spot recognition is improved. Even in rainy or foggy weather or under interference from oncoming high beams, the guide light features can still be stably locked, providing reliable input for subsequent coordinate transformation.

[0065] In some embodiments, coordinate transformation is performed based on the pixel position of the effective light spot in the forward view image and camera parameters to obtain the planar position point of the guide light in the vehicle coordinate system, including: Determine the difference height based on the camera optical center height and guide light height in the camera parameters; Determine the vertical and horizontal pixel deviations based on the principal point and pixel positions of the image; Determine the ratio of the difference height to the vertical pixel deviation, and multiply the ratio by the vertical focal length in the camera parameters to determine the initial longitudinal distance; The initial longitudinal distance is corrected based on the pitch angle and differential height in the camera parameters to obtain the longitudinal distance; Determine the product of the horizontal pixel deviation and the vertical distance, and use the ratio of this product to the horizontal focal length in the camera parameters as the horizontal distance. Based on the origin of the vehicle coordinate system, the planar position point is determined according to the longitudinal and lateral distances. The origin of the vehicle coordinate system is the projection point of the camera's optical center onto the road surface.

[0066] In practice, to address the technical problem that directly using image pixel coordinates for curve fitting results in coordinate distortion due to perspective effects ("near objects appear larger than distant objects") and fails to reflect the true geometry of the curve, a three-dimensional coordinate transformation of "difference height calculation + pixel deviation extraction + pitch angle correction" is adopted to calculate the planar position point.

[0067] During the coordinate transformation process, the difference in height ΔH = Hlight is first calculated based on the height of the camera's optical center Hcam and the known height of the guide light Hlight (0.8m ± 0.1m). Hcam. Difference height ΔH = Hlight Hcam directly reflects the vertical relative position of the guide light and the camera (ΔH>0 when the light is higher than the camera, and ΔH<0 when it is lower).

[0068] Then, using the principal point of the image as a reference, the vertical pixel deviation Δv=v of the effective light spot is extracted based on the pixel position. cy and lateral pixel deviation Δu=u cx, the vertical pixel deviation, represents the vertical distance of the light spot from the image center. Its ratio to ΔH, Δv / ΔH, reflects the proportion of physical distance per unit pixel. The initial vertical distance is calculated using the pinhole imaging principle: ΔH / Δv = Xbase / fy. :

[0069] Because the camera has a pitch angle θpitch, the initial longitudinal distance needs to be corrected. During pitch angle correction, the product of the cosine of the pitch angle cos(θpitch) and Xbase is used as the projected distance Xbase·cos(θpitch), and the product of the sine of the pitch angle sin(θpitch) and the difference height ΔH is used as the compensation distance (Hcam). Hlight·sin(θpitch), where the projected distance represents the distance of the hypotenuse projected onto the horizontal direction of the road surface, and the compensation distance is used to correct the longitudinal offset caused by the height difference. The sum of the two is the final longitudinal distance X = Xbase·cos(θpitch) + (Hcam)cos(θpitch)cos(θpitch)cos(θcam ... Hlight)·sin(θpitch) ensures the physical accuracy of the longitudinal distance.

[0070] Then, based on the vertical distance, calculate the horizontal distance (positive on the left, negative on the right):

[0071] Finally, using the origin of the vehicle coordinate system (e.g., the projection of the camera's optical center onto the road surface) as a reference, and combining the longitudinal distance X and the lateral distance Y, the planar position point (Y, X) (lateral and longitudinal) of the guide light on the road surface is obtained. This planar position point accurately corresponds to the actual position of the guide light on the road surface. The calculation process of the planar position point coordinates eliminates the influence of perspective distortion and installation errors, ensuring that the curvature error of the subsequent curve fitting is controlled within 0.001m. - Within ¹, it fully meets the requirements for perception accuracy.

[0072] In some embodiments, the initial longitudinal distance is corrected based on the pitch angle and differential height in the camera parameters to obtain the longitudinal distance, including: The product of the cosine of the pitch angle and the initial longitudinal distance is determined as the projected distance; The product of the sine of the pitch angle and the difference in altitude is determined as the compensation distance; The sum of the projected distance and the compensated distance is determined as the longitudinal distance.

[0073] In practical implementation, to address the technical problem of hypotenuse error in the initial longitudinal distance measurement caused by the camera's pitch installation, which fails to reflect the true horizontal distance of the road surface, a dual correction strategy of "cosine projection + sine compensation" is adopted for distance correction. The core of pitch angle correction is to convert the hypotenuse distance in the camera coordinate system to the horizontal distance in the vehicle coordinate system. The projected distance Xbase·cos(θpitch) uses trigonometric functions to project the initial longitudinal distance (hypotenuse) onto the road surface plane, and the compensated distance (Hcam)... Hlight·sin(θpitch) corrects for the longitudinal error caused by the difference in height between the camera and the guide light: when the camera is higher than the guide light, Hcam If Hlight > 0, the compensation distance is positive, increasing the longitudinal distance; conversely, if it is negative, the longitudinal distance is decreased. The sum of the projected distance and the compensation distance is then determined as the longitudinal distance X, which is the corrected true longitudinal distance. Through trigonometric decomposition, the hypotenuse distance in the camera coordinate system is converted to the horizontal distance in the vehicle coordinate system, eliminating longitudinal distance deviations caused by pitch installation (the maximum deviation can be reduced from 1.2 meters to within 0.05 meters). This ensures that the guide light's position accurately corresponds to the actual road surface location in slope scenarios such as uphill and downhill sections, avoiding curve fitting distortion caused by distance errors.

[0074] In some embodiments, compensating for planar position points to obtain compensated position points includes: The grid boundary is determined based on the preset grid size and the forward longitudinal distance of the trapezoidal region of interest; Determine the set of planar location points within each grid based on the grid boundary; For each set of points, the target grid corresponding to the set of points is determined. The average value of all lateral distances in the set of points is determined as the representative lateral distance of the target grid, and the longitudinal center distance of the target grid is determined as the representative longitudinal distance, thus obtaining the compensation position point.

[0075] In practice, to address the technical problem that directly fitting the original planar position points results in denser light spots near the edges and sparser ones in the distance due to perspective effects, and the fitting result is dominated by the near points while ignoring the curvature trend in the distance, a resampling compensation strategy of "equidistant grid division + grid averaging" is adopted to achieve compensation.

[0076] The longitudinal distance D corresponding to the trapezoidal region of interest (e.g., 40-80 meters, positively correlated with the current vehicle speed) is divided into multiple grids according to a preset grid size (e.g., 1 meter). Each grid corresponds to a longitudinal distance interval of [(k-1)·Δd, k·Δd) (k=1,...,N, Δd=1 meter, N=D / Δd). The set of planar position points within each grid is statistically analyzed, and the average lateral distance of all points in the set is calculated as the representative lateral distance of that grid. The longitudinal center distance of the grid (e.g., k·Δd+0.5Δd meters) is used as the representative longitudinal distance, thus obtaining equidistantly distributed compensation position points and eliminating fitting bias caused by uneven light spot density at different distances. Since the projection compression error compensation mainly involves lateral errors, the longitudinal distribution of points only needs to be reasonable; the accuracy of the lateral position is crucial. Therefore, the average longitudinal distance of all points in the set can be used as the representative longitudinal distance of that grid. By forcing uniform sampling, the non-uniformly distributed original points are transformed into uniformly distributed compensation position points, avoiding the dominance of densely packed nearby points in the fitting results and improving the stability of curve fitting. By balancing the weights of near and far light spots, the sensitivity of the fitted curve to distant curve features is increased by more than 40%, avoiding the "nearsightedness" phenomenon caused by dense near points. This ensures that the fitted path curve can fully reflect the curve direction from near to far, providing an accurate geometric basis for rendering.

[0077] For example, with a preset grid size of 3 meters and a forward longitudinal distance of 90 meters, the grid is divided into 30 grids, and there are planar position points P1(20,0.5), P2(20,0.52), P3(20,0.48), P4(40,1.2), and P5(60,2.5). If curve fitting is performed directly without any processing, the three nearest planar position points (at 20 meters) have a 3-fold influence on the curve. The one distant point (at 60 meters) has only a 1-fold influence. The algorithm may mistakenly assume that the road at 20 meters has severe lateral fluctuations (jagged edges), or forcibly straighten the curve to accommodate the nearby point group, causing the true curvature trend at a distance (Y=2.5 at 60 meters) to be ignored.

[0078] It can be seen that the planar position points P1(20,0.5), P2(20,0.52), and P3(20,0.48) exist within the grid range [18,21), corresponding to grid 7; the planar position point P4(40,1.2) exists within the grid range [39,42), corresponding to grid 14; and the planar position point P5(60,2.5) exists within the grid range [60,63), corresponding to grid 20. Since there is only one planar position point in both grid 14 and grid 20, the compensation position point for grid 14 is P4(40,1.2), and the compensation position point for grid 20 is P5(60,2.5). Optionally, the longitudinal grid center distance of the target grid can also be used as the representative longitudinal distance, so the compensation position point for grid 14 can also be P4(40.5,1.2), and the compensation position point for grid 20 can also be P5(61.5,2.5). Since there are 3 planar location points within grid 7, the average value of all lateral distances is 0.50, and the average value of all longitudinal distances is 20. Therefore, the compensation location point is (20, 0.5). Alternatively, the longitudinal center distance of the target grid (grid 7), which is 19.5, can be determined as the representative longitudinal distance, and the compensation location point can also be (19.5, 0.5).

[0079] In some embodiments, image rendering is performed in a virtual coordinate system based on the path curve to obtain a visual guide line for assisted driving, including: The reciprocal of the real-time curvature of the path curve is determined as the radius of curvature; Lateral acceleration is determined based on the current vehicle speed and radius of curvature; The path curve is transformed to the virtual image coordinate system to obtain the virtual image path curve, and the target segment curve corresponding to the real-time curvature is determined in the virtual image path curve. In response to a lateral acceleration greater than or equal to a preset safety acceleration, the target segment curve is rendered as a preset first color in the virtual image coordinate system; In response to a lateral acceleration less than a preset safe acceleration, the target segment curve is rendered as a preset second color in the virtual image coordinate system; The warning level for the first color is higher than that for the second color.

[0080] In practical implementation, to address the technical issues of AR-HUD rendering only statically overlaying navigation indicator lines, lacking risk level warnings, and being unable to dynamically adjust according to road conditions, a dynamic rendering method of "curvature radius calculation + lateral acceleration grading" is adopted for dynamic grading rendering. First, the reciprocal of the real-time curvature is determined as the curvature radius R = 1 / ρ, intuitively reflecting the sharpness of the curve. Then, combined with the current vehicle speed V, the lateral acceleration a = V is calculated. 2 / R quantifies the safety risks of vehicles cornering. Taking a preset safe acceleration threshold of 0.3g (approximately 2.94 m / s²) as an example; when a ≥ 0.3g, the target segment curve is rendered in the first color (e.g., red, high-risk warning) with a flashing effect; when a < 0.3g, the target segment curve is rendered in a second color different from the first color (e.g., green, indicating safety guidance). Simultaneously, the width of the light strip is dynamically adjusted according to vehicle speed (e.g., the width increases by 10% for every 20 km / h increase in speed). This transforms the abstract curvature of the curve into a risk level that the driver can directly perceive, reducing the accident rate on curves at night. At the same time, the dynamic light strip enhances immersion, solving the "floating" and misleading problems of traditional rendering, achieving a closed-loop safety assistance system of "perception-warning-interaction".

[0081] In some embodiments, determining candidate road surface regions in the forward view image based on camera intrinsic parameters and camera pitch angle from the camera parameters includes: The absolute value of the product of the vertical focal length and the tangent of the pitch angle in the camera's internal parameters is determined as the position offset; The difference between the vertical distance and the position offset of the principal point in the image is used as the vertical coordinate of the horizon. The area below the horizon in the front view image is identified as a candidate road surface area.

[0082] In practice, to address the technical problem that the horizon position calculation did not incorporate the camera's internal pitch angle, resulting in the trapezoidal region of interest containing a large number of non-road areas, low detection efficiency, and susceptibility to interference, a position offset of "vertical focal length × pitch angle tangent" is adopted to avoid interference.

[0083] When determining candidate road surface regions, the position offset is |fy·tan(θpitch)|, ensuring that the horizon position calculation is unaffected by the sign of the pitch angle. The ordinate cy of the principal point in the image is the vertical center of the image. Subtracting the position offset yields the ordinate vhorizon of the horizon. The area below the horizon is considered the candidate road surface region. For example, if cy=540, fy=1200, and θpitch=2°, then vhorizon=540. 1200×0.0349≈498, meaning the area below the 498th row of pixels in the image is the road surface region, effectively narrowing the detection range. Through rigorous geometric derivation, the positioning error of the horizon position is controlled within 2 pixels, ensuring that the road surface candidate area completely covers the effective road surface and does not include interference such as sky or trees. This provides a precise reference range for the subsequent dynamic adjustment of the trapezoidal ROI, further improving the system's adaptability to complex road conditions.

[0084] It should be noted that the method in this embodiment can be executed by a single device, such as a computer or server. The method can also be applied in a distributed scenario, where multiple devices cooperate to complete the task. In such a distributed scenario, one of these devices may execute only one or more steps of the method in this embodiment, and the multiple devices will interact with each other to complete the method described.

[0085] It should be noted that the above description describes some embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in a different order than that shown in the above embodiments and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0086] Based on the same inventive concept, corresponding to any of the above embodiments, this application also provides a guidance and control device for assisted driving.

[0087] refer to Figure 2 The assisted driving guidance and control device includes: The light spot positioning module 10 is configured to: determine the effective light spot corresponding to the guide light in the front view image; The coordinate transformation module 20 is configured to perform coordinate transformation based on the pixel position of the effective light spot in the front view image and the camera parameters to obtain the planar position point of the guide light in the vehicle coordinate system. The curve fitting module 30 is configured to: compensate for the planar position points to obtain compensated position points; and perform curve fitting based on the compensated position points to obtain the path curve. The dynamic rendering module 40 is configured to render an image in a virtual coordinate system based on the path curve to obtain a visual guide line for assisted driving.

[0088] In one embodiment, the spot positioning module 10 is further configured to: Determine the trapezoidal region of interest in the forward view image based on camera parameters and current vehicle speed; The effective light spot is determined in the trapezoidal region of interest based on the brightness threshold and the effective color temperature range.

[0089] In one embodiment, the spot positioning module 10 is further configured to: Based on the camera's internal parameters and camera pitch angle in the camera parameters, determine the road surface candidate area in the front view image; Based on the current vehicle speed and the front view image, the candidate area of ​​the road surface is dynamically adjusted and distortion corrected to obtain a trapezoidal region of interest.

[0090] In one embodiment, the spot positioning module 10 is further configured to: The image region with a brightness value greater than or equal to the brightness threshold in the trapezoidal region of interest is determined as the initial light spot; Determine the color temperature of the initial light spot, and identify the initial light spot whose color temperature falls within the effective color temperature range as the effective light spot.

[0091] In one embodiment, the coordinate transformation module 20 is further configured to: Determine the difference height based on the camera optical center height and guide light height in the camera parameters; Determine the vertical and horizontal pixel deviations based on the principal point and pixel positions of the image; Determine the ratio of the difference height to the vertical pixel deviation, and multiply the ratio by the vertical focal length in the camera parameters to determine the initial longitudinal distance; The initial longitudinal distance is corrected based on the pitch angle and differential height in the camera parameters to obtain the longitudinal distance; Determine the product of the horizontal pixel deviation and the vertical distance, and use the ratio of this product to the horizontal focal length in the camera parameters as the horizontal distance. Based on the origin of the vehicle coordinate system, the planar position point is determined according to the longitudinal and lateral distances. The origin of the vehicle coordinate system is the projection point of the camera's optical center onto the road surface.

[0092] In one embodiment, the coordinate transformation module 20 is further configured to: The product of the cosine of the pitch angle and the initial longitudinal distance is determined as the projected distance; The product of the sine of the pitch angle and the difference in altitude is determined as the compensation distance; The sum of the projected distance and the compensated distance is determined as the longitudinal distance.

[0093] In one embodiment, the curve fitting module 30 is further configured to: The grid boundary is determined based on the preset grid size and the forward longitudinal distance of the trapezoidal region of interest; Determine the set of planar location points within each grid based on the grid boundary; For each set of points, the target grid corresponding to the set of points is determined. The average value of all lateral distances in the set of points is determined as the representative lateral distance of the target grid, and the longitudinal center distance of the target grid is determined as the representative longitudinal distance, thus obtaining the compensation position point.

[0094] In one embodiment, the dynamic rendering module 40 is further configured as follows: The reciprocal of the real-time curvature of the path curve is determined as the radius of curvature; Lateral acceleration is determined based on the current vehicle speed and radius of curvature; The path curve is transformed to the virtual image coordinate system to obtain the virtual image path curve, and the target segment curve corresponding to the real-time curvature is determined in the virtual image path curve. In response to a lateral acceleration greater than or equal to a preset safety acceleration, the target segment curve is rendered as a preset first color in the virtual image coordinate system; In response to a lateral acceleration less than a preset safe acceleration, the target segment curve is rendered as a preset second color in the virtual image coordinate system; The warning level for the first color is higher than that for the second color.

[0095] In one embodiment, the spot positioning module 10 is further configured to: The absolute value of the product of the vertical focal length and the tangent of the pitch angle in the camera's internal parameters is determined as the position offset; The difference between the vertical distance and the position offset of the principal point in the image is used as the vertical coordinate of the horizon. The area below the horizon in the front view image is identified as a candidate road surface area.

[0096] For ease of description, the above devices are described in terms of function, divided into various modules. Of course, in implementing this application, the functions of each module can be implemented in one or more software and / or hardware.

[0097] The apparatus of the above embodiments is used to implement the corresponding assisted driving guidance and control method in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0098] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the assisted driving guidance and control method described in any of the above embodiments.

[0099] Figure 3 This embodiment illustrates a more specific hardware structure of an electronic device. The device may include a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, memory 1020, input / output interface 1030, and communication interface 1040 are interconnected internally via the bus 1050.

[0100] The processor 1010 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0101] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010.

[0102] The input / output interface 1030 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components within the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touchscreens, microphones, various sensors, etc., while output devices may include displays, speakers, vibrators, indicator lights, etc.

[0103] The communication interface 1040 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0104] Bus 1050 includes a pathway for transmitting information between various components of the device, such as processor 1010, memory 1020, input / output interface 1030, and communication interface 1040.

[0105] It should be noted that although the above-described device only shows the processor 1010, memory 1020, input / output interface 1030, communication interface 1040, and bus 1050, in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the embodiments of this specification, and not necessarily all the components shown in the figures.

[0106] The electronic devices described above are used to implement the corresponding assisted driving guidance and control methods in any of the foregoing embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0107] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, this application also provides a non-transitory computer-readable storage medium storing computer instructions for causing the computer to execute the assisted driving guidance and control method as described in any of the above embodiments.

[0108] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0109] The computer instructions stored in the storage medium of the above embodiments are used to cause the computer to execute the assisted driving guidance and control method as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0110] Based on the same inventive concept, corresponding to any of the above embodiments, this application also provides a vehicle, including the electronic equipment or driver assistance guidance and control device of the above embodiments, and executes the driver assistance guidance and control method as described in any of the above embodiments through the electronic equipment or driver assistance guidance and control device of the above embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0111] It is understood that before using the technical solutions of the various embodiments in this application, users will be informed of the type, scope of use, and usage scenarios of the personal information involved in an appropriate manner, and user authorization will be obtained.

[0112] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose, based on the prompt message, whether to provide personal information to the software or hardware such as electronic devices, applications, servers, or storage media performing the operations described in this application.

[0113] As an optional but not limited implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0114] It is understood that the above notification and user authorization process is merely illustrative and does not limit the implementation of this application. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this application.

[0115] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of this application is limited to these examples; under the concept of this application, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of the embodiments of this application as described above, which are not provided in detail for the sake of brevity.

[0116] Additionally, to simplify the description and discussion, and to avoid obscuring the embodiments of this application, the well-known power / ground connections to integrated circuit (IC) chips and other components may or may not be shown in the provided drawings. Furthermore, the apparatus may be shown in block diagram form to avoid obscuring the embodiments of this application, and this also takes into account the fact that the details of the implementation of these block diagram apparatuses are highly dependent on the platform on which the embodiments of this application will be implemented (i.e., these details should be fully understood by those skilled in the art). While specific details (e.g., circuits) have been set forth to describe exemplary embodiments of this application, it will be apparent to those skilled in the art that the embodiments of this application can be implemented without these specific details or with variations thereof. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0117] Although this application has been described in conjunction with specific embodiments thereof, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may be used with the embodiments discussed.

[0118] The embodiments of this application are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the claims of this application. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the embodiments of this application should be included within the protection scope of this application.

Claims

1. A guidance and control method for assisted driving, characterized in that, include: Identify the effective light spot corresponding to the guide light in the front view image; Based on the pixel position of the effective light spot in the front view image and the camera parameters, coordinate transformation is performed to obtain the planar position point of the guide light in the vehicle coordinate system; The planar position points are compensated to obtain compensated position points; The path curve is obtained by curve fitting based on the compensation location points; Based on the path curve, image rendering is performed in the virtual image coordinate system to obtain a visual guide line for assisted driving.

2. The assisted driving guidance and control method according to claim 1, characterized in that, The step of determining the effective light spot corresponding to the guide light in the front view image includes: Based on the camera parameters and the current vehicle speed, a trapezoidal region of interest is determined in the forward view image; The effective light spot is determined in the trapezoidal region of interest based on the brightness threshold and the effective color temperature range.

3. The assisted driving guidance and control method according to claim 2, characterized in that, The step of determining the trapezoidal region of interest in the forward-view image based on the camera parameters and the current vehicle speed includes: Based on the camera's internal parameters and camera pitch angle in the camera parameters, a candidate area for the road surface is determined in the front view image; The candidate road surface region is dynamically adjusted and distortion corrected based on the current vehicle speed and the front view image to obtain the trapezoidal region of interest.

4. The assisted driving guidance and control method according to claim 2, characterized in that, The step of determining the effective light spot in the trapezoidal region of interest based on the brightness threshold and the effective color temperature range includes: The image region in the trapezoidal region of interest whose brightness value is greater than or equal to the brightness threshold is determined as the initial light spot; Determine the color temperature of the initial light spot, and identify the initial light spot whose color temperature exists within the effective color temperature range as the effective light spot.

5. The assisted driving guidance and control method according to claim 1, characterized in that, The step of performing coordinate transformation based on the pixel position of the effective light spot in the front view image and camera parameters to obtain the planar position point of the guide light in the vehicle coordinate system includes: The difference height is determined based on the camera optical center height and guide light height in the camera parameters. The vertical pixel deviation and the horizontal pixel deviation are determined based on the position of the principal point in the image and the pixel position. Determine the ratio of the difference height to the vertical pixel deviation, and use the product of the ratio and the vertical focal length in the camera parameters as the initial longitudinal distance; The initial longitudinal distance is corrected based on the pitch angle and the difference height in the camera parameters to obtain the longitudinal distance; The product of the horizontal pixel deviation and the vertical distance is determined, and the ratio of the product to the horizontal focal length in the camera parameters is determined as the horizontal distance. The planar position point is determined based on the origin of the vehicle coordinate system, according to the longitudinal distance and the lateral distance. The origin of the vehicle coordinate system is the projection point of the camera's optical center onto the road surface.

6. The assisted driving guidance and control method according to claim 5, characterized in that, The step of correcting the initial longitudinal distance based on the pitch angle and the difference height in the camera parameters to obtain the longitudinal distance includes: The product of the cosine of the pitch angle and the initial longitudinal distance is determined as the projected distance; The product of the sine of the pitch angle and the difference in height is determined as the compensation distance; The sum of the projected distance and the compensated distance is determined as the longitudinal distance.

7. The assisted driving guidance and control method according to claim 2, characterized in that, The step of compensating the planar position point to obtain the compensated position point includes: The grid boundary is determined based on the preset grid size and the forward longitudinal distance of the trapezoidal region of interest; Determine the set of planar position points located within each grid based on the grid boundary; For each set of points, a target grid corresponding to the set of points is determined. The average value of all lateral distances in the set of points is determined as the representative lateral distance of the target grid, and the longitudinal grid center distance of the target grid is determined as the representative longitudinal distance, thus obtaining the compensation position point.

8. The assisted driving guidance and control method according to claim 1, characterized in that, The step of rendering an image in a virtual coordinate system based on the path curve to obtain a visual guide line for assisted driving includes: The reciprocal of the real-time curvature of the path curve is determined as the radius of curvature; Lateral acceleration is determined based on the current vehicle speed and the radius of curvature; The path curve is transformed to the virtual image coordinate system to obtain the virtual image path curve, and the target segment curve corresponding to the real-time curvature is determined in the virtual image path curve; In response to the lateral acceleration being greater than or equal to a preset safety acceleration, the target segment curve is rendered as a preset first color in the virtual image coordinate system; In response to the lateral acceleration being less than a preset safe acceleration, the target segment curve is rendered as a preset second color in the virtual image coordinate system; The warning level of the first color is higher than that of the second color.

9. The assisted driving guidance and control method according to claim 3, characterized in that, The step of determining the road surface candidate region in the front view image based on the camera's internal parameters and camera pitch angle in the camera parameters includes: The absolute value of the product of the vertical focal length and the tangent of the pitch angle in the internal parameters of the camera is determined as the position offset; The difference between the vertical distance of the principal point in the image and the position offset is determined as the horizon coordinate; The area in the front view image located below the horizon coordinate is determined as the road surface candidate area.

10. A vehicle, including electronic equipment, characterized in that, The electronic device includes a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the program to implement the method as described in any one of claims 1 to 8.

Images