Vehicle travel path planning method and electronic device

Abstract

This application discloses a vehicle driving path planning method and electronic device, relating to the field of vehicle control technology. The method includes: acquiring image data of the vehicle driving environment from at least two angles; processing the image data using a preset image processing strategy to obtain the location information of pothole regions. The preset image processing strategy processes the image data in the following order: image closing operation, image opening operation, edge detection, color segmentation, and contour detection; calculating the depth information of the pothole regions based on the image data and the location information of the pothole regions; and determining adjustment parameters for the vehicle driving path based on the location information of the pothole regions, the current vehicle speed, and the depth information of the pothole regions, thereby planning the vehicle driving path based on the adjustment parameters. This application's method achieves low-cost, high-real-time, and high-precision intelligent driving path planning in complex road environments, significantly improving vehicle driving safety and passenger comfort.

Description

Technical Field

[0001] This application relates to the field of vehicle control technology, and in particular to a vehicle driving path planning method and electronic device. Background Technology

[0002] With the rapid development of artificial intelligence and 5G communication technologies, the adaptability of intelligent driving to complex road environments is increasing. However, road defects remain a core challenge to driving safety and comfort. Existing road defect detection and path planning solutions are mainly divided into two categories: one is a multi-sensor fusion solution, which uses a combination of LiDAR and cameras to achieve pothole depth detection through 3D point cloud modeling; the other is a pure vision deep learning solution, which uses models such as YOLO (You Only Look Once, a single-shot object detection algorithm) and PointNet++ (Point Network Plus Plus, an upgraded version of point neural networks) to achieve end-to-end pothole location detection.

[0003] However, while LiDAR detection offers high accuracy, it suffers from high hardware costs, complex calibration, and insufficient real-time performance. Deep learning-based solutions suffer from high inference latency and high data maintenance costs. In low-visibility scenarios, drivers struggle to accurately and promptly assess pothole conditions, increasing the risk of accidents, and path planning lacks targeted obstacle avoidance strategies. Summary of the Invention

[0004] This application provides a vehicle driving path planning method and electronic device to at least solve the problems of insufficient real-time control and lack of path planning in related technologies.

[0005] Firstly, this application provides a vehicle driving path planning method, including:

[0006] Acquire image data of the vehicle's driving environment from at least two angles;

[0007] The image data is processed using a preset image processing strategy to obtain the location information of the pit area. The preset image processing strategy processes the image data in the following order: image closing operation, image opening operation, edge detection, color segmentation, and contour detection.

[0008] The depth information of the pit area is calculated based on the image data and the location information of the pit area.

[0009] Based on the location information of the pothole area, the current vehicle speed, and the depth information of the pothole area, the adjustment parameters of the vehicle driving path are determined, and the vehicle driving path is planned based on the adjustment parameters.

[0010] Secondly, this application also provides an electronic device, comprising:

[0011] Memory, used to store computer programs;

[0012] A processor is used to implement the steps of any of the methods in the first aspect when executing a computer program.

[0013] This application provides a vehicle driving path planning method and electronic device. The method acquires vehicle driving environment image data from at least two angles, enabling observation of the vehicle's surroundings from multiple perspectives. Images from different angles can complement and corroborate each other, helping to more accurately understand the spatial structure and object distribution of the vehicle's surroundings and improving the overall cognitive ability of the driving environment. The combined use of graph closing and opening operations makes it easier to fuse pothole locations. After morphological operations and edge detection, HSV (Hue, Saturation, Value) color segmentation is performed. Morphological operations first ensure region connectivity, edge detection provides precise boundary lines, and color segmentation provides brightness / color priority constraints, offering significant advantages in suppressing texture misdetections such as cracks and road surface textures, and improving positioning accuracy. Edges are used for positioning, color is used to eliminate shadow / wet spot misidentifications, and contours provide area and shape constraints, thus significantly improving accuracy. Using the above method for pothole location detection avoids the use of deep learning, improving detection efficiency. The depth information of potholes is a key factor in assessing their impact on vehicle driving, and accurately calculating this depth is crucial for rationally planning vehicle routes. By combining image data with the location information of the pothole area to calculate depth, inferences can be made using visual cues and geometric relationships in the image, enabling accurate estimation of the actual pothole depth while reducing costs. By comprehensively considering the location of the pothole area, current vehicle speed, and depth information, more reasonable and scientific driving route adjustment strategies can be developed, flexibly adjusting vehicle driving methods according to actual conditions to improve driving safety and comfort. Attached Figure Description

[0014] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 This is a schematic diagram illustrating an application scenario of a vehicle driving path planning method provided in an embodiment of this application.

[0016] Figure 2 A flowchart illustrating a vehicle driving path planning method provided in an embodiment of this application;

[0017] Figure 3 A flowchart illustrating a vehicle driving path planning method provided in another embodiment of this application;

[0018] Figure 4 This is a schematic diagram of the structure of a vehicle driving path planning device provided in an embodiment of this application;

[0019] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0020] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, other embodiments obtained by those of ordinary skill in the art without creative effort are all within the protection scope of this application.

[0021] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0022] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0023] With the rapid iteration of artificial intelligence and 5G communication technologies, the adaptability of intelligent driving vehicles to complex road environments is constantly increasing. However, road defects, such as potholes and cracks, remain key factors restricting driving safety and ride comfort. Existing technologies related to road defect detection and path planning mainly fall into two categories: The first is a multi-sensor fusion scheme, which uses a combination of LiDAR and cameras, leveraging 3D point cloud modeling technology to detect pothole depth. Some schemes also introduce data reliability verification and fusion mechanisms to obtain accurate pothole depth results and complete the detection process. The second is a pure vision deep learning scheme, relying on deep learning models such as YOLO and PointNet++ to achieve end-to-end pothole location recognition and detection, directly outputting the coordinate range of the pothole in the image. However, while LiDAR-related schemes can achieve a certain level of depth detection accuracy, they suffer from high hardware purchase costs, complex equipment installation and calibration processes, and insufficient real-time performance due to the large computational load of multi-sensor data fusion. Pure vision schemes based on deep learning, on the other hand, face shortcomings such as high model inference latency and high costs associated with collecting and maintaining massive amounts of labeled training data. Meanwhile, in low-visibility scenarios such as nighttime and rainy days, the detection accuracy of existing solutions is easily affected, and drivers also find it difficult to judge the actual condition of potholes in a timely and accurate manner. Furthermore, current path planning focuses on avoiding conventional obstacles and lacks specific avoidance strategies for road defects, which can easily lead to safety accidents.

[0024] Therefore, when facing the technical challenges of the aforementioned technologies, and addressing the issues of high hardware costs and complex integration of multi-sensor fusion solutions, a pure vision-based solution replaces the LiDAR combination. This solution simultaneously acquires image data using cameras at at least two angles, utilizing binocular vision principles to achieve depth detection. This eliminates the need for additional expensive sensors, significantly reducing hardware costs and simplifying the device integration process. To improve detection efficiency, complex deep learning models are abandoned in favor of a lightweight machine learning image processing strategy. Images are processed in the order of image closing, opening, edge detection, color segmentation, and contour detection. This effectively extracts the location information of the pit areas while reducing computational load, meeting real-time requirements, and eliminating the need for maintaining massive amounts of labeled data. Since the reliability of pit depth detection data is insufficient, affecting the accuracy of path planning, depth information is calculated by combining image data and the location information of the pit areas. Visual cues and geometric relationships in the image can be used for inference, resulting in more accurate depth information. To improve the real-time response of path planning, the location information of the pothole area, the final depth result, and the current vehicle speed are comprehensively used to determine path adjustment parameters such as steering angle and vehicle speed adjustment value. This allows the path planning to directly adapt to the actual state of the pothole, enabling targeted avoidance and improving driving safety and comfort.

[0025] Figure 1 This is a schematic diagram illustrating an application scenario corresponding to a vehicle driving path planning method provided in an embodiment of this application. For example... Figure 1 As shown, the application scenario includes: a first image acquisition device 101, a second image acquisition device 102, and a server 103. The vehicle path planning device can be integrated into the server 103.

[0026] It is understandable that the first image acquisition device 101 and the second image acquisition device 102 acquire image data of the vehicle's driving environment in real time.

[0027] Specifically, server 103 acquires image data in real time from the first image acquisition device 101 and the second image acquisition device 102, processes the image data using a preset image processing strategy to obtain the location information of the pothole area, and then calculates the depth information of the pothole area based on the image data and the location information of the pothole area. Finally, server 103 determines the adjustment parameters of the vehicle driving path based on the location information of the pothole area, the current vehicle speed, and the depth information of the pothole area, so as to plan the vehicle driving path based on the adjustment parameters.

[0028] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0029] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or modules, and may be electrical, mechanical, or other forms.

[0030] Figure 2 This is a flowchart illustrating a vehicle driving path planning method according to an embodiment of this application, as shown below. Figure 2 As shown, the execution subject of this embodiment is a vehicle driving path planning device. This device can be implemented through a computer program, or through a medium storing the relevant computer program, such as a USB flash drive and / or optical disc; alternatively, it can be implemented through a physical device integrating or installing the relevant computer program, such as a chip or electronic device. The electronic device can be a computer or a server, etc. The vehicle driving path planning method provided in this embodiment includes the following steps:

[0031] S201. Acquire image data of the vehicle's driving environment from at least two angles.

[0032] Among them, the image data of the vehicle driving environment refers to the visual information of the vehicle's surroundings related to driving safety and environmental judgment during the vehicle's movement, such as the image of the road ahead.

[0033] Optionally, by deploying image acquisition devices such as vehicle-mounted cameras at different locations on the vehicle, images can be acquired simultaneously while the vehicle is in motion.

[0034] Specifically, the vehicle and image acquisition equipment are activated, and a synchronous triggering mechanism is set to ensure that both cameras begin acquiring images at the same time. During vehicle operation, the vehicle path planning device continuously acquires images based on preset trigger signals, obtaining image data of the vehicle's driving environment from at least two angles simultaneously.

[0035] Optionally, the acquired dual-angle images are preliminarily processed to remove invalid data. For image noise caused by vehicle vibration and light changes during driving, a filtering algorithm is used to smooth the image, such as removing snow noise when shooting at night and reflective spots under strong light.

[0036] S202. The image data is processed using a preset image processing strategy to obtain the location information of the pit area. The preset image processing strategy processes the image data in the following order: image closing operation, image opening operation, edge detection, color segmentation, and contour detection.

[0037] The location information of the pit area refers to the specific location and range parameters of the pit in the image coordinate system and the real-world coordinate system, which are finally determined through image analysis and spatial calibration.

[0038] Specifically, the acquired image data is converted into a format suitable for morphological operations. Then, a structuring element matching the size of the potholes is selected, and image closing operations are performed in the order of dilation followed by erosion. Each pixel in the image is scanned one by one using the selected structuring element, replacing the grayscale value of pixels within the structuring element's coverage area with the maximum value within that range. This expands the edges of the pothole area outwards, filling the small black holes formed by shadows and stains inside the pothole, making the overall shape of the pothole area more complete. Based on the dilated image, the same structuring element is used to scan each pixel again, replacing the grayscale value of pixels within the structuring element's coverage area with the minimum value within that range. This shrinks the expanded pothole edges back to their original approximate shape, offsetting the edge enlargement effect of dilation while preserving the complete shape after filling the holes. After the closing operation, the small holes in the pothole area of ​​the image are filled, and adjacent minor damaged areas are connected into a whole, reducing interference in subsequent processing.

[0039] Furthermore, using the image processed by the closing operation as input, and employing the same or adapted structuring element as the closing operation, the process is performed in the order of erosion followed by dilation. The structuring element scans the image, replacing the minimum grayscale value of pixels within the coverage area, eliminating small noise points attached to the edges of potholes or the normal road surface, while simultaneously separating irrelevant small areas slightly adhered to the potholes. Based on the eroded image, the structuring element scans again, replacing the maximum grayscale value of pixels within the coverage area, restoring the eroded and shrunken pothole area to near its original size and shape. After the opening operation, noise points in the image are removed, the edges of the pothole area become smoother, and the boundary with the normal road surface is clearer.

[0040] Furthermore, using the image after opening as input, the edges of the pit regions are located by calculating the gradient of pixel grayscale values, and the shape contours and intensity abrupt change locations are extracted. Combining the contour cues obtained from edge detection, color segmentation is performed on the image. Based on the edge detection range, the range is further expanded, for example, by 10 pixels outward from the edge. The original image after the expansion is obtained, and the pixels within the expanded range remain unchanged, while other areas are turned black. After converting the acquired image to an HSV image, mask calculation is performed to determine the final pixel values ​​of the pit regions.

[0041] Furthermore, based on the color-segmented image, the complete contours of the pit regions are extracted, and their location information is determined. Starting from the edge of the image, edge points are traced and connected point by point along the boundary between the pit region and the background region to form a closed contour line. If multiple pits exist in the image, the corresponding closed contours are extracted separately, and the area ratio and average color value within the detected region are calculated. Regions that meet preset conditions are identified as pit regions. Based on the closed contours of the pit regions, their coordinate parameters in the image coordinate system are calculated, such as the coordinates of the center point of the contour, the coordinates of the upper left and lower right corners of the circumscribed rectangle of the contour, etc., to obtain the location information of the pit regions.

[0042] S203. Calculate the depth information of the pit area based on the image data and the location information of the pit area.

[0043] The depth information of the pothole area refers to information such as the vertical distance from the road surface reference plane to the lowest point of the pothole and the continuous length of the same depth.

[0044] Understandably, before performing depth information calculations, a normal, undamaged, and smooth road surface area around the pothole is selected from the dual-angle images. This area must appear in both viewpoints simultaneously. The pixel coordinates of this normal road surface area in the two viewpoint images are extracted. Combined with the camera's extrinsic and intrinsic parameters, the corresponding true 3D spatial coordinates of this area are calculated, and a flat plane equation is fitted. Its height value is recorded as the reference height H0.

[0045] Specifically, the location information of the pit area is retrieved, and pixel matching is performed on the pit contours from two viewpoints. Using the pixels of the pit contour from one viewpoint as a reference, the corresponding pixels with the same physical location in the pit contour from the other viewpoint are found. For example, a feature point on the edge of the pit has pixel coordinates (x1, y1) in the first image and corresponding pixel coordinates (x2, y2) in the second image. The disparity value d of the matched pixels is calculated. The disparity is calculated as the difference in horizontal coordinates between the two pixels, i.e., d = |x1 - x2|. Pixel matching and disparity calculation are performed one by one within the pit contour to obtain the disparity distribution data of the pit area.

[0046] Furthermore, the true height of each point in the pit is calculated using the formula Z = B × f / d, where Z is the vertical distance from the real-world point corresponding to the pixel to the camera, B is the straight-line distance between the centers of the two cameras, and f is the focal length of the camera. The difference between the true height Z of each point in the pit and the height H0 of the reference plane is calculated, and the depth D of a point in the pit is D = H0 - Z. min Z min This represents the minimum height within the pit area. By statistically analyzing the depth values ​​of points within the pit, key parameters such as the maximum depth and average depth of the pit are obtained.

[0047] S204. Based on the location information of the pothole area, the current vehicle speed, and the depth information of the pothole area, determine the adjustment parameters of the vehicle driving path, and plan the vehicle driving path based on the adjustment parameters.

[0048] Among them, the driving path adjustment parameters refer to the set of core parameters used to correct the original driving route of the vehicle, including the vehicle's steering angle, steering timing, driving speed correction value, etc.

[0049] The current vehicle speed refers to the actual driving speed collected by the onboard speed sensor at the moment the vehicle obtains information about the pothole.

[0050] Specifically, the system retrieves the pothole's location information in the real-world coordinate system, calculates its relative position to the vehicle's current lane, and assesses the bumpiness based on the pothole's depth and current vehicle speed. The required steering angle is determined based on the offset distance and the remaining distance from the vehicle to the pothole; the closer the remaining distance, the larger the required steering angle. Simultaneously, the vehicle's wheelbase and steering system characteristics must be considered to avoid excessive steering angles that could cause the vehicle to roll over. For example, if the vehicle's wheelbase is 2.8 meters and the remaining distance is 50 meters, the required steering angle is approximately... .

[0051] Optionally, if the pothole is located in the vehicle's current lane and the overlap between its center position and the vehicle's trajectory exceeds 50%, a significant adjustment to the path is required; if the pothole is located at the edge of the lane and the overlap with the trajectory is less than 20%, a minor adjustment to the path or slowing down is required to pass through; if the pothole is located in an adjacent lane, the original path can be maintained without adjustment.

[0052] Optionally, if the distance between the vehicle wheels is greater than the pothole area, and the vehicle can cross the pothole area, then the speed and direction are adjusted so that the wheels pass through the pothole area from both sides; if the vehicle cannot cross the pothole area, then the relationship between the pothole depth and the vehicle speed is calculated, and the principle of minimum steering is applied to reduce vehicle bumps and allow the vehicle to pass through.

[0053] The vehicle driving path planning method provided in this application acquires vehicle driving environment image data from at least two angles, enabling observation of the vehicle's surrounding environment from multiple perspectives. Images from different angles can complement and corroborate each other, helping to more accurately understand the spatial structure and object distribution of the vehicle's surrounding environment and improving the overall cognitive ability of the driving environment. The combined use of graph closing and opening operations makes it easier to fuse pothole locations. After morphological operations and edge detection, HSV color segmentation is performed. Morphological operations first ensure region connectivity, edge detection provides precise boundary lines, and color segmentation provides brightness / color priority constraints, offering significant advantages in suppressing texture false detections, such as cracks and road surface textures, and improving positioning accuracy. Edges are used for positioning, color is used to eliminate shadow / wet spot false detections, and contours provide area and shape constraints, thereby significantly improving accuracy. Using the above method for pothole location detection avoids the use of deep learning, improving detection efficiency. The depth information of the pothole region is a key factor in assessing its impact on vehicle driving; accurately calculating the depth information of the pothole region is crucial for rationally planning the vehicle driving path. By combining image data with the location information of the pothole area to calculate depth information, inferences can be made using visual cues and geometric relationships in the image. This allows for accurate estimation of the actual depth of the pothole while reducing costs. By comprehensively considering the location of the pothole area, current vehicle speed, and depth information, more reasonable and scientific driving path adjustment strategies can be formulated. Vehicle driving methods can be flexibly adjusted according to actual conditions, improving driving safety and comfort.

[0054] As an optional implementation, based on any of the above embodiments, a preset image processing strategy is used to process the image data to obtain the location information of the pit area, including:

[0055] Image data is preprocessed, including grayscale conversion, Gaussian blurring, and adaptive threshold segmentation.

[0056] The target pixel values ​​of the detection area are obtained by sequentially performing image closing, image opening, edge detection, and color segmentation on the preprocessed image data.

[0057] The area ratio and color mean of the detection region are calculated based on the target pixel value.

[0058] The detection area that meets the preset conditions in terms of both area ratio and average color value is identified as the pit area.

[0059] The target pixel value refers to the set of grayscale or color values ​​of pixels within the suspected pit area after morphological operations and color segmentation.

[0060] The area ratio refers to the proportion of pixels in the suspected pothole area to the total number of pixels in the effective area of ​​the image, used to distinguish potholes from small interference objects such as pebbles and cracks on the road.

[0061] The color mean refers to the average color value of pixels within the suspected pit area.

[0062] Among them, the preset conditions refer to the set threshold range, including the reasonable range of area ratio and the standard range of color average.

[0063] Specifically, the image data is converted to grayscale, projecting the three-channel color space onto a single brightness channel. This preserves brightness / texture information, removes color redundancy, reduces subsequent computational complexity, and makes the brightness-based thresholding / edge detection method more stable. Using the grayscale-converted image as input, an appropriate blur kernel size is selected to process each pixel in the image. Low-pass filtering suppresses high-frequency noise and smooths local brightness fluctuations, reducing false positives in thresholding and edge detection. Hyperparameter values ​​are set between 0.8 and 1.6; initially, larger values ​​are used to reduce the impact of noise and small pits, followed by smaller values ​​to make boundaries clearer. For the Gaussian-blurred grayscale image, local thresholds are calculated at each pixel for image binarization, ultimately outputting a clear black-and-white binarized image. Global thresholding, as a supplement to adaptive thresholding, captures pits with significant global contrast.

[0064] Furthermore, a structural element matching the size of the tiny pores inside the potholes is selected. The binarized image is first dilated, then eroded to connect the broken areas within the pothole into a complete whole. The image after the closing operation is then eroded again, followed by dilation to restore the original size of the pothole area and make its edge contours clearer. The pixel grayscale gradient of the image after the opening operation is calculated to locate the locations of abrupt changes in grayscale values, obtaining the initial outline of the pothole. Returning to the color image before preprocessing, the corresponding color region is delineated using the pothole contour obtained from edge detection as the boundary. The red, green, and blue values ​​of the pixels within this region are extracted to form a target pixel value set, thus obtaining the target pixel value.

[0065] Furthermore, the total number of pixels corresponding to the target pixel value is counted, and then the number of effective region pixels in the entire image is calculated. The ratio of the number of pixels in the suspected pit area to the number of pixels in the effective area is the area proportion. For the red, green, and blue values ​​of each pixel in the target pixel value, the average value is calculated separately. The red values ​​of the pixels are added together and then divided by the total number of pixels to obtain the red average value; similarly, the green and blue average values ​​are calculated. The combination of the three color average values ​​is the color average value of the suspected pit area.

[0066] Furthermore, the calculated area percentage and color average are compared with preset conditions to ultimately determine the location of the pit area. For example, the preset range for area percentage is 0.5%-5%, and the preset range for color average is 50-80 for red, 40-70 for green, and 30-60 for blue. If the area percentage of a suspected area falls within the preset range, and the color average also falls within the preset range, then the area is determined to be a pit area; if either condition is not met, it is determined to be an interference area.

[0067] The vehicle path planning method provided in this application preprocesses image data to effectively suppress noise and enhance the separability of target potholes from the background, providing stable input for subsequent thresholding / segmentation, morphological and contour detection. Performing image closing, image opening, edge detection, and color segmentation on the image data sequentially makes the structure of pothole regions in the image clearer, obtaining precise boundary contours of pothole regions, highlighting the color features of pothole regions, and thus significantly improving the accuracy of pothole detection. By calculating the area ratio and color mean of the detection region and setting corresponding preset conditions for judgment, the accuracy and reliability of recognition can be improved, reducing false positives and false negatives, and filtering out small targets, reducing the amount of subsequent computation.

[0068] As an optional implementation, based on any of the above embodiments, the depth information of the pit region is calculated based on image data and the location information of the pit region, including:

[0069] The image data acquisition equipment is calibrated and rectified in three dimensions to obtain relevant parameters of the acquisition equipment.

[0070] Parallax is calculated based on the relevant parameters of the acquisition device and image data, and then subpixel correction is performed.

[0071] The depth information of the pit area is calculated based on the parallax and the location information of the pit area.

[0072] Among them, three-dimensional rectification refers to the operation of geometric correction of images captured by two image acquisition devices.

[0073] The relevant parameters of the acquisition equipment refer to its intrinsic and extrinsic parameters. Intrinsic parameters are the inherent parameters of the camera, such as focal length, pixel size, and lens distortion coefficient; extrinsic parameters are the spatial position parameters between the two cameras, such as the distance between the camera centers and the relative angle. Specifically, the intrinsic parameters of the camera, such as focal length and pixel size, are calculated by analyzing the correspondence between the pixel coordinates of the calibration board corner points in the image and the real-world coordinates. Using one of the cameras as a reference, a world coordinate system is established. By analyzing the corner point positions of the calibration board in the rectified images of the two cameras, the spatial position relationship of the right camera relative to the left camera is calculated, thus obtaining the extrinsic parameters. Based on the calculated intrinsic and extrinsic parameters, a geometric transformation is performed on the rectified images of the two cameras to ensure that the camera detects the same road surface height.

[0074] Furthermore, using the rectified left-view image as a reference, a fixed-size pixel window, such as a 5×5 pixel region, is selected and slid from left to right and from top to bottom. For each pixel region within the window, the pixel window with the most similar grayscale value distribution is searched on the same horizontal row of the right-view image. The difference in pixel position in the horizontal direction between the reference window and the matching window is recorded to obtain the integer disparity.

[0075] Furthermore, interpolation calculations are performed on the grayscale values ​​near the disparity obtained from integer-pixel matching. By utilizing the grayscale value variation patterns of pixels surrounding the matching window, a continuous distribution curve of grayscale values ​​is fitted, and the positions corresponding to the extreme points of the curve are found. Sub-pixel correction is then used to improve the accuracy of disparity calculation.

[0076] Furthermore, the location information of the pothole area is retrieved. Based on the contour coordinates, the corresponding region is delineated in the disparity map, and the disparity values ​​of the pixels within that region are extracted to obtain the disparity set of the pothole area. Simultaneously, the disparity values ​​of the surrounding normal road surface area are extracted, and their average value is calculated as the disparity value of the road surface reference plane. Using the formula Z = B × f / d, the difference between the actual height Z of each point in the pothole and the height H0 of the reference plane is calculated, and the depth D of a point in the pothole is calculated as D = H0 - Z. min Z min It is the minimum height value within the pit area.

[0077] The vehicle path planning method provided in this application can eliminate the influence of equipment errors on depth calculation through calibration, thereby improving the accuracy of measurement results. Stereo rectification can geometrically correct images acquired by two or more cameras, ensuring they have the same coordinate system and viewpoint on the same plane. The relevant parameters of the acquisition equipment are the basis for subsequent calculations of disparity and depth information; accurate parameter relationships ensure the reliability of depth calculation. Disparity refers to the positional difference of the same object in different images; by calculating disparity, the object's depth information can be derived. Subpixel correction technology can further refine the disparity calculation results, improving the accuracy of disparity to the subpixel level. There is a direct mathematical relationship between disparity and depth; using known acquisition equipment parameters and calculated disparity values, the depth information of an object can be accurately calculated using the principle of triangulation. Combining the location information of the pit area, the depth value of each pixel or specific location in the pit area can be accurately determined, thereby obtaining the three-dimensional shape and depth distribution of the pit area.

[0078] As an optional implementation, based on any of the above embodiments, it further includes:

[0079] After the vehicle has traveled a preset distance, the step of acquiring image data to calculate the depth information of the pothole area is repeated at least twice.

[0080] A weighted fusion formula is used to fuse the results of each calculation to obtain the target depth information of the pit area.

[0081] If the target depth information is greater than or equal to the preset depth threshold, the adjustment parameters for the vehicle's driving path are determined.

[0082] If the target depth information is less than the preset depth threshold, the vehicle will continue to travel along the current driving path.

[0083] The preset distance refers to the pre-set vehicle mileage interval. Every time the vehicle travels this distance, a complete depth calculation process is triggered.

[0084] Among them, the target depth information refers to the final value of the pit depth obtained after weighted fusion.

[0085] Among them, the preset depth threshold refers to the pre-set depth threshold, which is the criterion for judging whether potholes will affect vehicle driving.

[0086] Specifically, a preset distance is established, and the vehicle's travel distance can be monitored in real time via an onboard odometer or GPS (Global Positioning System) device. After the vehicle completes the process of acquiring image data and calculating pothole depth for the first time, it continues to travel, and a repeat detection is automatically triggered every time the preset distance is accumulated. During the repeat detection, the parameters of the image acquisition device, the image processing strategy, and the depth calculation method are kept completely consistent with the first detection.

[0087] Furthermore, the pothole depth results obtained from each detection are stored together with the corresponding data such as the relative distance between the vehicle and the pothole during detection and the image sharpness score. According to the preset weight scoring rules, a corresponding weight coefficient is assigned to each group of depth calculation results, and the weighted fusion formula is called to calculate the target depth information of the pothole area.

[0088] Furthermore, a pre-set depth threshold is retrieved to determine the impact of the pothole on vehicle driving. If the target depth information is greater than or equal to the preset depth threshold, the subsequent process continues to determine the adjustment parameters of the vehicle driving path to plan a new safe driving path. If the target depth information is less than the preset depth threshold, it means that the pothole has no significant impact on vehicle driving, no path adjustment is required, and the vehicle can continue driving according to the current driving path.

[0089] The vehicle path planning method provided in this application embodiment may be affected by various accidental factors during a single image acquisition and depth calculation process. By repeatedly executing the relevant steps after the vehicle has traveled a preset distance, multiple sets of independent image data and depth calculation results can be obtained. Utilizing the averaging effect of multiple measurements, the impact of accidental errors on the final result is effectively reduced, improving data reliability. The weighted fusion formula can assign different weights to each calculation result based on its reliability and accuracy, fusing multiple sets of depth calculation results. This fully leverages the advantages of each set of data, resulting in a more accurate and stable target depth information for the pothole area, further improving the accuracy of depth measurement. When the target depth information is less than a preset depth threshold, it indicates that the pothole area has a relatively small impact on vehicle travel. The vehicle can continue traveling along the current path without unnecessary path adjustments, avoiding unnecessary detours due to excessive caution and improving vehicle travel efficiency.

[0090] As an optional implementation, based on any of the above embodiments, the adjustment parameters for the vehicle's driving path are determined according to the location information of the pothole area, the current vehicle speed, and the depth information of the pothole area, including:

[0091] Obtain preset vehicle stability value and preset vehicle bump value.

[0092] Determine whether the pothole area is smaller than the distance between the vehicle wheels based on the location information of the pothole area.

[0093] If so, the steering angle is calculated based on the preset vehicle stability value and the current speed.

[0094] If not, the vehicle speed adjustment value is calculated based on the depth information of the pothole area and the preset vehicle bump value.

[0095] Among them, the preset vehicle stability value refers to the threshold parameter that is set in advance based on the vehicle's own dynamic characteristics, which is used to constrain the calculation range of the steering angle.

[0096] Among them, the preset vehicle bump value refers to the threshold parameter set in advance based on the performance of the vehicle's suspension system and the shock absorption capacity of the tires, which is used to measure the maximum degree of bumps that the vehicle can withstand.

[0097] Among them, the speed adjustment value refers to the speed value that the vehicle needs to increase or decrease based on the original speed.

[0098] Specifically, the system retrieves pre-stored preset vehicle stability and bump values. It then retrieves the location information of the pothole area and extracts its lateral width. Finally, it retrieves the left and right track widths of the current vehicle from the vehicle database. The lateral width of the pothole is then compared with the vehicle's track width.

[0099] Furthermore, if the lateral width of the pothole is less than the vehicle's wheelbase, it means the vehicle can make slight turns to allow both wheels to pass over the normal road surface on either side of the pothole without having to drive over it. The vehicle stability S can then be calculated using equation (1). v The relationship between vehicle speed and steering angle.

[0100]

[0101] Where β is a hyperparameter, v is the current vehicle speed, and R is the steering angle.

[0102] If the lateral width of the pothole is greater than or equal to the vehicle's wheelbase, it means that no matter how the vehicle turns, at least one wheel will run over the pothole, making complete avoidance impossible. Therefore, the vehicle bump value a is calculated using equation (2). h The relationship between vehicle speed and pothole depth.

[0103]

[0104] Where α is a hyperparameter, v' is the vehicle speed adjustment value, and h p For the depth of the pit, l p This represents the continuous length of that depth.

[0105] The vehicle path planning method provided in this application determines whether the pothole area is smaller than the distance between the vehicle wheels, accurately identifying the impact range of the pothole on vehicle driving. This allows for the development of more targeted path adjustment schemes based on actual conditions, improving adjustment effectiveness. When the pothole area is smaller than the wheel spacing, the steering angle is calculated based on a preset vehicle stability value and the current speed, enabling the vehicle to precisely avoid the pothole. This prevents wheels from getting stuck in the pothole and causing loss of vehicle control, ensuring vehicle stability during steering and preventing dangerous situations such as skidding or fishtailing due to excessively sharp or gentle steering. When the pothole area is greater than or equal to the wheel spacing, the pothole affects multiple wheels, exacerbating vehicle bumps. Calculating the vehicle adjustment value based on the pothole depth information and a preset vehicle bump value allows for reasonable adjustment of the vehicle's driving state according to the actual depth of the pothole, reducing the impact force when the vehicle passes through the pothole and lowering the degree of bumps.

[0106] As an optional implementation, based on any of the above embodiments, before processing the image data using a preset image processing strategy, the method further includes:

[0107] The image of the vehicle's driving environment includes the vehicle ahead, and the trajectory, speed, and relative position of the vehicle ahead are determined based on the image data.

[0108] The vehicle bump value is calculated based on the relative position of the vehicle in front.

[0109] If the vehicle's bump value is less than the preset bump threshold, it will proceed according to the trajectory and speed of the vehicle in front.

[0110] If the vehicle bump value is greater than or equal to the preset bump threshold, the image data will be processed using the preset image processing strategy.

[0111] Among them, the vehicle bump value refers to a quantitative indicator that reflects the degree of road bumpiness, calculated based on the driving status of the vehicles ahead.

[0112] Among them, the preset bump threshold refers to the pre-set critical standard for bump value, which is the basis for determining whether to start the pothole image processing strategy.

[0113] Specifically, image recognition technology is used to locate the outline of the vehicle in front in the image. Combined with the intrinsic and extrinsic parameter data calibrated by the camera in the previous step, the image pixel coordinates are converted into real-world coordinates. Using the known width of the vehicle (e.g., a typical sedan is approximately 1.8 meters wide), its pixel width in the image is compared, and the perspective principle (objects appear larger when closer and smaller when farther away) is applied to calculate the actual longitudinal distance between the current vehicle and the vehicle in front. Using the centerline of the current vehicle's lane as a reference, the horizontal distance between the centerline of the vehicle in front and the centerline of the current vehicle's lane is calculated, yielding the lateral offset. The speed of the vehicle in front can be calculated using the longitudinal distance and the current vehicle's speed. Based on the position data of the vehicle in front in multiple consecutive frames of images, the driving path curve of the vehicle in front, i.e., its trajectory, is fitted.

[0114] Furthermore, the vehicle bump value is calculated based on the vertical undulation of the center point of the vehicle ahead relative to the ground. Different strategies are executed based on the comparison between the vehicle bump value and a preset bump threshold. If the calculated bump value is lower than the threshold, it indicates that there are likely no obvious potholes on the road ahead, and the pothole detection image processing flow does not need to be initiated. The vehicle is controlled to follow the trajectory and speed of the vehicle ahead, maintaining a safe longitudinal distance and lateral offset. If the calculated bump value is higher than or equal to the threshold, it indicates that there is a very high probability of potholes on the road ahead, triggering the preset image processing strategy. The acquired image data undergoes preprocessing, morphological operations, and feature calculations according to the established process.

[0115] The vehicle path planning method provided in this application calculates the vehicle bump value based on relative position. It comprehensively considers factors such as the distance between the vehicle ahead and the vehicle itself, and the bump amplitude of the vehicle ahead, providing the vehicle with an intuitive bump level indicator. This allows for the selection of an appropriate driving mode based on actual conditions, avoiding loss of vehicle control or passenger discomfort due to sudden severe bumps, thus improving driving safety and comfort. When the vehicle bump value is less than a preset bump threshold, it indicates that the road conditions ahead are relatively stable, and the vehicle can safely follow the vehicle ahead. Driving based on the trajectory and speed of the vehicle ahead reduces the computational load.

[0116] As an optional implementation, based on any of the above embodiments, before performing the fusion calculation on the results of each calculation, the method further includes:

[0117] Calculate the deviation value by arbitrarily selecting two sets of calculation results.

[0118] If the deviation between the two sets of calculation results is greater than or equal to the preset reliability threshold, then discard the two sets of calculation results and re-execute the step of obtaining image data to calculate the depth information of the pit area.

[0119] If the deviation between the two sets of calculation results is less than the preset reliability threshold, the two sets of calculation results will be included in the valid data set until the amount of data in the valid data set meets the requirements for fusion calculation.

[0120] Among them, the preset reliability threshold refers to the pre-set critical standard for deviation value, which is the basis for judging whether two sets of depth data are consistent.

[0121] Specifically, the pit depth calculation results obtained from multiple previous repeated tests are retrieved, and two sets of data are randomly selected for consistency verification. The absolute value of the difference between the two sets of depth data is calculated, i.e., the deviation value. A pre-calibrated preset reliability threshold is retrieved. If the deviation value of the two sets of depth data is greater than or equal to the preset reliability threshold, it indicates that the consistency between the two sets of data is poor, and at least one set of data has a significant error. These two sets of data are directly discarded, and a re-testing process is triggered. If the deviation value of the two sets of depth data is less than the preset reliability threshold, it indicates that the consistency between the two sets of data is high and they are reliable. Both sets of data are then included in the valid dataset.

[0122] The vehicle path planning method provided in this application can precisely measure the degree of difference between different calculation results in a quantitative manner by calculating the deviation between any two sets of calculation results. The preset reliability threshold, determined through extensive experiments and data analysis, serves as a standard for judging the reliability of the calculation results. When the deviation between two sets of calculation results is greater than or equal to this threshold, it indicates a significant difference between the two sets of results, suggesting that at least one set of results may be erroneous or inaccurate. Discarding these unreliable data prevents them from negatively impacting subsequent fusion calculations, ensuring the accuracy and reliability of the final calculation results. Including calculation results with deviations less than the preset reliability threshold in the valid data set provides a solid foundation for subsequent fusion calculations.

[0123] As an optional implementation, based on any of the above embodiments, when determining the adjustment parameters for the vehicle driving path, the method further includes:

[0124] Determine whether there are other vehicles on both sides of the vehicle and whether there is space to avoid them.

[0125] If there is space to avoid on one side, the steering angle and vehicle speed adjustment values ​​for that side are calculated based on the location and depth information of the pothole area.

[0126] If there is no space to avoid the pothole on either side, the minimum safe speed is calculated based on the pothole depth information.

[0127] The minimum safe speed refers to the lowest speed at which a vehicle can minimize the bumps and avoid losing control when it has to drive over potholes and cannot avoid them.

[0128] Specifically, by combining data from in-vehicle cameras and other sensing devices, the system simultaneously identifies the road conditions on both the left and right sides of the vehicle. For adjacent lanes, it detects whether other vehicles are traveling, recording their relative positions and speeds. For road edges, it detects whether there are fixed obstacles such as guardrails, green belts, and shoulders, determining the lateral distance between these obstacles and the vehicle. Based on the detected environmental data from both sides, it calculates and determines whether a dodgeable space exists. The criteria for this dodgeable space are: its lateral width must be greater than or equal to the vehicle's width plus a safety clearance, and its continuous length must be greater than or equal to the minimum distance required for the vehicle to complete a turn. If the lateral space on either the left or right side meets the criteria, and there are no other vehicles or obstacles on that side, it is determined that one side is dodgeable. If the lateral space on both sides is less than the criteria, or if there are other vehicles or fixed obstacles on both sides, it is determined that there is no dodgeable space.

[0129] Furthermore, for situations where there is space to avoid a pothole on one side, the steering angle and speed adjustment values ​​for that side are calculated. Combining the preset vehicle stability value and the remaining distance from the vehicle to the pothole, the required steering angle is calculated. Simultaneously, a slight reduction in vehicle speed is necessary. The speed adjustment value is calculated based on the pothole depth and steering operation requirements; the greater the pothole depth, the larger the speed adjustment; the larger the steering angle, the larger the speed adjustment.

[0130] Furthermore, if there is no space to avoid the pothole on either side, and the vehicle cannot steer to avoid it and can only drive over it, then the minimum safe speed is calculated. From the preset vehicle bump values, the maximum safe speed corresponding to the current pothole depth is retrieved. The minimum safe speed is usually taken as the maximum safe speed corresponding to the pothole depth. If the current speed is lower than this benchmark, no adjustment is needed; if the current speed is higher than this benchmark, the speed adjustment value is calculated as: current speed - minimum safe speed, meaning the vehicle needs to reduce to the minimum safe speed.

[0131] Optionally, real-time traffic data, such as congested areas and construction sections, can be incorporated into the route planning stage. Combined with a pothole depth and vehicle speed relationship model, avoidance strategies can be dynamically adjusted, such as detouring or slowing down, to avoid entering high-risk areas. Real-time traffic information, such as congestion index and construction section coordinates, can be obtained through an onboard communication module. Traffic risk factors can be introduced into the route planning model, prioritizing routes with low congestion and low construction risk. If multiple high-risk potholes exist on the target route, multi-path planning is triggered, selecting the route with the lowest overall risk. Through joint analysis of real-time traffic data and pothole risk, the global optimization capability of route planning can be significantly improved, reducing detours or congestion caused by local avoidance, while also reducing the risk of vehicle collisions in complex traffic environments.

[0132] The vehicle path planning method provided in this application can promptly detect potential collision risks by determining whether there are other vehicles traveling on both sides of the vehicle. Understanding the avoidable space on both sides of the vehicle is a key prerequisite for determining whether it is safe to avoid potholes. Combining the avoidable space information with the calculation of the steering angle on that side based on the pothole's location information, the vehicle can bypass the pothole at the most suitable angle, avoiding excessive steering leading to loss of control or insufficient steering to avoid the pothole. Calculating the vehicle speed adjustment value based on the pothole depth information allows for a reasonable reduction in vehicle speed according to the pothole's depth, ensuring vehicle stability when passing through potholes and reducing bumps and impacts.

[0133] As an optional implementation, based on any of the above embodiments, a weighted fusion formula is used to fuse the calculation results of each calculation, including:

[0134] The confidence level of each calculation result is determined based on the ambient light intensity, image clarity, and equipment operating status during the acquisition process.

[0135] The confidence level is combined with the variance of the calculation result to construct a comprehensive weighting coefficient.

[0136] Based on the comprehensive weighting coefficients of each calculation result, the calculation results are fused to obtain the target depth information of the pit area.

[0137] Among them, the equipment operating status refers to the working status of the image acquisition equipment, such as whether the camera is shaking or whether the sensor is malfunctioning.

[0138] Specifically, the illumination sensor data during image acquisition is retrieved, and the illumination intensity is categorized into normal illumination, strong light, and weak light. Image sharpness is categorized into high sharpness, medium sharpness, and low sharpness. The device operation log is retrieved to assess the stability of the acquisition process, categorized into stable state, minor anomalies, and severe anomalies. The scores of the above three dimensions are weighted and averaged to obtain the confidence level of the depth calculation result. The weight allocation is set according to the degree of influence of each factor on the result; for example, illumination intensity has a weight of 0.3, image sharpness has a weight of 0.5, and device operation status has a weight of 0.2.

[0139] Furthermore, the average depth calculation results in the effective dataset are calculated; then, the square of the difference between the current depth value and the average value is calculated to obtain the variance of the current result. The confidence level is multiplied by the normalized variance value to obtain the initial weight of the current result; then, the initial weights of the results are summed, and the single initial weight is divided by the total initial weight to obtain the final comprehensive weight coefficient. Using the constructed comprehensive weight coefficient, the depth values ​​in the effective dataset are weighted and summed to obtain the final pit target depth information.

[0140] The vehicle driving path planning method provided in this application combines confidence and variance to construct a comprehensive weight coefficient, which can comprehensively consider the reliability and stability of the data, more accurately reflect the value of each calculation result in the fusion calculation, and improve the accuracy of the fusion result.

[0141] Figure 3 This is a flowchart illustrating a vehicle driving path planning method provided in another embodiment of this application, as shown below. Figure 3 As shown, the vehicle driving path planning method provided in this embodiment includes the following steps:

[0142] S301. Acquire image data of the vehicle's driving environment from at least two angles.

[0143] S302, In response to the image of the vehicle's driving environment including the vehicle ahead, determine the trajectory of the vehicle ahead, the speed of the vehicle ahead, and the relative position of the vehicle ahead based on the image data.

[0144] S303. Calculate the vehicle bump value based on the relative position of the vehicle in front.

[0145] S304. If the vehicle's bump value is less than the preset bump threshold, then drive according to the trajectory and speed of the vehicle in front.

[0146] S305. If the vehicle bump value is greater than or equal to the preset bump threshold, then execute S306.

[0147] S306. Preprocess the image data, including grayscale conversion, Gaussian blurring, and adaptive threshold segmentation.

[0148] S307. The target pixel value of the detection area is obtained by sequentially performing image closing operation, image opening operation, edge detection and color segmentation on the preprocessed image data.

[0149] S308. Calculate the area ratio and color mean of the detection region based on the target pixel value.

[0150] S309. The detection area that meets the preset conditions in terms of both area ratio and color average is identified as the pit area.

[0151] S310. The image data acquisition device is calibrated and rectified in three dimensions to obtain relevant parameters of the acquisition device.

[0152] S311. Calculate the parallax based on the relevant parameters of the acquisition device and the image data, and combine it with subpixel correction.

[0153] S312. Calculate the depth information of the pit area based on the parallax and the location information of the pit area.

[0154] S313. In response to the vehicle traveling a preset distance, repeat S301 and S306-S312 at least twice.

[0155] S314. A weighted fusion formula is used to fuse the results of each calculation to obtain the target depth information of the pit area.

[0156] S315. If the target depth information is greater than or equal to the preset depth threshold, then execute S317.

[0157] S316. If the target depth information is less than the preset depth threshold, then continue driving according to the current vehicle driving path.

[0158] S317. Obtain the preset vehicle stability value and preset vehicle bump value.

[0159] S318. Determine whether the pothole area is smaller than the distance between vehicle wheels based on the location information of the pothole area.

[0160] S319. If so, the steering angle is calculated based on the preset vehicle stability value and the current vehicle speed.

[0161] S320. If not, calculate the vehicle speed adjustment value based on the depth information of the pothole area and the preset vehicle bump value.

[0162] It should be noted that the execution order of S304-S305, S315-S316, and S319-S320 is not important.

[0163] In this embodiment, the implementation method and technical effect of S301-S320 are similar to those of the corresponding solutions in the above embodiments, and will not be repeated here.

[0164] Figure 4 This is a schematic diagram of a vehicle driving path planning device provided in an embodiment of this application. Figure 4 As shown, the vehicle driving path planning device provided in this embodiment is located in an electronic device. The vehicle driving path planning device 40 provided in this embodiment includes: an acquisition module 41, a processing module 42, a calculation module 43, and a determination module 44.

[0165] Specifically, the acquisition module 41 is used to acquire image data of the vehicle driving environment from at least two angles; the processing module 42 is used to process the image data using a preset image processing strategy to obtain the location information of the pothole area. The preset image processing strategy is to process the image data in the following order: image closing operation, image opening operation, edge detection, color segmentation, and contour detection; the calculation module 43 is used to calculate the depth information of the pothole area based on the image data and the location information of the pothole area; and the determination module 44 is used to determine the adjustment parameters of the vehicle driving path based on the location information of the pothole area, the current vehicle speed, and the depth information of the pothole area, so as to plan the vehicle driving path based on the adjustment parameters.

[0166] Optionally, when processing image data using a preset image processing strategy to obtain the location information of the pit region, the processing module 42 specifically performs the following: preprocessing of the image data, including grayscale conversion, Gaussian blurring, and adaptive threshold segmentation; sequentially performing image closing, image opening, edge detection, and color segmentation on the preprocessed image data to obtain the target pixel value of the detection area; calculating the area ratio and color mean of the detection area based on the target pixel value; and identifying the detection area whose area ratio and color mean both meet the preset conditions as the pit region.

[0167] Optionally, when calculating the depth information of the pit region based on the image data and the location information of the pit region, the calculation module 43 is specifically used for: calibrating and stereo rectifying the image data acquisition device to obtain relevant parameters of the acquisition device; calculating the parallax based on the relevant parameters of the acquisition device and the image data and combining it with subpixel correction; and calculating the depth information of the pit region based on the parallax and the location information of the pit region.

[0168] Optionally, the vehicle driving path planning device provided in this application also includes an execution module.

[0169] Accordingly, the execution module is used to repeatedly execute the step of acquiring image data to calculate the depth information of the pothole area at least twice in response to the vehicle traveling a preset distance; the calculation module 43 is also used to perform fusion calculation on the calculation results of each calculation using a weighted fusion formula to obtain the target depth information of the pothole area; the determination module 44 is also used to determine the adjustment parameters of the vehicle driving path if the target depth information is greater than or equal to a preset depth threshold; if the target depth information is less than the preset depth threshold, the vehicle continues to drive according to the current driving path.

[0170] Optionally, the determining module 44, when determining the adjustment parameters of the vehicle driving path based on the location information of the pothole area, the current vehicle speed, and the depth information of the pothole area, is specifically used for: obtaining a preset vehicle stability value and a preset vehicle bump value; determining whether the pothole area is smaller than the distance between the vehicle wheels based on the location information of the pothole area; if so, calculating the steering angle based on the preset vehicle stability value and the current vehicle speed; if not, calculating the vehicle speed adjustment value based on the depth information of the pothole area and the preset vehicle bump value.

[0171] Optionally, the determining module 44 is further configured to, in response to the image of the vehicle's driving environment including a vehicle ahead, determine the trajectory, speed, and relative position of the vehicle ahead based on the image data; the calculating module 43 is further configured to calculate the vehicle bump value based on the relative position of the vehicle ahead; the determining module 44 is further configured to, if the vehicle bump value is less than a preset bump threshold, drive according to the trajectory and speed of the vehicle ahead; if the vehicle bump value is greater than or equal to the preset bump threshold, process the image data using a preset image processing strategy.

[0172] Optionally, the calculation module 43 is further configured to arbitrarily select two sets of calculation results to calculate the deviation value; the determination module 44 is further configured to discard the two sets of calculation results and re-execute the step of acquiring image data to calculate the depth information of the pit area if the deviation between the two sets of calculation results is greater than or equal to a preset reliability threshold; if the deviation between the two sets of calculation results is less than the preset reliability threshold, the two sets of calculation results are included in the effective data set until the amount of data in the effective data set meets the fusion calculation requirements.

[0173] Optionally, the determining module 44 is also used to determine whether there are other vehicles on both sides of the vehicle and whether there is space to avoid them; the calculating module 43 is also used to calculate the steering angle and speed adjustment value of the side if there is space to avoid on one side, based on the position and depth information of the pothole area; if there is no space to avoid on either side, calculate the minimum safe speed based on the depth information of the pothole.

[0174] Optionally, the vehicle driving path planning device provided in this application also includes a construction module.

[0175] Accordingly, the determination module 44 is also used to determine the confidence level of each calculation result based on the ambient light intensity, image clarity and equipment operating status during the acquisition process; the construction module is used to combine the confidence level with the variance of the calculation result to construct a comprehensive weight coefficient; the calculation module 43 is also used to perform fusion calculation on each calculation result based on the comprehensive weight coefficient of each calculation result to obtain the target depth information of the pit area.

[0176] It should be noted that the technical effects of the vehicle driving path planning device in this embodiment have been explained in the embodiments of the above-mentioned vehicle driving path planning method, and therefore will not be repeated in this embodiment.

[0177] Figure 5 A schematic diagram of the structure of the electronic device provided in this application. Figure 5 As shown, the electronic device 50 provided in this embodiment includes at least one processor 51 and a memory 52. ​​Optionally, the electronic device 50 further includes a communication component 53. The processor 51, memory 52, and communication component 53 are connected via a bus.

[0178] In a specific implementation, at least one processor 51 executes computer execution instructions stored in memory 52, causing at least one processor 51 to execute the above-described vehicle driving path planning method embodiment.

[0179] The specific implementation process of processor 51 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0180] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0181] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0182] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0183] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the vehicle driving path planning method embodiments described above when running.

[0184] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0185] The embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the vehicle driving path planning method embodiments described above.

[0186] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the vehicle driving path planning method embodiments described above.

[0187] Any of the components, modules, units, parts, methods, and operations described herein can be implemented using software, firmware, hardware (e.g., fixed logic circuitry), manual processing, or any combination thereof. Alternatively or additionally, any functionality described herein can be executed at least in part by one or more hardware logic components, such as, but not limited to, a central processing unit (CPU), a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), an application-specific standard product (ASSP), a system-on-a-chip (SoC), a complex programmable logic device (CPLD), a microprocessor (MCU), etc. The terms "system," "computing device," or "apparatus" as used herein encompass various means, devices, and machines for processing data, including, for example, one or more programmable processors, computers, SoCs, or combinations thereof. The apparatus may also include code that creates an execution environment for the computer program in question, such as code constituting processor firmware, a protocol stack, a database management system, an operating system, a cross-platform runtime environment, a virtual machine, or one or more combinations thereof. The aforementioned computer program (also known as a program, software, software application, app, script, or code) can be written in any form of programming language, including compiled or interpreted languages, declarative or procedural languages, and can be deployed in any form, including as a standalone program or as a module, component, subroutine, object, or other unit suitable for a computing environment.

[0188] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0189] The present application provides a detailed description of a vehicle driving path planning method and electronic device. Specific examples have been used to illustrate the principles and implementation methods of the present application. The descriptions of these embodiments are merely illustrative and are intended to help understand the method and its core concepts. It should be noted that those skilled in the art can make various improvements and modifications to the present application without departing from its principles, and these improvements and modifications also fall within the scope of protection of the claims.

Claims

1. A vehicle travel path planning method, characterized in that, include: Acquire image data of the vehicle's driving environment from at least two angles; The image data is processed using a preset image processing strategy to obtain the location information of the pit area. The preset image processing strategy processes the image data in the following order: image closing operation, image opening operation, edge detection, color segmentation, and contour detection. The depth information of the pit area is calculated based on the image data and the location information of the pit area. Based on the location information of the pothole area, the current vehicle speed, and the depth information of the pothole area, adjustment parameters for the vehicle's driving path are determined, and the vehicle's driving path is planned based on the adjustment parameters.

2. The method according to claim 1, characterized in that, The step of processing the image data using a preset image processing strategy to obtain the location information of the pit area includes: The image data is preprocessed, including grayscale conversion, Gaussian blurring, and adaptive threshold segmentation. The target pixel value of the detection area is obtained by sequentially performing image closing operation, image opening operation, edge detection, and color segmentation on the preprocessed image data. Calculate the area ratio and average color value of the detection region based on the target pixel values; The detection area that meets the preset conditions in terms of both area ratio and average color value is identified as the pit area.

3. The method according to claim 1, characterized in that, The step of calculating the depth information of the pit region based on the image data and the location information of the pit region includes: The image data acquisition device is calibrated and stereo rectified to obtain relevant parameters of the acquisition device; Parallax is calculated based on the relevant parameters of the acquisition device and image data, and subpixel correction is applied. The depth information of the pit area is calculated based on the parallax and the location information of the pit area.

4. The method according to claim 3, characterized in that, Also includes: After the vehicle has traveled a preset distance, the step of acquiring image data to calculate the depth information of the pothole area is repeated at least twice; A weighted fusion formula is used to fuse the results of each calculation to obtain the target depth information of the pit area; If the target depth information is greater than or equal to a preset depth threshold, then the adjustment parameters for the vehicle's driving path are determined. If the target depth information is less than a preset depth threshold, then the vehicle continues to travel along the current driving path.

5. The method according to claim 1, characterized in that, The step of determining the adjustment parameters for the vehicle's driving path based on the location information of the pothole area, the current vehicle speed, and the depth information of the pothole area includes: Obtain preset vehicle stability values ​​and preset vehicle bump values; Determine whether the pothole area is smaller than the distance between vehicle wheels based on the location information of the pothole area. If so, the steering angle is calculated based on the preset vehicle stability value and the current vehicle speed; If not, the vehicle speed adjustment value is calculated based on the depth information of the pothole area and the preset vehicle bump value.

6. The method according to claim 1, characterized in that, Before processing the image data using a preset image processing strategy, the method further includes: In response to the image of the vehicle's driving environment including the vehicle ahead, the trajectory, speed, and relative position of the vehicle ahead are determined based on the image data. The vehicle bump value is calculated based on the relative position of the vehicle ahead; If the vehicle's bump value is less than the preset bump threshold, then the vehicle will proceed according to the trajectory and speed of the vehicle in front. If the vehicle bump value is greater than or equal to a preset bump threshold, then the image data is processed using a preset image processing strategy.

7. The method according to claim 4, characterized in that, Before performing the fusion calculation on the results of each calculation, the following is also included: Calculate the deviation value by arbitrarily selecting two sets of calculation results; If the deviation between the two sets of calculation results is greater than or equal to the preset reliability threshold, then discard the two sets of calculation results and re-execute the step of obtaining image data to calculate the depth information of the pit area; If the deviation between the two sets of calculation results is less than the preset reliability threshold, the two sets of calculation results will be included in the valid data set until the amount of data in the valid data set meets the requirements for fusion calculation.

8. The method according to claim 1, characterized in that, The method of determining the adjustment parameters for the vehicle's driving path also includes: Determine whether there are other vehicles on either side of the vehicle and whether there is space to avoid them; If there is space to avoid on one side, the steering angle and vehicle speed adjustment values ​​on that side are calculated based on the location and depth information of the pothole area. If there is no space to avoid the pothole on either side, the minimum safe speed is calculated based on the pothole depth information.

9. The method according to claim 4, characterized in that, The weighted fusion formula is used to fuse the results of each calculation, including: The confidence level of each calculation result is determined based on the ambient light intensity, image clarity, and equipment operating status during the acquisition process. The confidence level is combined with the variance of the calculation result to construct a comprehensive weighting coefficient; Based on the comprehensive weighting coefficients of each calculation result, the calculation results are fused to obtain the target depth information of the pit area.

10. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the method as described in any one of claims 1 to 9.

Images