Road condition early warning method and device, vehicle and storage medium

Abstract

The invention relates to a road condition early warning method and device, a vehicle and a storage medium, and the method comprises the steps: obtaining road detection information corresponding to a plurality of sensors in the vehicle when the vehicle is in a driving state; fusing the road detection information to obtain target fusion information; determining road abnormal data according to the target fusion information; and carrying out road condition early warning based on the road abnormal data. In the driving process, the road condition information is automatically collected through a plurality of sensors, the road abnormal data are determined, and then road condition early warning is carried out based on the road abnormal data, so that a user can know the road condition ahead in time, the driving hidden danger is eliminated, and the driving safety is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle navigation, and in particular to a road condition warning method, a road condition warning device, a vehicle and a computer-readable storage medium. Background Art

[0002] In the road detection solutions currently used by various map software, detection equipment is usually used for road mapping. However, these detection equipment are large in size and inconvenient to carry on a daily basis. In addition, the detection process requires a lot of human resources and has a long data update cycle, resulting in some road information not being updated to the map navigation in a timely manner, which will mislead users and prevent users from knowing the road conditions in a timely manner. Summary of the invention

[0003] One of the purposes of the present invention is to provide a road condition warning method to solve the problem in the prior art that users cannot timely know the road conditions, which may easily bring hidden dangers to driving safety; the second purpose is to provide a road condition warning device; the third purpose is to provide a vehicle; the fourth purpose is to provide a computer-readable storage medium.

[0004] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0005] A road condition warning method, the method comprising:

[0006] When the vehicle is in a driving state, obtaining road detection information corresponding to multiple sensors in the vehicle;

[0007] Fusing the road detection information to obtain target fusion information;

[0008] determining road abnormality data according to the target fusion information;

[0009] A road condition warning is performed based on the road abnormality data.

[0010] Optionally, the step of obtaining road detection information corresponding to multiple sensors in the vehicle:

[0011] Obtaining road detection information from a laser radar in the vehicle; and / or,

[0012] Acquiring road detection information of a posture sensor in the vehicle; and / or,

[0013] Obtain road detection information from a visual sensor in the vehicle.

[0014] Optionally, the step of fusing the road detection information to obtain target fusion information includes:

[0015] Determining first abnormal position information based on the road detection information of the laser radar;

[0016] Determining second abnormal position information according to the road detection information of the posture sensor;

[0017] Comparing the road detection information of the laser radar with the road detection information of the visual sensor to determine third abnormal position information;

[0018] The first abnormal location information, the second abnormal location information and the third abnormal location information are fused based on location to obtain target fusion information.

[0019] Optionally, the step of determining the second abnormal position information based on the road detection information of the posture sensor includes:

[0020] Identifying the number of sensors that generate road detection information for the posture sensor;

[0021] Acquire the sensor type of the posture sensor;

[0022] In response to the number of sensors being not less than the types of sensors, second abnormal position information is determined based on the road detection information of the posture sensor.

[0023] Optionally, the step of comparing the road detection information of the laser radar with the road detection information of the visual sensor to determine the third abnormal position information includes:

[0024] Converting the road detection information of the laser radar into two-dimensional image information;

[0025] Based on the same abnormal position area, comparing the two-dimensional image information with the road detection information of the visual sensor;

[0026] In response to a matching degree between the two-dimensional image information and the road detection information of the visual sensor being greater than a preset matching threshold, third abnormal position information is determined based on the abnormal position area.

[0027] Optionally, the step of determining road abnormality data according to the target fusion information includes:

[0028] When the number of occurrences of the same abnormal position area in the plurality of target fusion information is greater than a preset number threshold, road abnormality data is determined based on the same abnormal position area.

[0029] Optionally, the method further comprises:

[0030] Path planning is performed based on the road abnormality data.

[0031] Optionally, the method further comprises:

[0032] Vehicle navigation is performed based on the road abnormality data.

[0033] A road condition warning device, comprising:

[0034] An acquisition module, used for acquiring road detection information corresponding to a plurality of sensors in the vehicle when the vehicle is in a driving state;

[0035] A fusion module, used for fusing the road detection information to obtain target fusion information;

[0036] A determination module, used for determining road abnormality data according to the target fusion information;

[0037] The warning module is used to issue a road condition warning based on the road abnormality data.

[0038] A vehicle comprises a processor, a memory and a computer program stored in the memory and capable of running on the processor, wherein the computer program implements the steps of the road condition warning method as described above when executed by the processor.

[0039] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the road condition warning method described above are implemented.

[0040] Beneficial effects of the present invention:

[0041] (1) The present invention automatically collects road condition information through multiple sensors during driving, determines road abnormality data, and then issues road condition warnings based on the road abnormality data, so that users can understand the road conditions ahead in a timely manner, eliminate driving hazards, and improve driving safety;

[0042] (2) The present invention can determine the road condition ahead through the road detection information of the vehicle's own sensors, reducing the need for special equipment to detect and update roads or maps, thereby reducing the cost of hardware equipment and facilitating detection at any time, thereby achieving low-cost and high-efficiency map updates. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A flowchart of a road condition warning method embodiment of the present invention;

[0044] Figure 2 A flowchart of another method for early warning of road conditions according to an embodiment of the present invention;

[0045] Figure 3 A data fusion schematic diagram of an example of a road condition warning method of the present invention;

[0046] Figure 4 A schematic diagram of an application example of a road condition warning method of the present invention;

[0047] Figure 5 A schematic diagram of an example step of a road condition warning method of the present invention;

[0048] Figure 6 It is a structural block diagram of an embodiment of a road condition warning device of the present invention;

[0049] Figure 7 is a structural block diagram of a vehicle embodiment of the present invention;

[0050] Figure 8 A structural block diagram of a computer-readable storage medium embodiment of the present invention. DETAILED DESCRIPTION

[0051] The following will describe the embodiments of the present invention with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention, not for limiting the scope of protection of the present invention.

[0052] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and thus the drawings only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0053] As intelligent driving technology continues to develop, more users are beginning to try out intelligent driving functions. However, the vehicle's intelligent driving is not good at judging road conditions, which causes users to spend more energy paying attention to road conditions when using intelligent driving, and even turns on intelligent driving. Existing high-end models use magic carpet suspension to adjust the vehicle suspension height in advance, allowing the vehicle to pass through potholes stably, but this is costly, technically difficult, and requires high computing power. In addition, each controller needs to quickly adjust the suspension, shock absorber and other systems in a very short time, and the fault tolerance rate is low.

[0054] The present invention relates to the field of road detection technology and the Internet of Things technology. With the help of abundant sensors and recognition algorithms on the car, after the vehicle turns on the road information recording function, the road condition information is automatically collected during driving, and the vehicle data of the same car company or even multiple car companies can be collected and shared. When a vehicle passes through a road with poor road conditions, the location is automatically detected and marked and uploaded to the cloud. When a certain amount of data at the same location is uploaded, the location information is synchronized to the navigation map, and all subsequent car owners can know the road condition information of this location in advance to avoid some unnecessary safety accidents. In addition, the vehicle automatically records the road information, which is conducive to the maintenance of map information. When the road is repaired, other vehicles will pass through this section repeatedly, and the data can be automatically updated and maintained.

[0055] At the same time, as a further extension of the present invention, the detection of road conditions can, to a certain extent, serve as a basis for traffic flow analysis and road planning in a certain area. For example, in real life, the probability of potholes on roads where transport vehicles often pass is much higher than that on other roads. Therefore, the present invention can introduce an AI big model, which is of certain significance to road condition management and road planning in combination with data analysis.

[0056] In summary, the embodiments of the present invention utilize various sensors and vehicle control chips commonly used in automobiles and connect to the vehicle cloud to obtain multi-vehicle data and perform multi-data fusion judgment, thereby better performing road condition detection and data analysis, and summarizing the data as map navigation warnings and intelligent driving decision-making solutions.

[0057] Reference Figure 1 , shows a flow chart of the steps of an embodiment of a road condition warning method of the present invention. The road condition warning method may specifically include the following steps:

[0058] Step 101, when the vehicle is in a driving state, obtaining road detection information corresponding to multiple sensors in the vehicle;

[0059] In the embodiment of the present invention, a plurality of different sensors may be installed on the vehicle, including but not limited to visual sensors, laser radars, accelerometers, gyroscopes, positioners, travel switches, etc. The number of each type of sensor is not specified, such as 6 visual sensors, 4 accelerometers and gyroscopes, 1 laser radar, etc.

[0060] When the vehicle is in motion, multiple sensors in the vehicle will detect the road it is traveling on and generate road detection information. The road detection information corresponding to these sensors can be obtained.

[0061] Step 102, fusing the road detection information to obtain target fusion information;

[0062] The obtained road detection information can be fused, thereby fusing the road conditions detected by different sensors to obtain target fusion information, so that the road detection information of multiple different sensors can be used to identify and verify each other.

[0063] Step 103, determining road abnormality data according to the target fusion information;

[0064] Based on the fused target fusion information, the road detection information of multiple different sensors is mutually identified and verified, the abnormal location area on the driving road is determined, and the road abnormality data is generated. The road abnormality data is used to characterize the road area where the road has abnormalities.

[0065] Step 104: Provide a road condition warning based on the road abnormality data.

[0066] After obtaining the road anomaly data, it is possible to know the road areas where abnormalities exist. When the user is about to enter the corresponding abnormal road area, the road condition can be warned in advance, allowing the user to prepare for the route in advance, thereby improving driving safety.

[0067] The embodiment of the present invention obtains road detection information corresponding to multiple sensors in the vehicle when the vehicle is in a driving state; fuses the road detection information to obtain target fusion information; determines road abnormality data based on the target fusion information; and issues a road condition warning based on the road abnormality data. By automatically collecting road condition information through multiple sensors during driving, determining road abnormality data, and then issuing a road condition warning based on the road abnormality data, the user can timely understand the road conditions ahead, eliminate driving hazards, and improve driving safety.

[0068] Reference Figure 2 , shows a flowchart of another method for early warning of road conditions according to an embodiment of the present invention. The method for early warning of road conditions may specifically include the following steps:

[0069] Step 201, when the vehicle is in a driving state, obtaining road detection information corresponding to multiple sensors in the vehicle;

[0070] When the vehicle is in motion, the road detection information corresponding to multiple sensors in the vehicle can be obtained. In addition, the road detection information corresponding to multiple sensors in different vehicles can also be obtained. Among them, these sensors include but are not limited to laser radar, visual sensor, and attitude sensor. The attitude sensor is a sensor that detects the vehicle attitude, such as an accelerometer and a gyroscope.

[0071] The embodiments of the present invention do not specifically limit the method for detecting whether the vehicle is in a driving state. For example, the vehicle internal state information, such as vehicle speed, steering wheel angle, brake state, etc., is obtained through vehicle-mounted sensors. By monitoring the changes of the vehicle internal sensors, it is possible to determine in real time whether the vehicle is in a driving state, and integrate the detection results with the external sensors to improve the accuracy of vehicle state detection.

[0072] In an optional embodiment of the present invention, the step of obtaining road detection information corresponding to multiple sensors in the vehicle includes: obtaining road detection information of a laser radar in the vehicle; and / or obtaining road detection information of a posture sensor in the vehicle; and / or obtaining road detection information of a visual sensor in the vehicle.

[0073] Regarding the acquisition of road detection information, it can be specifically determined based on the hardware configuration of the vehicle itself. For example, if the vehicle is equipped with laser radar, visual sensor, and posture sensor, then the road detection information of the laser radar, visual sensor, and posture sensor can be obtained; if only some of the laser radar, visual sensor, and posture sensor are provided, then the road detection information of the corresponding existing sensors can be obtained. That is, the road detection information of the laser radar in the vehicle can be obtained; and / or, the road detection information of the posture sensor in the vehicle can be obtained; and / or, the road detection information of the visual sensor in the vehicle can be obtained. Among them, different sensors can be synchronously acquired based on the same timing synchronization, or asynchronously acquired, and the embodiment of the present invention does not specifically limit this.

[0074] Step 202, fusing the road detection information to obtain target fusion information;

[0075] The road detection information obtained from different sensors can be fused to obtain target fusion information.

[0076] In an optional embodiment of the present invention, the step of fusing the road detection information to obtain target fusion information includes: determining first abnormal position information based on the road detection information of the laser radar; determining second abnormal position information based on the road detection information of the posture sensor; comparing the road detection information of the laser radar with the road detection information of the visual sensor to determine third abnormal position information; fusing the first abnormal location information, the second abnormal location information and the third abnormal location information based on position to obtain target fusion information.

[0077] First, the road detection information of the laser radar can be used to fit and generate road information based on the road detection signal of the laser radar, and the first abnormal position information can be generated from the abnormal area determined in the road. Among them, the laser radar obtains the spatial position information of the target object by emitting laser pulses and receiving the reflected signals. These reflected signals are converted into point cloud data, and each point contains information such as three-dimensional coordinates (XYZ) and laser reflection intensity (Intensity). This information is the basis for subsequent processing. Then the point cloud data is preprocessed. Denoising and filtering: The original point cloud data often contains noise and clutter, which may come from reflections on the surface of the object, errors in the laser radar equipment, etc. Therefore, it is necessary to filter and denoise the point cloud data to improve the accuracy of subsequent processing. Common filtering methods include mean filtering, median filtering, and Gaussian filtering. Data format conversion: The original point cloud data is usually stored in a specific format, such as pcap format. In order to facilitate subsequent processing, it needs to be converted into a more general format, such as pcd format. The pcd format file contains information such as the three-dimensional coordinates and reflection intensity of the point cloud, which is convenient for subsequent analysis and processing. Processing and analysis of point cloud data: 1. Point cloud clustering: Since the number of point clouds in road point cloud data is very large, it is necessary to cluster the point clouds to reduce the amount of data and improve processing efficiency. Common clustering methods include K-means clustering, neural network clustering, density-based clustering, etc. Through clustering, point cloud data can be divided into different clusters, each cluster represents a road feature or obstacle, etc. 2. Feature extraction: On the basis of clustering, feature extraction of point cloud data is required. The purpose of feature extraction is to extract useful information from point cloud data, such as road edge lines, lane lines, traffic signs, etc. Common feature extraction methods include algorithms based on point cloud fitting, algorithms based on feature extraction, and algorithms based on density clustering. 3. Motion distortion compensation: In motion scenes, such as when a vehicle is driving at high speed or turning, the point cloud data collected by the lidar may produce motion distortion. In order to eliminate this distortion, it is necessary to perform motion distortion compensation on the point cloud data. The compensation method is usually to unify the point cloud of the same frame into the coordinate system at the same time to eliminate the error caused by motion. Then, for road marking extraction, based on the processed point cloud data, a specific algorithm can be used to extract road markings. For example, a road marking extraction method based on connection points can be used to determine the position and direction of road markings through clustering algorithms and connection point calculations. In addition, methods such as intensity calibration and threshold segmentation can be used to improve the accuracy of road marking extraction. Integrating road information: After extracting road markings, other road information needs to be integrated, such as road width, curvature, slope, etc. This information can be obtained by analyzing the feature information in the point cloud data. For example, the width and curvature of the road can be estimated by calculating the distance and angle between adjacent points in the point cloud data.Then, an abnormal road position is determined, namely, first abnormal position information. The first abnormal position information is used to characterize the abnormal position detected by the laser radar.

[0078] Then, the road detection information of the attitude sensor can be used to fit and generate information about road changes based on the road detection signal of the attitude sensor, and the second abnormal position information can be generated from the abnormal area determined in the road. The second abnormal position information is used to characterize the abnormal position detected by the attitude sensor. The accelerometer and gyroscope (usually integrated in the same sensor module) can be installed on the vehicle, usually near the center of gravity of the vehicle to reduce the impact of rotational acceleration on the measurement. The sensor module will collect acceleration and angular velocity data along three axes in real time. These data are transmitted to a data acquisition device (such as a computer or a digital signal processor DSP) via a data line for further processing. After obtaining the raw data of the attitude sensor, it is necessary to convert the unit to convert the acceleration data into g (gravitational acceleration) or m / s 2 , convert the angular velocity data to ° / s or rad / s. This usually involves converting the value read by the register to the sensitivity (Sensitivity) or resolution (LSB / unit) of the sensor. Calculate the pitch and roll angles. Use the angular velocity data measured by the gyroscope to integrate the angle function with respect to time to calculate the pitch and roll angles of the vehicle. These angles reflect the degree of inclination of the vehicle relative to the horizontal plane. Correct the acceleration data. Since the accelerometer measures the gravity component when the vehicle is tilted, the acceleration data needs to be corrected using the pitch and roll angles. By subtracting the gravity component caused by the tilt from the acceleration data, the actual motion acceleration of the vehicle can be obtained. Detect road changes. The corrected acceleration data can be used to detect changes in the road. For example, when a vehicle is driving on an uneven road, additional acceleration changes will occur, which can be captured and analyzed by acceleration data. At the same time, the angular velocity data measured by the gyroscope can also be used to detect changes such as steering and tilt of the road. Determine the abnormal position in the road based on these changes.

[0079] Then, the road detection information detected by the visual sensor can be identified to obtain the corresponding road conditions, and then based on the comparison of the corresponding road conditions with the road detection information of the laser radar, it is determined whether the corresponding abnormal position detected by the road detection information of the laser radar and the road detection information of the visual sensor is actually abnormal, thereby generating third abnormal position information. The data of the visual sensor is identified to obtain the road conditions. For example, image preprocessing is first performed. Reduce the image resolution: In order to reduce the amount of calculation for subsequent processing, the original image is usually processed by reducing the resolution. Remove noise: The image may contain various noises, such as Gaussian noise, salt and pepper noise, etc. These noises will interfere with subsequent image processing, so they need to be removed by filtering and other methods. Adjust the contrast: By adjusting the contrast of the image, the useful information in the image can be enhanced, making the subsequent processing more accurate. Then feature extraction is performed. In the preprocessed image, features related to the road need to be extracted. These features include information such as edges, textures, colors, depths, etc., which can reflect the shape, structure and relationship of the road with other objects. Feature extraction methods are usually based on computer vision and image processing technologies, such as edge detection, texture analysis, color space conversion, etc. Then perform feature classification and segmentation in the image. Road classification: divide the pixels in the image into road pixels and non-road pixels. This step is usually achieved through machine learning algorithms or deep learning algorithms, such as support vector machines (SVM) and convolutional neural networks (CNN). These algorithms can classify pixels based on the extracted features to distinguish between road and non-road areas. Road segmentation: on the basis of classification, separate road pixels and non-road pixels. This step usually uses semantic segmentation algorithms, such as full convolutional networks (FCN). Semantic segmentation algorithms can finely classify each pixel in the image to obtain more accurate and complete road information. Finally, enter the post-processing stage, and further correct and improve the segmentation results. This includes operations such as eliminating noise, reducing cracks, and filling holes to improve the accuracy and completeness of road information. At the same time, the road information needs to be smoothed to reduce jitter and unevenness caused by image noise and segmentation errors. After the above processing steps, the visual sensor can generate specific road information. This information includes parameters such as the location, shape, width, curvature, and the relationship between the road and other objects (such as vehicles, pedestrians, traffic signs, etc.).

[0080] Then, the first abnormal location information, the second abnormal location information and the third abnormal location information are fused based on the locations to obtain target fused information.

[0081] Specifically, the step of determining the second abnormal position information based on the road detection information of the posture sensor includes: identifying the number of sensors that generate the road detection information of the posture sensor; obtaining the sensor type of the posture sensor; in response to the number of sensors being not less than the sensor type, determining the second abnormal position information based on the road detection information of the posture sensor.

[0082] For the determination of the second abnormal position information of the attitude sensor, in order to ensure the accuracy of recognition, the number of sensors that need to detect abnormal information is greater than the type of sensors to avoid false detection and recognition errors. The number of sensors that generate the road detection information of the attitude sensor can be identified; the number of sensors can be determined based on the source of the road detection information. And because the sensors on the vehicle have been determined before leaving the factory. Its type will not change. The sensor type of the attitude sensor can be obtained from the relevant control memory of the vehicle; it is determined whether the number of sensors is not less than the sensor type. When the number of sensors is not less than the sensor type, it means that there is no false detection. In response to the number of sensors being not less than the sensor type, the second abnormal position information can be determined based on the road detection information of the attitude sensor. When the number of sensors is less than the sensor type, it means that there is a false detection. In response to the number of sensors being less than the sensor type, the second abnormal position information can be discarded.

[0083] Furthermore, the step of comparing the road detection information of the laser radar with the road detection information of the visual sensor to determine the third abnormal position information includes: converting the road detection information of the laser radar into two-dimensional image information; based on the same abnormal position area, comparing the two-dimensional image information with the road detection information of the visual sensor; in response to the degree of match between the two-dimensional image information and the road detection information of the visual sensor being greater than a preset matching threshold, determining the third abnormal position information based on the abnormal position area.

[0084] For the comparison between the road detection information of the laser radar and the road detection information of the visual sensor, the road detection information of the laser radar can be converted into two-dimensional image information; the specific conversion process can be: 1. Projection transformation, such as spherical projection: Spherical projection is a method of projecting a three-dimensional point cloud onto a two-dimensional plane. It is based on the spherical coordinate system and converts the three-dimensional point (x, y, z) into spherical coordinates (r, θ, φ), where r is the distance from the point to the origin, θ is the azimuth, and φ is the zenith angle. Then, the three-dimensional point is projected onto the two-dimensional plane through the mapping relationship from the sphere to the plane. Usually, this mapping relationship takes into account factors such as the vertical field of view (FOV) and horizontal resolution of the laser radar to ensure that the projected image can retain as much useful information as possible. The projected two-dimensional image is usually called a range image (RangeImages), in which the value of each pixel represents the distance from the corresponding point to the laser radar. 2. Image processing. Image preprocessing: After the projection transformation, the obtained two-dimensional image may need to be preprocessed to remove noise, enhance contrast, etc. These preprocessing operations help improve the accuracy and efficiency of subsequent image processing. Feature extraction: In the preprocessed image, features related to the road or obstacle can be extracted to form a two-dimensional image. These features can be edges, textures, shapes, etc., which can reflect the shape, structure and location information of the road or obstacle.

[0085] Then, based on the same abnormal position area, the two-dimensional image information is compared with the image in the road detection information of the visual sensor; the matching degree of the two-dimensional image information of the laser radar and the road detection information of the visual sensor is obtained. It is judged whether the matching degree of the two-dimensional image information of the laser radar and the road detection information of the visual sensor is greater than the preset matching threshold. The size of the preset matching threshold can be determined according to the actual situation, and the embodiment of the present invention does not specifically limit this. When the matching degree of the two-dimensional image information of the laser radar and the road detection information of the visual sensor is greater than the preset matching threshold, it means that the abnormal position really exists, and the third abnormal position information can be determined based on the abnormal position area in response to the matching degree of the two-dimensional image information and the road detection information of the visual sensor being greater than the preset matching threshold. On the contrary, when the matching degree of the two-dimensional image information of the laser radar and the road detection information of the visual sensor is not greater than the preset matching threshold, it means that the abnormal position does not exist and needs to wait until the next round of verification.

[0086] Step 203, determining road abnormality data according to the target fusion information;

[0087] The target fusion information of each sensor in the target fusion information is identified to determine the road abnormality data.

[0088] In an optional embodiment of the present invention, the step of determining road abnormality data based on the target fusion information includes: when the number of occurrences of the same abnormal position area in multiple pieces of the target fusion information is greater than a preset number threshold, determining road abnormality data based on the same abnormal position area.

[0089] For the data generated when a single vehicle passes through the same abnormal location area, or when multiple vehicles pass through the same abnormal location area, when it is determined that the number of occurrences of the same abnormal location area in multiple target fusion information is greater than a preset number threshold, the road abnormality data is determined according to the actual geographical location of the same abnormal location area. The preset number threshold can also be determined according to actual conditions, and the embodiment of the present invention does not specifically limit this.

[0090] For the process of data fusion judgment, please refer to Figure 3 The laser radar can perform point cloud preprocessing on the road section in front of the vehicle while driving, and obtain the path 3D detection frame through the point cloud data difference, and convert the detection frame into 3D image and 2D image through coordinate transformation. After comparing with the intact road point cloud data for noise reduction and troubleshooting of erroneous records, it is synchronously recorded as radar data 1.

[0091] The gyroscope and other sensors need to be calibrated for data of driving under normal and smooth road conditions at various vehicle speeds. When the data of each sensor deviates greatly from the normal data during vehicle driving, the sensor abnormal data 2 at this time needs to be recorded. Considering that during vehicle driving, when the vehicle brakes suddenly on a stable road, just goes uphill, or just goes downhill, the gyroscope, three-axis acceleration sensor, etc. may have data that is greatly different from the data during horizontal driving. Therefore, the sensor abnormal data 2 is recorded only when the number of sensors A with abnormal data on the vehicle is greater than or equal to the number of sensor types B (avoiding data abnormalities caused by abnormalities of the sensors themselves, such as one tire pressure sensor for each of the four tires, such as a single tire blowout on the vehicle causing abnormal tire pressure sensor data, and the number of sensors involved in the tire pressure change caused by the front and rear wheels of the vehicle passing through potholes at the same time. That is, when the data of multiple sensors of the same type on the vehicle changes at the same time, and a sensor is abnormal, but other multiple sensors all recognize the abnormality, the possibility of road abnormality is far greater than the vehicle itself abnormality.).

[0092] The body camera is used as a supplement to the data collection information. When the vehicle sensor abnormal data 2 appears during the driving process of the vehicle, the vehicle camera is called to record the road surface image information, and the acquired road surface image is fused and compared with the 3D image and 2D image converted from the laser radar point cloud data 1. If the matching degree is greater than a certain threshold, it is recorded as road abnormal data 3, and the location information is simultaneously recorded and uploaded to the cloud;

[0093] The solution involves instruments such as laser radar, body cameras, and posture detection sensors, which may be affected by the driver's driving skills, light brightness, different lanes, etc., resulting in erroneous data recording and affecting users. To avoid the impact that may be caused by this situation, when the cloud matches that multiple vehicles record road abnormality data 3 near the same location, it will be officially recorded as road abnormality data.

[0094] In this example, in order to prevent normal road designs such as speed bumps from affecting vehicle recording information, it is necessary to collect various driving characteristics of normal road designs such as speed bumps in advance and screen out such data. The radar data and sensor data of normal road conditions such as normal driving through road speed bumps, normal driving through road shock deceleration lines, and normal road slopes at various speeds are collected and stored in the cloud database. If the radar data and abnormal data obtained by the sensor when the vehicle passes through the road are significantly different from the normal data in the database, they will be treated as abnormal data.

[0095] Step 204, providing a road condition warning based on the road abnormality data;

[0096] After obtaining the road abnormality data, a road condition warning can be issued based on the abnormal road surface, so as to promptly remind the user to pay attention to related driving operations.

[0097] Step 205, performing path planning based on the road abnormality data;

[0098] For the application of road anomaly data, the road anomaly data can also be used for route planning, thereby avoiding driving on abnormal roads.

[0099] Step 206: Perform vehicle navigation based on the road anomaly data.

[0100] Road anomaly data can also be used for vehicle navigation to guide users to pass abnormal roads safely.

[0101] For the application of road abnormal data, please refer to Figure 4 The recorded data can be used as a basis for acceleration and deceleration decisions during intelligent driving. After the vehicle obtains the abnormal road data on the map in advance, it starts to slow down in advance to avoid dangerous situations caused by assisted driving and cruise control. At the same time, the recorded road abnormal data is updated regularly according to the vehicle driving situation. When multiple vehicles pass this location in the future, the re-collected data is characterized as normal data. The cloud deletes the previous abnormal data and synchronizes the deletion of the warning prompt information in the map.

[0102] As a type of road condition collection system, this system can combine AI big models to collect statistics on abnormal road locations stored in the cloud, and label sections where abnormalities are prone to occur and sections where abnormalities have not been repaired in time (for example, if a section frequently experiences abnormalities → normal → abnormal, it can be considered that there is a maintenance risk for this section, and it can be improved during road planning). Through large-scale and long-term data analysis and training, the accuracy of road condition recognition and the timeliness of data updates are improved. Focus on analyzing sections where abnormal road conditions frequently occur, analyze the characteristics of vehicles in abnormal sections by judging the traffic volume and vehicle types, and formulate labels to help with subsequent road traffic planning and maintenance, further saving road operation and maintenance costs.

[0103] The embodiment of the present invention obtains road detection information corresponding to multiple sensors in the vehicle when the vehicle is in a driving state; fuses the road detection information to obtain target fusion information; determines road abnormality data based on the target fusion information; performs road condition warning based on the road abnormality data; performs path planning based on the road abnormality data; and performs vehicle navigation based on the road abnormality data. By automatically collecting road condition information through multiple sensors during driving, determining road abnormality data, and then performing road condition warning based on road abnormality data, the user can timely understand the road condition ahead, eliminate driving hazards, and improve driving safety; the road condition ahead can be determined through the road detection information of the vehicle's own sensors, reducing the use of special equipment for road or map detection and update, thereby reducing the cost of hardware equipment and facilitating detection at any time, and realizing low-cost and high-efficiency map update; by obtaining vehicle data for multi-data fusion judgment, road condition detection and data analysis can be better performed, and the data can be summarized as a map for navigation, warning and intelligent driving decision-making solutions, so that the road abnormality data can be used in the vehicle's road condition warning, path planning and vehicle navigation, which is more practical.

[0104] In order to make the implementation process of the embodiment of the present invention clear to those skilled in the art, an example is used to illustrate the following:

[0105] You can refer to Figure 5When driving, the vehicle comprehensively judges the current vehicle driving posture and road surface data based on the data from the laser radar (if there is no radar, this data will not be counted), various cameras on the vehicle body, and posture detection sensors (such as gyroscopes, three-axis acceleration sensors, vehicle height sensors, etc.). If the radar data can clearly identify that the road features are potholes / bumps, and the data of vehicle posture sensors such as the vehicle's three-axis acceleration sensor and vehicle height sensor suddenly oscillate significantly after being in a long-term stable state, then this part of the data can be regarded as abnormal road data; and the road surface data that meets the abnormal conditions will be uploaded to the cloud. When other vehicles pass the same section of road, if there is the same data record, it will be listed as prominent data. After the data is recorded a certain number of times, it can be determined as an abnormal road condition point, which can be used as a prompt basis for the map software.

[0106] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0107] Reference Figure 6 , shows a structural block diagram of an embodiment of a road condition warning device of the present invention, the device comprising:

[0108] An acquisition module 601 is used to acquire road detection information corresponding to multiple sensors in the vehicle when the vehicle is in a driving state;

[0109] A fusion module 602 is used to fuse the road detection information to obtain target fusion information;

[0110] A determination module 603, configured to determine road abnormality data according to the target fusion information;

[0111] The warning module 604 is used to issue a road condition warning based on the road abnormality data.

[0112] In an optional embodiment of the present invention, the acquisition module 601 includes:

[0113] A first acquisition submodule is used to acquire road detection information of a laser radar in the vehicle; and / or,

[0114] A second acquisition submodule is used to acquire road detection information of a posture sensor in the vehicle; and / or,

[0115] The third acquisition submodule is used to acquire road detection information of the visual sensor in the vehicle.

[0116] In an optional embodiment of the present invention, the fusion module 602 includes:

[0117] A first abnormal position determination submodule, used to determine first abnormal position information according to the road detection information of the laser radar;

[0118] A second abnormal position determination submodule, used to determine second abnormal position information according to the road detection information of the posture sensor;

[0119] A third abnormal position determination submodule is used to compare the road detection information of the laser radar with the road detection information of the visual sensor to determine third abnormal position information;

[0120] The fusion submodule is used to fuse the first abnormal location information, the second abnormal location information and the third abnormal location information based on the location to obtain target fusion information.

[0121] In an optional embodiment of the present invention, the second abnormal position determination submodule includes:

[0122] An identification unit, used to identify the number of sensors that generate the road detection information of the posture sensor;

[0123] An acquisition unit, used for acquiring the sensor type of the posture sensor;

[0124] The first response unit is used to determine second abnormal position information according to the road detection information of the posture sensor in response to the number of sensors being not less than the type of sensors.

[0125] In an optional embodiment of the present invention, the third abnormal position determination submodule includes:

[0126] A conversion unit, used to convert the road detection information of the laser radar into two-dimensional image information;

[0127] A comparison unit, used for comparing the two-dimensional image information with the road detection information of the visual sensor based on the same abnormal position area;

[0128] The second response unit is used to determine third abnormal position information based on the abnormal position area in response to the matching degree between the two-dimensional image information and the road detection information of the visual sensor being greater than a preset matching threshold.

[0129] In an optional embodiment of the present invention, the determining module 603 includes:

[0130] The abnormality determination submodule is used to determine road abnormality data based on the same abnormal position area when the number of occurrences of the same abnormal position area in the multiple target fusion information is greater than a preset number threshold.

[0131] In an optional embodiment of the present invention, the device further comprises:

[0132] Path planning is performed based on the road abnormality data.

[0133] In an optional embodiment of the present invention, the device further comprises:

[0134] Vehicle navigation is performed based on the road abnormality data.

[0135] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0136] Reference Figure 7 , an embodiment of the present invention further provides a vehicle, comprising:

[0137] A processor 701 and a storage medium 702, wherein the storage medium 702 stores a computer program executable by the processor 701, and when the vehicle is running, the processor 701 executes the computer program to perform the road condition warning method as described in any one of the embodiments of the present invention.

[0138] The road condition warning method comprises:

[0139] When the vehicle is in a driving state, obtaining road detection information corresponding to multiple sensors in the vehicle;

[0140] Fusing the road detection information to obtain target fusion information;

[0141] determining road abnormality data according to the target fusion information;

[0142] A road condition warning is performed based on the road abnormality data.

[0143] Optionally, the step of obtaining road detection information corresponding to multiple sensors in the vehicle:

[0144] Obtaining road detection information from a laser radar in the vehicle; and / or,

[0145] Acquiring road detection information of a posture sensor in the vehicle; and / or,

[0146] Obtain road detection information from a visual sensor in the vehicle.

[0147] Optionally, the step of fusing the road detection information to obtain target fusion information includes:

[0148] Determining first abnormal position information based on the road detection information of the laser radar;

[0149] Determining second abnormal position information according to the road detection information of the posture sensor;

[0150] Comparing the road detection information of the laser radar with the road detection information of the visual sensor to determine third abnormal position information;

[0151] The first abnormal location information, the second abnormal location information and the third abnormal location information are fused based on location to obtain target fusion information.

[0152] Optionally, the step of determining the second abnormal position information based on the road detection information of the posture sensor includes:

[0153] Identify the number of sensors that generate road detection information for the posture sensor;

[0154] Acquire the sensor type of the posture sensor;

[0155] In response to the number of sensors being not less than the types of sensors, second abnormal position information is determined based on the road detection information of the posture sensor.

[0156] Optionally, the step of comparing the road detection information of the laser radar with the road detection information of the visual sensor to determine the third abnormal position information includes:

[0157] Converting the road detection information of the laser radar into two-dimensional image information;

[0158] Based on the same abnormal position area, comparing the two-dimensional image information with the road detection information of the visual sensor;

[0159] In response to a matching degree between the two-dimensional image information and the road detection information of the visual sensor being greater than a preset matching threshold, third abnormal position information is determined based on the abnormal position area.

[0160] Optionally, the step of determining road abnormality data according to the target fusion information includes:

[0161] When the number of occurrences of the same abnormal position area in the plurality of target fusion information is greater than a preset number threshold, road abnormality data is determined based on the same abnormal position area.

[0162] Optionally, the method further comprises:

[0163] Path planning is performed based on the road abnormality data.

[0164] Optionally, the method further comprises:

[0165] Vehicle navigation is performed based on the road abnormality data.

[0166] The memory may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the processor.

[0167] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0168] Reference Figure 8 The embodiment of the present invention further provides a computer-readable storage medium 801, on which a computer program is stored. When the computer program is executed by a processor, the road condition warning method as described in any one of the embodiments of the present invention is executed.

[0169] The road condition warning method comprises:

[0170] When the vehicle is in a driving state, obtaining road detection information corresponding to multiple sensors in the vehicle;

[0171] Fusing the road detection information to obtain target fusion information;

[0172] determining road abnormality data according to the target fusion information;

[0173] A road condition warning is performed based on the road abnormality data.

[0174] Optionally, the step of obtaining road detection information corresponding to multiple sensors in the vehicle:

[0175] Obtaining road detection information from a laser radar in the vehicle; and / or,

[0176] Acquiring road detection information of a posture sensor in the vehicle; and / or,

[0177] Obtain road detection information from a visual sensor in the vehicle.

[0178] Optionally, the step of fusing the road detection information to obtain target fusion information includes:

[0179] Determining first abnormal position information based on the road detection information of the laser radar;

[0180] Determining second abnormal position information according to the road detection information of the posture sensor;

[0181] Comparing the road detection information of the laser radar with the road detection information of the visual sensor to determine third abnormal position information;

[0182] The first abnormal location information, the second abnormal location information and the third abnormal location information are fused based on location to obtain target fusion information.

[0183] Optionally, the step of determining the second abnormal position information based on the road detection information of the posture sensor includes:

[0184] Identifying the number of sensors that generate road detection information for the posture sensor;

[0185] Acquire the sensor type of the posture sensor;

[0186] In response to the number of sensors being not less than the types of sensors, second abnormal position information is determined based on the road detection information of the posture sensor.

[0187] Optionally, the step of comparing the road detection information of the laser radar with the road detection information of the visual sensor to determine the third abnormal position information includes:

[0188] Converting the road detection information of the laser radar into two-dimensional image information;

[0189] Based on the same abnormal position area, comparing the two-dimensional image information with the road detection information of the visual sensor;

[0190] In response to a matching degree between the two-dimensional image information and the road detection information of the visual sensor being greater than a preset matching threshold, third abnormal position information is determined based on the abnormal position area.

[0191] Optionally, the step of determining road abnormality data according to the target fusion information includes:

[0192] When the number of occurrences of the same abnormal position area in the plurality of target fusion information is greater than a preset number threshold, road abnormality data is determined based on the same abnormal position area.

[0193] Optionally, the method further comprises:

[0194] Path planning is performed based on the road abnormality data.

[0195] Optionally, the method further comprises:

[0196] Vehicle navigation is performed based on the road abnormality data.

[0197] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0198] It will be appreciated by those skilled in the art that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0199] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0200] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0201] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0202] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0203] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0204] The above embodiments are only preferred embodiments for fully illustrating the present invention, and the protection scope of the present invention is not limited thereto. Any equivalent substitution or change made by a person skilled in the art based on the present invention is within the protection scope of the present invention.

Claims

1. A road condition warning method, characterized in that: The method comprises: When the vehicle is in a driving state, obtaining road detection information corresponding to multiple sensors in the vehicle; Fusing the road detection information to obtain target fusion information; determining road abnormality data according to the target fusion information; A road condition warning is performed based on the road abnormality data.

2. The method according to claim 1, characterized in that The step of obtaining the road detection information corresponding to the multiple sensors in the vehicle: Obtaining road detection information from a laser radar in the vehicle; and / or, Acquiring road detection information of a posture sensor in the vehicle; and / or, Obtain road detection information from a visual sensor in the vehicle.

3. The method according to claim 2, characterized in that The step of fusing the road detection information to obtain target fusion information comprises: Determining first abnormal position information based on the road detection information of the laser radar; Determining second abnormal position information according to the road detection information of the posture sensor; Comparing the road detection information of the laser radar with the road detection information of the visual sensor to determine third abnormal position information; The first abnormal location information, the second abnormal location information and the third abnormal location information are fused based on location to obtain target fusion information.

4. The method according to claim 3, characterized in that: The step of determining the second abnormal position information based on the road detection information of the posture sensor comprises: Identify the number of sensors that generate road detection information for the posture sensor; Acquire the sensor type of the posture sensor; In response to the number of sensors being not less than the types of sensors, second abnormal position information is determined based on the road detection information of the posture sensor.

5. The method according to claim 3, characterized in that: The step of comparing the road detection information of the laser radar with the road detection information of the visual sensor to determine the third abnormal position information includes: Converting the road detection information of the laser radar into two-dimensional image information; Based on the same abnormal position area, comparing the two-dimensional image information with the road detection information of the visual sensor; In response to a matching degree between the two-dimensional image information and the road detection information of the visual sensor being greater than a preset matching threshold, third abnormal position information is determined based on the abnormal position area.

6. The method according to claim 1, characterized in that The step of determining road abnormality data based on the target fusion information comprises: When the number of occurrences of the same abnormal position area in the plurality of target fusion information is greater than a preset number threshold, road abnormality data is determined based on the same abnormal position area.

7. The method according to claim 1, characterized in that The method further comprises: Path planning is performed based on the road abnormality data.

8. The method according to claim 1, characterized in that: The method further comprises: Vehicle navigation is performed based on the road abnormality data.

9. A road condition warning device, characterized in that: The device comprises: An acquisition module, used for acquiring road detection information corresponding to a plurality of sensors in the vehicle when the vehicle is in a driving state; A fusion module, used for fusing the road detection information to obtain target fusion information; A determination module, used for determining road abnormality data according to the target fusion information; The warning module is used to issue a road condition warning based on the road abnormality data.

10. A vehicle, characterized in that: The method comprises a processor, a memory and a computer program stored in the memory and capable of running on the processor, wherein the computer program, when executed by the processor, implements the steps of the road condition warning method as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the mixed road condition warning method according to any one of claims 1 to 8 are implemented.

Images