Road pothole warning method, device, equipment and storage medium

Abstract

The present invention discloses a method, device, equipment and storage medium for road pothole warning, which relates to the field of automotive intelligent driving, and includes: obtaining road information through the first millimeter-wave radar and the second millimeter-wave radar; when it is determined that there is a pothole on the road according to the road information, obtaining the pothole depth; performing road pothole warning processing according to the pothole depth information and a preset pothole depth threshold. By using two millimeter-wave radars, it is possible to accurately determine whether there is a pothole on the road ahead, and timely alarm the driver, so that the driver has time to select to cut out the automatic driving mode and brake to decelerate.

Description

Technical Field

[0001] The present invention relates to the field of intelligent vehicle driving, and particularly to a method, device, equipment and storage medium for warning of road potholes. Background Art

[0002] With the popularization of intelligent driving, most users often do not pay attention to road conditions when using the automatic driving function. When the vehicle encounters a road pothole, it is difficult for the vehicle to respond effectively in the automatic driving state. When the depth of the pothole is relatively large, the vehicle tire may get stuck in it and scratch the vehicle chassis. Even when the depth of the pothole is not large, the vehicle jitter caused by the new energy vehicle rashly passing through the pothole at the original speed in the automatic driving mode will also make the driver feel a strong sense of jerk, posing a great safety risk. Therefore, when the vehicle is in the automatic driving mode, it is very necessary for the new energy vehicle to automatically identify road potholes in advance and perform vehicle deceleration and pre-warning according to the depth of the potholes.

[0003] However, the current road pothole identification methods have problems such as high cost, difficulty in accurately identifying in a model, and poor real-time performance.

[0004] The above content is only used to assist in understanding the technical solution of the present invention, and does not represent an admission that the above content is prior art. Summary of the Invention

[0005] The main purpose of the present invention is to provide a method, device, equipment and storage medium for warning of road potholes, aiming to solve the technical problem of how to accurately identify potholes.

[0006] To achieve the above object, the present invention provides a method for warning of road potholes, which is applied to a road pothole warning system. The road pothole warning system includes a first millimeter-wave radar and a second millimeter-wave radar. The method for warning of road potholes includes the following steps:

[0007] Obtain road information through the first millimeter-wave radar and the second millimeter-wave radar;

[0008] When it is determined according to the road information that there is a pothole on the road, obtain the depth of the pothole;

[0009] Perform road pothole warning processing according to the pothole depth information and a preset pothole depth threshold.

[0010] In an embodiment, the step of obtaining the depth of the pothole when it is determined according to the road information that there is a pothole on the road includes:

[0011] When it is determined according to the road information that there is a pothole on the road, determine whether the pothole is located on the vehicle driving trajectory;

[0012] When it is determined that the pothole is not located on the vehicle driving trajectory, control the vehicle to decelerate;

[0013] When it is determined that the pothole is on the vehicle driving trajectory, obtain the pothole depth.

[0014] In one embodiment, the step of determining whether the pothole is on the vehicle driving trajectory when it is determined that there is a pothole according to the road information includes:

[0015] Based on the electromagnetic wave emission speed of the millimeter-wave radar, the time from the first millimeter-wave radar emission to return, and the time from the second millimeter-wave radar emission to return, determine the distance between the first millimeter-wave radar and the pothole and the distance between the second millimeter-wave radar and the pothole;

[0016] Obtain the distance between the first millimeter-wave radar and the left front of the vehicle and the distance between the second millimeter-wave radar and the right front of the vehicle;

[0017] Based on the distance between the first millimeter-wave radar and the left front of the vehicle, the distance between the second millimeter-wave radar and the right front of the vehicle, the distance between the first millimeter-wave radar and the pothole, the maximum emission angle of the millimeter-wave radar, and the distance between the second millimeter-wave radar and the pothole, determine the first angle between the line connecting the pothole and the first millimeter-wave radar and the left boundary of the vehicle head and the second angle between the line connecting the pothole and the second millimeter-wave radar and the right boundary of the vehicle head;

[0018] When the first angle is greater than the preset angle threshold and the second angle is greater than the preset angle threshold, determine that the pothole is on the vehicle driving trajectory.

[0019] In one embodiment, the step of obtaining the pothole depth when it is determined that the pothole is on the vehicle driving trajectory includes:

[0020] When it is determined that the pothole is on the vehicle driving trajectory, obtain the pitch angle between the direction of the millimeter-wave radar electromagnetic wave return and the horizontal plane and the height of the millimeter-wave radar from the ground, where the pitch angle is the vertical angle between the direction of the millimeter-wave radar electromagnetic wave return and the horizontal plane;

[0021] Based on the distance between the first millimeter-wave radar and the pothole and the distance between the second millimeter-wave radar and the pothole, determine the distance between the pothole and the vehicle;

[0022] Based on the distance between the pothole and the vehicle, the pitch angle, and the height of the millimeter-wave radar from the ground, obtain the pothole depth and determine the pothole depth

[0023] In one embodiment, the step of performing road pothole warning processing according to the pothole depth and the preset pothole depth threshold includes:

[0024] Determine whether the pothole depth is less than the first preset pothole depth threshold;

[0025] When the pothole depth is less than the first preset pothole depth threshold, control the motor to reduce the speed to control the vehicle to decelerate.

[0026] In one embodiment, after the step of determining whether the pothole depth is less than the first preset pothole depth threshold, the method further includes:

[0027] When the pothole depth is greater than or equal to the first preset pothole depth threshold, determine whether the pothole depth is less than the second preset pothole depth threshold;

[0028] When the pothole depth is less than the second preset pothole depth threshold, give a road pothole warning to prompt the driver, and control the motor to reduce the speed to control the vehicle to decelerate.

[0029] In one embodiment, after the step of determining whether the pothole depth is less than the second preset pothole depth threshold, the method further includes:

[0030] When the pothole depth is greater than or equal to the second preset pothole depth threshold, identify the pothole and determine an avoidance trajectory;

[0031] Control the vehicle to avoid based on the avoidance trajectory.

[0032] In addition, to achieve the above object, the present invention also provides a road pothole warning device, which includes:

[0033] A road information acquisition module, configured to acquire road information through a first millimeter-wave radar and a second millimeter-wave radar;

[0034] A pothole depth determination module, configured to acquire the pothole depth when it is determined that there is a pothole on the road according to the road information;

[0035] A processing module, configured to perform road pothole warning processing according to the pothole depth information and the preset pothole depth threshold.

[0036] In addition, to achieve the above object, the present invention also provides a road pothole warning device, which includes: a memory, a processor, and a road pothole warning program stored on the memory and executable on the processor, where the road pothole warning program is configured to implement the steps of the road pothole warning method as described above.

[0037] In addition, to achieve the above object, the present invention also provides a storage medium, on which a road pothole warning program is stored, and when the road pothole warning program is executed by a processor, the steps of the road pothole warning method as described above are implemented.

[0038] One or more technical solutions proposed in this application have at least the following technical effects:

[0039] Obtain road information through the first millimeter-wave radar and the second millimeter-wave radar; when it is determined that there are potholes on the road according to the road information, obtain the pothole depth; perform road pothole warning processing according to the pothole depth information and the preset pothole depth threshold. By installing two millimeter-wave radars on a new energy vehicle to identify road potholes, it is possible to more accurately judge whether there are potholes on the road ahead, and the estimated pothole depth is also more accurate. The vehicle terminal compares the estimated pothole depth value with the preset value, which can avoid the vehicle terminal from frequently alarming the driver. The selective alarm can remind the driver at critical moments while not disturbing the driver's daily entertainment life. When the estimated pothole depth is greater than the preset threshold, the vehicle terminal sends a message to the CAN bus, and an alarm is realized through the alarm on the CAN bus, allowing the driver to have time to select to cut out the autonomous driving mode and brake to decelerate. Description of the Drawings

[0040] The drawings here are incorporated into the specification and form a part of this specification, showing embodiments consistent with this application, and are used together with the specification to explain the principles of this application.

[0041] In order to more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0042] Figure 1 It is a schematic flowchart provided for the first embodiment of the road pothole warning method of this application;

[0043] Figure 2 It is a diagram of the installation positions of millimeter-wave radars provided for the first embodiment of the road pothole warning method of this application;

[0044] Figure 3 It is a schematic flowchart provided for the second embodiment of the road pothole warning method of this application;

[0045] Figure 4 It is a schematic diagram of the ranging principle of millimeter-wave radars on a new energy vehicle provided for the second embodiment of the road pothole warning method of this application;

[0046] Figure 5 It is a schematic diagram for the second embodiment of the road pothole warning method of this application to determine whether a pothole is on the driving road of a new energy vehicle;

[0047] Figure 6Schematic diagram for estimating the depth of potholes in a new energy vehicle provided in the second embodiment of the road pothole warning method of the present application;

[0048] Figure 7 Flow chart provided in the third embodiment of the road pothole warning method of the present application;

[0049] Figure 8 Schematic diagram of the module structure of the road pothole warning device in the embodiment of the present application;

[0050] Figure 9 Schematic diagram of the device structure of the hardware operating environment involved in the road pothole warning method in the embodiment of the present application.

[0051] The implementation, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. Detailed implementation manners

[0052] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0053] For a better understanding of the technical solutions of the present application, the following will be described in detail with reference to the accompanying drawings of the specification and specific implementation manners.

[0054] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device, a road pothole warning device, etc. that can implement the above functions. The following takes the road pothole warning device as an example to illustrate this embodiment and the following embodiments.

[0055] Based on this, the embodiment of the present application provides a road pothole warning method, referring to Figure 1 , Figure 1 Flow chart of the first embodiment of the road pothole warning method of the present application.

[0056] In this embodiment, the road pothole warning method includes steps S10 to S30:

[0057] Step S10, obtaining road information through the first millimeter-wave radar and the second millimeter-wave radar;

[0058] It should be noted that before obtaining road information, it is necessary to ensure that the vehicle is equipped with Advanced Driver Assistance Systems (ADAS). This system can obtain the surrounding environment at any time during vehicle driving, collect data, identify, detect, and track static and dynamic objects, and combine navigation map data for system operation and analysis, so as to pre-warn the driver of potential dangers and effectively improve the comfort and safety of vehicle driving. In addition, two millimeter-wave radars (i.e., the first millimeter-wave radar and the second millimeter-wave radar) are installed on the front of the vehicle. These two devices are crucial for real-time measurement of the physical environment around the vehicle. Especially for detecting road potholes ahead, they can obtain the distance between the pothole and the vehicle while determining whether the road pothole is on the road where the vehicle is driving. Ensure that all radars and systems are correctly installed and connected to the vehicle's central processing unit.

[0059] As Figure 2 shown, Figure 2 Figure 1 is a diagram of the installation positions of millimeter-wave radars. Among them, d is the distance between the first millimeter-wave radar and the second millimeter-wave radar, d1 is the distance between the first millimeter-wave radar and the left front of the vehicle, and d2 is the distance between the second millimeter-wave radar and the right front of the vehicle. Among them, the left and right millimeter-wave radars on the ADAS can be fabricated separately and then installed, or can be fabricated as a combined module during the manufacturing process. This can make the confirmation of the distance d between the left and right millimeter-wave radars and the distances d1 and d2 between them and the left and right fronts of the vehicle more accurate.

[0060] It should be understood that before starting any driving, it is necessary to check the working status of the left and right millimeter-wave radars. This includes, but is not limited to, checking whether they can successfully transmit and receive electromagnetic wave signals, as well as the intensity and stability of the signals. This is to ensure that during the autonomous driving process, the millimeter-wave radar can accurately obtain information about the road ahead and provide a reliable basis for subsequent data processing.

[0061] It can be understood that this strategy is applied in the autonomous driving mode. In this mode, the vehicle will independently determine key operations such as driving routes and speed adjustments according to the built-in autonomous driving algorithm. At the same time, the system will continuously collect data from the millimeter-wave radar to real-time monitor whether there are potholes ahead. The road information includes whether there are potholes on the road ahead.

[0062] Furthermore, there may be scenarios where the millimeter-wave radar adopted in this strategy can be replaced by an ultrasonic radar. However, using ultrasonic waves of the ultrasonic radar for pothole judgment will result in poor real-time performance due to the distance limitation of the ultrasonic radar itself, and there will be a low accuracy situation when the vehicle is driving at high speed. However, compared with the millimeter-wave radar, the ultrasonic radar is cheaper and the overall vehicle cost is smaller.

[0063] Step S20, when it is determined according to the road information that there is a pothole on the road, obtain the depth of the pothole;

[0064] It should be noted that when the vehicle encounters a pothole ahead during the automatic driving process, the two millimeter-wave radars installed on the vehicle head will respectively emit electromagnetic waves to the ground.

[0065] In a specific implementation, the millimeter-wave radar determines the depth of the pothole by receiving the electromagnetic waves returned from the pothole.

[0066] Step S30, perform road pothole warning processing according to the pothole depth information and a preset pothole depth threshold;

[0067] It should be noted that the preset pothole depth threshold can be set, such as 20 cm, 50 cm. The calculated pothole depth will be compared with the preset safety threshold of 50 cm of the system. If the depth is less than 50 cm, it is considered that the pothole has little impact on the vehicle driving and no special treatment is required; on the contrary, if the depth is greater than 50 cm, it is considered that there is a high driving risk, and the system will send an alarm message to the driver through the in-vehicle network (CAN bus) and remind the driver to pay attention to the pothole ahead through the dashboard or the sound and light alarm device.

[0068] This embodiment provides a road pothole warning method. The road information is obtained through the first millimeter-wave radar and the second millimeter-wave radar; when it is determined according to the road information that there is a pothole on the road, obtain the depth of the pothole; perform road pothole warning processing according to the pothole depth information and a preset pothole depth threshold. Through the two millimeter-wave radars, it can accurately judge whether there is a pothole on the road ahead, alarm the driver in time, and let the driver have time to select to cut out the automatic driving mode and brake to decelerate.

[0069] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar content as the above-mentioned embodiment one can be referred to the above introduction and will not be repeated hereinafter. On this basis, please refer to Figure 3 , step S20 includes steps S201~S203:

[0070] Step S201, when it is determined according to the road information that there is a pothole on the road, determine whether the pothole is on the vehicle driving trajectory;

[0071] It should be noted that as Figure 4 shown, Figure 4 is the millimeter-wave radar ranging principle diagram of a new energy vehicle. First, the FWCM signal proposed is emitted to the pothole through the millimeter-wave, and then the returned signal is received through the wireless receiving device, and then the distance rate processing is performed through the first matching filter, the first low-pass filter, and the first frequency analysis device to complete the determination of the pothole distance.

[0072] It should be understood that when the vehicle encounters a pothole ahead during autonomous driving, the two millimeter-wave radars on the left and right installed on the vehicle head will respectively emit electromagnetic waves towards the ground.

[0073] First, the millimeter-wave radar will determine whether this pothole is on the vehicle's driving road rather than on both sides of the vehicle's driving road.

[0074] In a feasible implementation manner, step S201 includes steps A11 to A14:

[0075] Step A11: Based on the electromagnetic wave emission speed of the millimeter-wave radar, the time from the first millimeter-wave radar's emission to return, and the time from the second millimeter-wave radar's emission to return, determine the distance between the first millimeter-wave radar and the pothole and the distance between the second millimeter-wave radar and the pothole;

[0076] In a specific implementation, first obtain the time from the left millimeter-wave radar's electromagnetic wave emission to return, denoted as , and the time from the right millimeter-wave radar's electromagnetic wave emission to return, denoted as . It is known that the electromagnetic wave emission speed of the millimeter-wave radar is . Then, through the mathematical formula:

[0077]

[0078]

[0079] Obtain the distance between the left millimeter-wave radar and the pothole and the distance between the right millimeter-wave radar and the pothole.

[0080] It should be noted that the left millimeter-wave radar is the first millimeter-wave radar, and the right millimeter-wave radar is the second millimeter-wave radar.

[0081] Step A12: Obtain the distance between the first millimeter-wave radar and the left vehicle head and the distance between the second millimeter-wave radar and the right vehicle head;

[0082] In a specific implementation, obtain the distance between the left millimeter-wave radar and the left edge of the vehicle head as , and the distance between the right millimeter-wave radar and the left edge of the vehicle head as .

[0083] Step A13: Determine a first angle between the line connecting the pothole and the first millimeter-wave radar and the left boundary of the vehicle head, and a second angle between the line connecting the pothole and the second millimeter-wave radar and the right boundary of the vehicle head, based on the distance between the first millimeter-wave radar and the left vehicle head, the distance between the second millimeter-wave radar and the right vehicle head, the distance between the first millimeter-wave radar and the pothole, the maximum transmission angle of the millimeter-wave radar, and the distance between the second millimeter-wave radar and the pothole;

[0084] In a specific implementation, through the mathematical formula:

[0085]

[0086]

[0087] Transform to obtain:

[0088]

[0089] Obtain the angle between the pothole and the vertical cross-section on the left side of the vehicle head and the angle between the pothole and the vertical cross-section on the right side of the vehicle head .

[0090] Step A14: When the first angle is greater than a preset angle threshold and the second angle is greater than the preset angle threshold, determine that the pothole is on the vehicle driving trajectory.

[0091] In a specific implementation, if and , then it is considered that the pothole is on the vehicle driving road, and in other cases, it is considered that the pothole is on both sides of the vehicle driving road.

[0092] It can be understood that a vertical cross-section is drawn with the vehicle head, and then a horizontal line is drawn from the left and right millimeter-wave radars respectively. The first angle and the second angle are compared with 90° - θ / 2 respectively. If both are greater, it means that the pothole is in the deep orange area in the center of the figure

[0093] As Figure 5 shown, Figure 5 is a schematic diagram for a new energy vehicle to determine whether a pothole is on the driving road. Among them, is the transmission angle of the millimeter-wave radar, is the distance between the pothole and the vehicle, α is the first angle, β is the second angle. If the pothole is within the transmission angles of the left and right millimeter-wave radars, it is determined that the pothole is on the driving road. If the pothole is within the transmission angle of the left or right millimeter-wave radar, it is determined that the pothole is on both sides of the driving road.

[0094] Step S202, when it is determined that the pothole is not on the vehicle's driving trajectory, control the vehicle to decelerate;

[0095] It should be noted that at this time, only control the vehicle to pass through the area with the pothole at a relatively stable vehicle speed, and there is no need for sudden deceleration.

[0096] Step S203, when it is determined that the pothole is on the vehicle's driving trajectory, obtain the depth of the pothole.

[0097] In a specific implementation, after determining that the pothole is on the vehicle's driving road, calculate the depth of the pothole through the distance and angle data obtained by the millimeter-wave radar.

[0098] In a feasible implementation manner, step S203 includes steps A21 to A23:

[0099] Step A21: When it is determined that the pothole is on the vehicle's driving trajectory, obtain the elevation angle between the direction of the millimeter-wave radar electromagnetic wave returning and the horizontal plane, and the height between the millimeter-wave radar and the ground, where the elevation angle is the vertical angle between the direction of the millimeter-wave radar electromagnetic wave returning and the horizontal plane;

[0100] It should be noted that the distance between the pothole and the vehicle is , and the angle between the direction of the millimeter-wave radar electromagnetic wave and the vertical horizontal plane is ; the height between the millimeter-wave radar and the ground is .

[0101] The millimeter-wave radar emits electromagnetic waves, and when the electromagnetic waves encounter obstacles such as potholes, they will be reflected back. There is an angle between the direction of this returned electromagnetic wave and the horizontal plane, that is, the elevation angle.

[0102] It should be understood that the millimeter-wave radar is installed on the vehicle, and its height from the ground is a relatively fixed parameter, and the approximate height between the millimeter-wave radar and the ground can be obtained through the vehicle's height sensor. For example, install a height sensor near the vehicle's suspension system to monitor the distance between the vehicle body and the ground in real time, so as to indirectly obtain the height between the millimeter-wave radar and the ground.

[0103] Step A22: Determine the distance between the pothole and the vehicle based on the distance between the first millimeter-wave radar and the pothole and the distance between the second millimeter-wave radar and the pothole;

[0104] In a specific implementation, the method of taking the median value can be used to determine the distance between the pothole and the vehicle. This is because the two millimeter-wave radars installed on the vehicle are generally symmetrically distributed, and the distances from them to the pothole can, to a certain extent, reflect the relative position between the vehicle and the pothole. Specifically, it is to take the median value of the distance between the first millimeter-wave radar and the pothole and the distance between the second millimeter-wave radar and the pothole as the distance between the pothole and the vehicle. For example, if the distance from the first millimeter-wave radar to the pothole is d1 and the distance from the second millimeter-wave radar to the pothole is d2, then the distance d between the pothole and the vehicle can be expressed as d= , and a relatively reasonable estimated value of the distance between the vehicle and the pothole can be obtained quickly.

[0105] Step A23: Obtain the pothole depth based on the distance between the pothole and the vehicle, the angle between the direction of the millimeter-wave radar electromagnetic wave and the vertical horizontal plane, and the height of the millimeter-wave radar from the ground, and determine the pothole depth.

[0106] It should be noted that since the vehicle is moving, the depth of this pothole obtained each time will naturally be different. However, during driving, this value will definitely tend to be stable (it can be understood that at the beginning, it may not hit the deepest part of the pothole, but as the vehicle gets closer to the pothole during driving, it will definitely hit the deepest part of the pothole). As long as the obtained pothole depth is greater than the preset value, even if it does not hit the deepest part, it can naturally be judged that this pothole is relatively deep and requires deceleration and other treatments.

[0107] It should be understood that after calculating the pothole depth, the calculation result can be verified and confirmed. This may involve further calibration of the millimeter-wave radar data, considering the influence of surrounding environmental factors on the measurement result, and comparing the data of other sensors to comprehensively determine the accuracy of the pothole depth, so as to finally determine the pothole depth and provide a basis for the vehicle's driving safety decision-making.

[0108] As Figure 6 shown, Figure 6 is a schematic diagram for estimating the pothole depth of a new energy vehicle. Among them, is a point where the electromagnetic wave emitted by the vehicle's millimeter-wave radar hits the pothole, is the angle between the direction of this beam of electromagnetic wave and the vertical horizontal plane, is the height of the millimeter-wave radar from the ground, is the longitudinal distance between the millimeter-wave radar and the point inside the pothole, is the distance between the vehicle and the point inside the pothole.

[0109] In a specific implementation, it is known that the horizontal width of the pothole can be ignored compared with the distance between the vehicle and the pothole. It is known that the vehicle and the point where it hits the pothole The distance is , and the angle between the direction of this electromagnetic wave and the horizontal plane perpendicular to the ground is . Given that the height of the millimeter-wave radar from the ground is .

[0110] Then through the mathematical formula:

[0111]

[0112]

[0113] The estimated depth of the pit at this time is obtained .

[0114] It should be noted that as the vehicle moves forward continuously, the estimated value of also changes continuously. When the value of tends to be stable after a period of time, the final estimated value of the pit depth is obtained

[0115] It can be assumed that a new energy vehicle is equipped with an ADAS system and two millimeter-wave radars are installed on the left and right. When driving in the autonomous driving mode, a pit is encountered ahead. The time from the emission to the return of the electromagnetic waves of the left and right millimeter-wave radars is 10 microseconds. Through calculation:

[0116] The distance between the pit and the vehicle is: 5 microseconds × the electromagnetic wave emission speed (the electromagnetic wave emission speed is about 8 m / s) = 150 m.

[0117] Given that the interval between the left and right millimeter-wave radars is 1 m, and the distances from the left and right millimeter-wave radars to the left and right vehicle heads , are both 30 cm, and the maximum emission angles of the left and right millimeter-wave radars are both 70°.

[0118] From the mathematical formula it is obtained that is about 50°, which is less than , Similarly, it is thus judged that the pit is on the vehicle driving road.

[0119] The height of the millimeter-wave radar from the ground is 80 cm, and the angle between the direction of the electromagnetic wave and the horizontal plane perpendicular to the ground is 88°. When the estimated depth tends to be stable, the distance between the vehicle and the pit is 50 m.

[0120] Then the final estimated value of the depth is obtained as .

[0121] The vehicle end compares it with the pre-set safety threshold of 0.5 m of the system, and gets the conclusion that the depth is greater than 0.5 m. It is considered that there is a high driving risk for autonomous driving at this time. The system sends an alarm message to the driver through the CAN bus, and reminds the driver to pay attention to the pothole ahead through the dashboard or the audible and visual alarm device. After that, the system will also send an instruction to the motor control system of the vehicle through the CAN bus to appropriately reduce the motor speed and realize the vehicle's self-deceleration.

[0122] This embodiment provides a method for warning of road potholes. Two millimeter-wave radars are installed on a new energy vehicle to identify road potholes. When it is estimated that the depth of the pothole is greater than the preset threshold, the vehicle end sends a message to the CAN bus, and the alarm on the CAN bus is used to give an alarm. There is no need to carry an additional lidar. Only two millimeter-wave radars need to be additionally installed on the basis of the original ADAS. The electromagnetic wave emitted by the millimeter-wave radar is less affected by the environment, and its vertical field of view is large. By using the method of installing two millimeter-wave radars, irrelevant road data on both sides of the vehicle can be effectively screened out, which is convenient for data modeling. The millimeter-wave radar has a very short transceiver time and strong real-time performance.

[0123] Based on the first and second embodiments of the present application, in the third embodiment of the present application, the same or similar content as the above embodiments can be referred to the above introduction and will not be repeated hereinafter. On this basis, please refer to Figure 7 , step S30 includes steps S301 to S306:

[0124] Step S301, determining whether the depth of the pothole is less than the first preset pothole depth threshold;

[0125] It should be noted that the first preset pothole depth threshold can be set, such as 20 cm;

[0126] Step S302, when the depth of the pothole is less than the first preset pothole depth threshold, controlling the motor to reduce the speed to control the vehicle to decelerate;

[0127] In a specific implementation, when the depth of the pothole is less than 20 cm, the pothole information does not pass through the alarm at this time, and the message is transmitted to the motor through the CAN bus, and the command to reduce the motor speed is issued to achieve the purpose of deceleration.

[0128] Step S303, when the depth of the pothole is greater than or equal to the first preset pothole depth threshold, determining whether the depth of the pothole is less than the second preset pothole depth threshold;

[0129] It should be noted that the second preset pothole depth can be set, such as 50 cm. At this time, the second preset pothole depth can also be called the depth danger value.

[0130] Step S304, when the pothole depth is less than the second preset pothole depth threshold, give a road pothole warning to prompt the driver, and control the motor to reduce the speed to control the vehicle to decelerate;

[0131] It should be noted that when the pothole depth is less than 50 cm and greater than 20 cm, the pothole information will pass through the alarm, and the alarm will handle the alarm, which is reflected to the vehicle end to remind the driver, and a message will be sent to the motor through the CAN bus to issue an order to reduce the motor speed, so as to achieve the purpose of deceleration.

[0132] Step S305, when the pothole depth is greater than or equal to the second preset pothole depth threshold, identify the pothole and determine the avoidance trajectory;

[0133] It should be noted that when the pothole depth is greater than 50 cm, the pothole can be identified through the high-precision map at this time, and an avoidance instruction is sent to the vehicle computer, and a reasonable avoidance trajectory is specified.

[0134] Step S306, control the vehicle to avoid based on the avoidance trajectory;

[0135] In specific implementation, that is, control the vehicle to avoid the pothole along the avoidance trajectory through the autonomous driving mode to ensure the safety of the vehicle and the driver.

[0136] This embodiment provides a road pothole warning method. By comparing the estimated pothole depth value with the preset value at the vehicle end, it is possible to avoid the vehicle end from frequently alarming the driver. The screened alarm can remind the driver at a critical moment while not disturbing the driver's daily entertainment life; realizing the vehicle deceleration function by sending a message through the CAN bus can avoid the problem that the driver fails to switch the mode in time to brake and decelerate by himself after receiving the alarm, reducing potential safety hazards.

[0137] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the road pothole warning method of this application. Based on this technical concept, more forms of simple transformations are within the protection scope of this application.

[0138] This application also provides a road pothole warning device. Please refer to Figure 8 , the road pothole warning device includes:

[0139] The road information acquisition module 10 is used to acquire road information through the first millimeter-wave radar and the second millimeter-wave radar;

[0140] The pothole depth determination module 20 is used to obtain the pothole depth when it is determined that there is a pothole on the road according to the road information;

[0141] The processing module 30 is used to perform road pothole warning processing according to the pothole depth information and the preset pothole depth threshold.

[0142] The road pothole warning device provided by this application adopts the road pothole warning method in the above-mentioned embodiment, and can solve the technical problem of how to accurately identify potholes. Compared with the prior art, the beneficial effects of the road pothole warning device provided by this application are the same as those of the road pothole warning method provided by the above-mentioned embodiment, and other technical features in the road pothole warning device are the same as those disclosed in the above-mentioned embodiment method, which will not be elaborated here.

[0143] In one embodiment, when the pothole depth determination module 20 is further configured to determine that there is a pothole on the road according to road information, it is also used to determine whether the pothole is on the vehicle driving trajectory;

[0144] When it is determined that the pothole is not on the vehicle driving trajectory, the vehicle is controlled to decelerate;

[0145] When it is determined that the pothole is on the vehicle driving trajectory, the pothole depth is obtained.

[0146] In one embodiment, the pothole depth determination module 20 is further configured to determine the distance between the first millimeter-wave radar and the pothole and the distance between the second millimeter-wave radar and the pothole based on the electromagnetic wave emission speed of the millimeter-wave radar, the time from the first millimeter-wave radar emission to return, and the time from the second millimeter-wave radar emission to return;

[0147] Obtain the distance between the first millimeter-wave radar and the left front of the vehicle and the distance between the second millimeter-wave radar and the right front of the vehicle;

[0148] Based on the distance between the first millimeter-wave radar and the left front of the vehicle, the distance between the second millimeter-wave radar and the right front of the vehicle, the distance between the first millimeter-wave radar and the pothole, the maximum emission angle of the millimeter-wave radar, and the distance between the second millimeter-wave radar and the pothole, determine the first angle between the connection line of the pothole and the first millimeter-wave radar and the left boundary of the vehicle head and the second angle between the connection line of the pothole and the second millimeter-wave radar and the right boundary of the vehicle head;

[0149] When the first angle is greater than the preset angle threshold and the second angle is greater than the preset angle threshold, it is determined that the pothole is on the vehicle driving trajectory.

[0150] In one embodiment, when the pothole depth determination module 20 is further configured to determine that the pothole is on the vehicle driving trajectory, it is also used to obtain the pitch angle between the direction of the millimeter-wave radar electromagnetic wave return and the horizontal plane and the height of the millimeter-wave radar from the ground, where the pitch angle is the vertical angle between the direction of the millimeter-wave radar electromagnetic wave return and the horizontal plane;

[0151] Determine the distance between the pothole and the vehicle based on the distance between the first millimeter-wave radar and the pothole and the distance between the second millimeter-wave radar and the pothole;

[0152] Obtain the pothole depth based on the distance between the pothole and the vehicle, the pitch angle, and the height of the millimeter-wave radar from the ground, and determine the pothole depth

[0153] In one embodiment, the processing module 30 is further configured to determine whether the pothole depth is less than a first preset pothole depth threshold;

[0154] When the pothole depth is less than the first preset pothole depth threshold, control the motor to reduce the speed to control the vehicle to decelerate.

[0155] In one embodiment, the processing module 30 is further configured to determine whether the pothole depth is less than a second preset pothole depth threshold when the pothole depth is greater than or equal to the first preset pothole depth threshold;

[0156] When the pothole depth is less than the second preset pothole depth threshold, give a road pothole warning to prompt the driver, and control the motor to reduce the speed to control the vehicle to decelerate.

[0157] In one embodiment, the processing module 30 is further configured to identify the pothole and determine an avoidance trajectory when the pothole depth is greater than or equal to the second preset pothole depth threshold;

[0158] Control the vehicle to avoid based on the avoidance trajectory.

[0159] This application provides a road pothole warning device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the road pothole warning method in the first embodiment above.

[0160] Next, refer to Figure 9 , which shows a schematic structural diagram of a road pothole warning device suitable for implementing the embodiments of the present application. The road pothole warning device in the embodiments of the present application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistant), PADs (Portable Application Description), PMPs (Portable Media Player), vehicle terminals (such as vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 9The shown road pothole warning device is merely an example and should not impose any limitations on the functions and application scope of the embodiments of this application.

[0161] As Figure 9 shown, the road pothole warning device may include a processing device 1001 (such as a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM: Read Only Memory) 1002 or the program loaded from the storage device 1003 into the random access memory (RAM: Random Access Memory) 1004. In the RAM 1004, various programs and data required for the operation of the road pothole warning device are also stored. The processing device 1001, the ROM 1002, and the RAM 1004 are connected to each other through a bus 1005. The input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems may be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the road pothole warning device to communicate with other devices wirelessly or wirelessly to exchange data. Although the figure shows a road pothole warning device with various systems, it should be understood that it is not required to implement or include all the shown systems. More or fewer systems may be implemented or included alternatively.

[0162] In particular, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in this application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program codes for executing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device, or installed from the storage device 1003, or installed from the ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the methods of the embodiments disclosed in this application are executed.

[0163] The road pothole warning device provided by the present application adopts the road pothole warning method in the above-mentioned embodiment, which can solve the technical problem of how to accurately identify potholes. Compared with the prior art, the beneficial effects of the road pothole warning device provided by the present application are the same as those of the road pothole warning method provided by the above-mentioned embodiment, and other technical features in the road pothole warning device are the same as the features disclosed in the method of the previous embodiment, which will not be elaborated here.

[0164] It should be understood that each part disclosed in the present application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in a suitable manner in any one or more embodiments or examples.

[0165] The above is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present application, and all should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

[0166] The present application provides a computer-readable storage medium having computer-readable program instructions (i.e., computer programs) stored thereon, and the computer-readable program instructions are used to execute the road pothole warning method in the above-mentioned embodiment.

[0167] The computer-readable storage medium provided by the present application can be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM) or flash memory, optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this embodiment, the computer-readable storage medium can be any tangible medium that contains or stores a program, and the program can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium can be transmitted by any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0168] The above computer-readable storage medium may be included in the road pothole warning device; or it may exist independently without being assembled into the road pothole warning device.

[0169] The above computer-readable storage medium carries one or more programs. When the one or more programs are executed by the road pothole warning device, the road pothole warning device is caused to: obtain road information through the first millimeter-wave radar and the second millimeter-wave radar; when it is determined according to the road information that there is a pothole on the road, obtain the pothole depth; and perform road pothole warning processing according to the pothole depth information and a preset pothole depth threshold.

[0170] Computer program code for performing the operations of the present application may be written in one or more programming languages or combinations thereof. The programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, executed as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (for example, by connecting through an Internet service provider using the Internet).

[0171] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, may be implemented by a dedicated hardware-based system for performing the specified functions or operations, or may be implemented by a combination of dedicated hardware and computer instructions.

[0172] The modules involved in the embodiments of the present application can be implemented in software or in hardware. In some cases, the name of the module does not constitute a limitation on the unit itself.

[0173] The readable storage medium provided by the present application is a computer-readable storage medium. The computer-readable storage medium stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned road pothole warning method, and can solve the technical problem of how to accurately identify potholes. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as those of the road pothole warning method provided by the above embodiments, and will not be elaborated here.

[0174] The present application also provides a computer program product, including a computer program, and the steps of the above-mentioned road pothole warning method are implemented when the computer program is executed by a processor.

[0175] The computer program product provided by the present application can solve the technical problem of how to accurately identify potholes. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as those of the road pothole warning method provided by the above embodiments, and will not be elaborated here.

[0176] The above are only some embodiments of the present application, and do not limit the patent scope of the present application. Any equivalent structural transformation made by using the content of the specification and drawings of the present application under the technical concept of the present application, or direct / indirect application in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A method for warning of road potholes, characterized in that, Applied to a road pothole warning system, the road pothole warning system includes a first millimeter-wave radar and a second millimeter-wave radar, and the road pothole warning method includes the following steps: Obtain road information through the first millimeter-wave radar and the second millimeter-wave radar; When it is determined that there is a pothole on the road according to the road information, obtain the pothole depth; Perform road pothole warning processing according to the pothole depth information and a preset pothole depth threshold; The step of obtaining the pothole depth when it is determined that there is a pothole on the road according to the road information includes: When it is determined that there is a pothole on the road according to the road information, determine whether the pothole is on the vehicle driving trajectory; When it is determined that the pothole is not on the vehicle driving trajectory, control the vehicle to decelerate; When it is determined that the pothole is on the vehicle driving trajectory, obtain the pothole depth; The step of determining whether the pothole is on the vehicle driving trajectory when it is determined that there is a pothole on the road according to the road information includes: Based on the electromagnetic wave emission speed of the millimeter-wave radar, the time from the first millimeter-wave radar to return, and the time from the second millimeter-wave radar to return, determine the distance between the first millimeter-wave radar and the pothole and the distance between the second millimeter-wave radar and the pothole; Obtain the distance between the first millimeter-wave radar and the left front of the vehicle and the distance between the second millimeter-wave radar and the right front of the vehicle; Based on the distance between the first millimeter-wave radar and the left front of the vehicle, the distance between the second millimeter-wave radar and the right front of the vehicle, the distance between the first millimeter-wave radar and the pothole, the maximum emission angle of the millimeter-wave radar, and the distance between the second millimeter-wave radar and the pothole, determine the first angle between the line connecting the pothole and the first millimeter-wave radar and the left boundary of the vehicle head and the second angle between the line connecting the pothole and the second millimeter-wave radar and the right boundary of the vehicle head; When the first angle is greater than a preset angle threshold and the second angle is greater than the preset angle threshold, determine that the pothole is on the vehicle driving trajectory.

2. The road pothole warning method according to claim 1, wherein, The step of obtaining the pothole depth when it is determined that the pothole is on the vehicle driving trajectory includes: When it is determined that the pothole is on the vehicle driving trajectory, obtain the elevation angle between the direction of the millimeter-wave radar electromagnetic wave returning and the horizontal plane and the height of the millimeter-wave radar from the ground, where the elevation angle is the vertical angle between the direction of the millimeter-wave radar electromagnetic wave returning and the horizontal plane; Based on the distance between the first millimeter-wave radar and the pothole and the distance between the second millimeter-wave radar and the pothole, determine the distance between the pothole and the vehicle; Based on the distance between the pothole and the vehicle, the elevation angle, and the height of the millimeter-wave radar from the ground, obtain the pothole depth and determine the pothole depth.

3. The road pothole warning method according to claim 1, wherein, The step of performing road pothole warning processing according to the pothole depth and a preset pothole depth threshold includes: Determine whether the pothole depth is less than a first preset pothole depth threshold; When the pothole depth is less than the first preset pothole depth threshold, control the motor to reduce the speed to control the vehicle to decelerate.

4. The road pothole warning method according to claim 3, characterized in that After the step of determining whether the pothole depth is less than the first preset pothole depth threshold, the following steps are further included: When the pothole depth is greater than or equal to the first preset pothole depth threshold, determine whether the pothole depth is less than the second preset pothole depth threshold; When the pothole depth is less than the second preset pothole depth threshold, give a road pothole warning to prompt the driver, and control the motor to reduce the speed to control the vehicle to decelerate.

5. The road pothole warning method according to claim 4, characterized in that, After the step of determining whether the pothole depth is less than the second preset pothole depth threshold, the following steps are further included: When the pothole depth is greater than or equal to the second preset pothole depth threshold, identify the pothole and determine an avoidance trajectory; Control the vehicle to avoid based on the avoidance trajectory.

6. A road pothole warning device, characterized in that, The device includes: A road information acquisition module, configured to acquire road information through a first millimeter-wave radar and a second millimeter-wave radar; A pothole depth determination module, configured to acquire the pothole depth when it is determined that there is a pothole on the road according to the road information; The pothole depth determination module is further configured to determine whether the pothole is located on the vehicle driving trajectory when it is determined that there is a pothole on the road according to the road information; when it is determined that the pothole is not located on the vehicle driving trajectory, control the vehicle to decelerate; when it is determined that the pothole is located on the vehicle driving trajectory, acquire the pothole depth; The pothole depth determination module is further configured to determine the distance between the first millimeter-wave radar and the pothole and the distance between the second millimeter-wave radar and the pothole based on the electromagnetic wave emission speed of the millimeter-wave radar, the time from the first millimeter-wave radar's emission to return, and the time from the second millimeter-wave radar's emission to return; acquire the distance between the first millimeter-wave radar and the left front of the vehicle and the distance between the second millimeter-wave radar and the right front of the vehicle; based on the distance between the first millimeter-wave radar and the left front of the vehicle, the distance between the second millimeter-wave radar and the right front of the vehicle, the distance between the first millimeter-wave radar and the pothole, the maximum emission angle of the millimeter-wave radar, and the distance between the second millimeter-wave radar and the pothole, determine a first angle between the line connecting the pothole and the first millimeter-wave radar and the left boundary of the vehicle head and a second angle between the line connecting the pothole and the second millimeter-wave radar and the right boundary of the vehicle head; when the first angle is greater than a preset angle threshold and the second angle is greater than the preset angle threshold, determine that the pothole is located on the vehicle driving trajectory; A processing module, configured to perform road pothole warning processing according to the pothole depth information and the preset pothole depth threshold.

7. A road pothole warning device, characterized in that, The device includes: a memory, a processor, and a road pothole warning program stored on the memory and executable on the processor, and the road pothole warning program is configured to implement the steps of the road pothole warning method as described in any one of claims 1 to 5.

8. A storage medium, characterized in that, A road pothole warning program is stored on the storage medium, and when the road pothole warning program is executed by the processor, it implements the steps of the road pothole warning method as described in any one of claims 1 to 5.

Images