Road pit slot warning method, device and equipment and storage medium

Abstract

The invention discloses a road pit slot warning method, device and equipment and a storage medium, and relates to the field of automobile intelligent driving, and the method comprises the steps: obtaining road information through a first millimeter wave radar and a second millimeter wave radar; when it is determined that a pit slot exists in the road according to the road information, the pit slot depth is obtained; road pit slot alarm processing is carried out according to the pit slot depth information and a preset pit slot depth threshold value, whether a pit slot exists in the front road or not can be accurately judged through the two millimeter wave radars, an alarm is given to a driver in time, and the driver has time to choose to switch out an automatic driving mode for braking and deceleration.

Description

Technical Field

[0001] The present invention relates to the field of intelligent vehicle driving, and particularly to a method, device, equipment and storage medium for warning of road potholes. Background Art

[0002] With the popularization of intelligent driving, most users often do not pay attention to road conditions when using the automatic driving function. When the vehicle encounters a road pothole, it is difficult for the vehicle to respond effectively in the automatic driving state. When the depth of the pothole is relatively large, the vehicle tire may get stuck in it and scrape the vehicle chassis. Even when the depth of the pothole is not large, the vehicle jitter caused by the new energy vehicle rashly passing through the pothole at the original speed in the automatic driving mode will also make the driver feel a strong sense of jerk, which has a great safety risk. Therefore, when the vehicle is in the automatic driving mode, it is very necessary for the new energy vehicle to automatically identify road potholes in advance and perform vehicle deceleration and pre-warning according to the depth of the pothole.

[0003] However, the current road pothole recognition methods have problems such as high cost, difficulty in accurately identifying through modeling, and poor real-time performance.

[0004] The above content is only used to assist in understanding the technical solution of the present invention, and does not represent an admission that the above content is prior art. Summary of the Invention

[0005] The main object of the present invention is to provide a method, device, equipment and storage medium for warning of road potholes, aiming to solve the technical problem of how to accurately identify potholes.

[0006] To achieve the above object, the present invention provides a method for warning of road potholes, which is applied to a road pothole warning system. The road pothole warning system includes a first millimeter-wave radar and a second millimeter-wave radar. The method for warning of road potholes includes the following steps: Obtain road information through the first millimeter-wave radar and the second millimeter-wave radar; When it is determined according to the road information that there is a pothole on the road, obtain the depth of the pothole; Perform road pothole warning processing according to the pothole depth information and a preset pothole depth threshold.

[0007] In an embodiment, the step of obtaining the depth of the pothole when it is determined according to the road information that there is a pothole on the road includes: When it is determined according to the road information that there is a pothole on the road, determine whether the pothole is located on the vehicle driving trajectory; When it is determined that the pothole is not located on the vehicle driving trajectory, control the vehicle to decelerate; When it is determined that the pothole is located on the vehicle driving trajectory, obtain the depth of the pothole.

[0008] In one embodiment, when it is determined that there is a pothole on the road according to the road information, the step of determining whether the pothole is located on the vehicle driving trajectory includes: Based on the electromagnetic wave emission speed of the millimeter-wave radar, the time from the first millimeter-wave radar transmitting to returning, and the time from the second millimeter-wave radar transmitting to returning, determine the distance between the first millimeter-wave radar and the pothole and the distance between the second millimeter-wave radar and the pothole; Obtain the distance between the first millimeter-wave radar and the left front of the vehicle and the distance between the second millimeter-wave radar and the right front of the vehicle; Based on the distance between the first millimeter-wave radar and the left front of the vehicle, the distance between the second millimeter-wave radar and the right front of the vehicle, the distance between the first millimeter-wave radar and the pothole, the maximum emission angle of the millimeter-wave radar, and the distance between the second millimeter-wave radar and the pothole, determine the first angle between the line connecting the pothole and the first millimeter-wave radar and the left boundary of the vehicle head and the second angle between the line connecting the pothole and the second millimeter-wave radar and the right boundary of the vehicle head; When the first angle is greater than the preset angle threshold and the second angle is greater than the preset angle threshold, determine that the pothole is located on the vehicle driving trajectory.

[0009] In one embodiment, when it is determined that the pothole is located on the vehicle driving trajectory, the step of obtaining the pothole depth includes: When it is determined that the pothole is located on the vehicle driving trajectory, obtain the pitch angle between the direction of the millimeter-wave radar electromagnetic wave returning and the horizontal plane and the height of the millimeter-wave radar from the ground, where the pitch angle is the vertical angle between the direction of the millimeter-wave radar electromagnetic wave returning and the horizontal plane; Based on the distance between the first millimeter-wave radar and the pothole and the distance between the second millimeter-wave radar and the pothole, determine the distance between the pothole and the vehicle; Based on the distance between the pothole and the vehicle, the pitch angle, and the height of the millimeter-wave radar from the ground, obtain the pothole depth and determine the pothole depth In one embodiment, the step of performing road pothole warning processing according to the pothole depth and the preset pothole depth threshold includes: Determine whether the pothole depth is less than the first preset pothole depth threshold; When the pothole depth is less than the first preset pothole depth threshold, control the motor to reduce the speed to control the vehicle to decelerate.

[0010] In one embodiment, after the step of determining whether the pothole depth is less than the first preset pothole depth threshold, it further includes: When the pothole depth is greater than or equal to the first preset pothole depth threshold, determine whether the pothole depth is less than the second preset pothole depth threshold; When the pothole depth is less than the second preset pothole depth threshold, give a road pothole warning to prompt the driver, and control the motor to reduce the speed to control the vehicle to decelerate.

[0011] In one embodiment, after the step of determining whether the pothole depth is less than the second preset pothole depth threshold, the method further includes: When the pothole depth is greater than or equal to the second preset pothole depth threshold, identify the pothole and determine an avoidance trajectory; Control the vehicle to avoid based on the avoidance trajectory.

[0012] In addition, to achieve the above object, the present invention also provides a road pothole warning device, the device includes: A road information acquisition module, configured to acquire road information through a first millimeter wave radar and a second millimeter wave radar; A pothole depth determination module, configured to acquire the pothole depth when it is determined that there is a pothole on the road according to the road information; A processing module, configured to perform road pothole warning processing according to the pothole depth information and the preset pothole depth threshold.

[0013] In addition, to achieve the above object, the present invention also provides a road pothole warning device, the device includes: a memory, a processor, and a road pothole warning program stored on the memory and executable on the processor, the road pothole warning program is configured to implement the steps of the road pothole warning method as described above.

[0014] In addition, to achieve the above object, the present invention also provides a storage medium, on which a road pothole warning program is stored, and when the road pothole warning program is executed by a processor, the steps of the road pothole warning method as described above are implemented.

[0015] One or more technical solutions proposed in this application have at least the following technical effects: Obtain road information through the first millimeter-wave radar and the second millimeter-wave radar; when it is determined that there is a pothole on the road according to the road information, obtain the pothole depth; perform road pothole warning processing according to the pothole depth information and a preset pothole depth threshold. By installing two millimeter-wave radars on a new energy vehicle to identify road potholes, it is possible to more accurately determine whether there are potholes on the road ahead, and the estimated pothole depth is also more accurate. The vehicle end compares the estimated pothole depth value with a preset value, which can avoid the vehicle end from frequently alarming the driver. Selective alarming can remind the driver at critical moments while not disturbing the driver's daily entertainment life. When the estimated pothole depth is greater than the preset threshold, the vehicle end sends a message to the CAN bus, and an alarm on the CAN bus is used to give an alarm, allowing the driver to have time to select to cut out the autonomous driving mode and brake to decelerate. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings herein are incorporated into the specification and form a part of the specification, showing embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application.

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application or in the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0018] Figure 1 It is a schematic flowchart provided for the first embodiment of the road pothole warning method of the present application; Figure 2 It is a schematic diagram of the installation positions of millimeter-wave radars provided for the first embodiment of the road pothole warning method of the present application; Figure 3 It is a schematic flowchart provided for the second embodiment of the road pothole warning method of the present application; Figure 4 It is a schematic diagram of the ranging principle of millimeter-wave radars of a new energy vehicle provided for the second embodiment of the road pothole warning method of the present application; Figure 5 It is a schematic diagram of a new energy vehicle for determining whether a pothole is on the driving road provided for the second embodiment of the road pothole warning method of the present application; Figure 6 It is a schematic diagram of pothole depth estimation of a new energy vehicle provided for the second embodiment of the road pothole warning method of the present application; Figure 7 It is a schematic flowchart provided for the third embodiment of the road pothole warning method of the present application; Figure 8 It is a schematic diagram of the module structure of the road pothole warning device according to the embodiment of the present application; Figure 9This is a schematic diagram of the device structure of the hardware operating environment involved in the road pothole warning method in the embodiments of the present application.

[0019] The implementation, functional characteristics, and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. Detailed implementation manners

[0020] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0021] To better understand the technical solutions of the present application, the following will be described in detail with reference to the accompanying drawings of the specification and specific implementation manners.

[0022] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication, and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device, a road pothole warning device, etc. that can implement the above functions. The following takes the road pothole warning device as an example to illustrate this embodiment and the following embodiments.

[0023] Based on this, the embodiments of the present application provide a road pothole warning method, referring to Figure 1 , Figure 1 This is a flowchart of the first embodiment of the road pothole warning method of the present application.

[0024] In this embodiment, the road pothole warning method includes steps S10 to S30: Step S10, obtaining road information through the first millimeter-wave radar and the second millimeter-wave radar; It should be noted that before obtaining the road information, it is necessary to ensure that the vehicle is equipped with advanced driver assistance (ADAS). This system is used to obtain the surrounding environment at any time during driving, collect data, identify, detect, and track static and dynamic objects, and combine the navigation map data to perform system operations and analyses, so as to pre-let the driver be aware of possible dangers and effectively improve the comfort and safety of driving. In addition, two millimeter-wave radars (i.e., the first millimeter-wave radar and the second millimeter-wave radar) are installed on the front of the vehicle. These two devices are crucial for real-time measurement of the physical environment around the vehicle, especially for detecting road potholes ahead. They can obtain the distance between the pothole and the vehicle while determining whether the road pothole is on the road where the vehicle is driving. Ensure that all radars and systems are correctly installed and connected to the central processing unit of the vehicle.

[0025] As Figure 2 shown, Figure 2It is a mounting position diagram of millimeter-wave radars. Here, d is the distance between the first millimeter-wave radar and the second millimeter-wave radar, d1 is the distance between the first millimeter-wave radar and the left front of the vehicle, and d2 is the distance between the second millimeter-wave radar and the right front of the vehicle. Among them, the left and right millimeter-wave radars on the ADAS can be fabricated separately and then mounted, or they can be fabricated in combination to form a single module during the manufacturing process. In this way, the confirmation of the distance d between the left and right millimeter-wave radars and the distances d1 and d2 between them and the left and right fronts of the vehicle respectively can be more accurate.

[0026] It should be understood that before starting any driving, the working states of the left and right millimeter-wave radars must be checked. This includes but is not limited to checking whether they can successfully transmit and receive electromagnetic wave signals, as well as the intensity and stability of the signals. This is to ensure that during the autonomous driving process, the millimeter-wave radars can accurately obtain information about the road ahead and provide a reliable basis for subsequent data processing.

[0027] It can be understood that this strategy is applied in the autonomous driving mode. In this mode, the vehicle will independently determine key operations such as the driving route and speed adjustment based on the built-in autonomous driving algorithm. At the same time, the system will continuously collect data from the millimeter-wave radars to monitor in real time whether there are potholes ahead, and the road information includes whether there are potholes on the road ahead.

[0028] Furthermore, the millimeter-wave radars adopted in this strategy can be replaced by ultrasonic radars in some scenarios. However, using ultrasonic waves of ultrasonic radars to judge potholes will result in poor real-time performance due to the distance limitation of ultrasonic radars themselves, and there is a low accuracy situation when the vehicle is driving at high speed. However, compared with millimeter-wave radars, ultrasonic radars are cheaper and the overall vehicle cost is smaller.

[0029] Step S20: When it is determined according to the road information that there is a pothole on the road, obtain the depth of the pothole; It should be noted that when the vehicle encounters a pothole ahead during autonomous driving, the left and right millimeter-wave radars installed on the vehicle head will respectively emit electromagnetic waves to the ground.

[0030] In a specific implementation, the millimeter-wave radar judges the depth of the pothole by receiving the electromagnetic waves returned from the pothole.

[0031] Step S30: Perform road pothole warning processing according to the pothole depth information and the preset pothole depth threshold; It should be noted that the preset pit depth threshold can be set, such as 20 cm or 50 cm. The calculated pit depth will be compared with the system's preset safety threshold of 50 cm. If the depth is less than 50 cm, it is considered that the pit has little impact on vehicle driving and no special treatment is required. On the contrary, if the depth is greater than 50 cm, it is considered that there is a high driving risk. The system will send an alarm message to the driver through the in-vehicle network (CAN bus) and remind the driver to pay attention to the pit ahead through the dashboard or the sound and light alarm device.

[0032] This embodiment provides a method for warning of road pits. Road information is obtained through the first millimeter-wave radar and the second millimeter-wave radar. When it is determined that there is a pit on the road according to the road information, the pit depth is obtained. Road pit warning processing is performed according to the pit depth information and the preset pit depth threshold. Through two millimeter-wave radars, it is possible to accurately determine whether there is a pit on the road ahead and timely alarm the driver, allowing the driver to have time to select to cut out the automatic driving mode and brake to decelerate.

[0033] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar content as in the above-mentioned first embodiment can be referred to the above introduction and will not be repeated hereinafter. On this basis, please refer to Figure 3 , step S20 includes steps S201 to S203: Step S201, when it is determined that there is a pit on the road according to the road information, determine whether the pit is located on the vehicle driving trajectory; It should be noted that as Figure 4 shown, Figure 4 is the schematic diagram of the millimeter-wave radar ranging principle for new energy vehicles. First, the FWCM signal is emitted to the pit through the millimeter wave, and then the returned signal is received through the wireless receiving device. Then, distance and speed processing is performed through the first matching filter, the first low-pass filter, and the first frequency analysis device to complete the determination of the pit distance.

[0034] It should be understood that when the vehicle encounters a pit ahead during automatic driving, the two millimeter-wave radars installed on the front of the vehicle will respectively emit electromagnetic waves to the ground.

[0035] First, the millimeter-wave radar will determine whether this pit is on the vehicle driving road rather than on both sides of the vehicle driving road.

[0036] In a feasible implementation manner, step S201 includes steps A11 to A14: Step A11: Based on the electromagnetic wave emission speed of the millimeter-wave radar, the time from the first millimeter-wave radar emission to return, and the time from the second millimeter-wave radar emission to return, determine the distance between the first millimeter-wave radar and the pit and the distance between the second millimeter-wave radar and the pit; In a specific implementation, first obtain the time from the emission to the return of the electromagnetic wave of the left millimeter-wave radar, denoted as , and the time from the emission to the return of the electromagnetic wave of the right millimeter-wave radar, denoted as . Given that the emission speed of the electromagnetic wave of the millimeter-wave radar is , and then through the mathematical formula:

[0037]

[0038] obtain the distance from the left millimeter-wave radar to the pothole and the distance from the right millimeter-wave radar to the pothole.

[0039] It should be noted that the left millimeter-wave radar is the first millimeter-wave radar, and the right millimeter-wave radar is the second millimeter-wave radar.

[0040] Step A12: Obtain the distance between the first millimeter-wave radar and the left front of the vehicle and the distance between the second millimeter-wave radar and the right front of the vehicle; In a specific implementation, obtain the distance from the left millimeter-wave radar to the left edge of the front of the vehicle as , and the distance from the right millimeter-wave radar to the left edge of the front of the vehicle as .

[0041] Step A13: Based on the distance between the first millimeter-wave radar and the left front of the vehicle, the distance between the second millimeter-wave radar and the right front of the vehicle, the distance between the first millimeter-wave radar and the pothole, the maximum emission angle of the millimeter-wave radar, and the distance between the second millimeter-wave radar and the pothole, determine the first angle between the line connecting the pothole and the first millimeter-wave radar and the left boundary of the front of the vehicle and the second angle between the line connecting the pothole and the second millimeter-wave radar and the right boundary of the front of the vehicle; In a specific implementation, through the mathematical formula:

[0042]

[0043] transform to obtain:

[0044]

[0045] obtain the included angle between the pothole and the vertical cross-section on the left side of the front of the vehicle and the included angle between the pothole and the vertical cross-section on the right side of the front of the vehicle.

[0046] Step A14: When the first included angle is greater than the preset included angle threshold and the second included angle is greater than the preset included angle threshold, it is determined that the pothole is on the vehicle driving trajectory.

[0047] In a specific implementation, if and at this time, it is considered that the pothole is on the vehicle driving road. In other cases, it is considered that the pothole is on both sides of the vehicle driving road.

[0048] It can be understood that a vertical cross-section is drawn with the vehicle head, and then horizontal lines are drawn from the left and right millimeter-wave radars respectively. The first included angle and the second included angle are respectively compared with 90° - θ / 2. If both are greater, it means that the pothole is in the dark orange area in the center of the figure. As Figure 5 shown, Figure 5 is a schematic diagram for a new energy vehicle to determine whether a pothole is on the driving road. Among them, is the emission angle of the millimeter-wave radar, is the distance between the pothole and the vehicle, α is the first included angle, β is the second included angle. If the pothole is within the emission angles of the left and right millimeter-wave radars, it is determined that the pothole is on the driving road. If the pothole is within the emission angle of the left or right millimeter-wave radar, it is determined that the pothole is on both sides of the driving road.

[0049] Step S202: When it is determined that the pothole is not on the vehicle driving trajectory, control the vehicle to decelerate; It should be noted that at this time, only control the vehicle to pass through the area with the pothole at a relatively stable vehicle speed, and there is no need for sudden deceleration.

[0050] Step S203: When it is determined that the pothole is on the vehicle driving trajectory, obtain the depth of the pothole.

[0051] In a specific implementation, after determining that the pothole is on the vehicle driving road, calculate the depth of the pothole through the distance and angle data obtained by the millimeter-wave radar.

[0052] In a feasible implementation manner, step S203 includes steps A21 to A23: Step A21: When it is determined that the pothole is on the vehicle driving trajectory, obtain the pitch angle between the direction of the millimeter-wave radar electromagnetic wave returning and the horizontal plane and the height of the millimeter-wave radar from the ground, where the pitch angle is the vertical included angle between the direction of the millimeter-wave radar electromagnetic wave returning and the horizontal plane; It should be noted that the distance between the pothole and the vehicle is , the included angle between the direction of the millimeter-wave radar electromagnetic wave and the vertical horizontal plane is ; the height of the millimeter-wave radar from the ground is .

[0053] The millimeter-wave radar emits electromagnetic waves, which will be reflected back when the electromagnetic waves encounter obstacles such as potholes. There is an angle between the direction of the returned electromagnetic wave and the horizontal plane, that is, the pitch angle.

[0054] It should be understood that the millimeter-wave radar is installed on the vehicle, and its height from the ground is a relatively fixed parameter, which can be obtained through the height sensor of the vehicle. For example, a height sensor is installed near the suspension system of the vehicle to monitor the distance between the vehicle body and the ground in real time, so as to indirectly obtain the height of the millimeter-wave radar from the ground.

[0055] Step A22: Determine the distance between the pothole and the vehicle based on the distance between the first millimeter-wave radar and the pothole and the distance between the second millimeter-wave radar and the pothole; In a specific implementation, the method of taking the median value can be used to determine the distance between the pothole and the vehicle. This is because the two millimeter-wave radars installed on the vehicle are generally symmetrically distributed, and the distances from them to the pothole can reflect the relative position between the vehicle and the pothole to a certain extent. Specifically, it is to take the median value of the two values of the distance between the first millimeter-wave radar and the pothole and the distance between the second millimeter-wave radar and the pothole as the distance between the pothole and the vehicle. For example, if the distance from the first millimeter-wave radar to the pothole is d1 and the distance from the second millimeter-wave radar to the pothole is d2, then the distance d between the pothole and the vehicle can be expressed as d = , and a relatively reasonable estimated value of the distance between the vehicle and the pothole can be obtained quickly.

[0056] Step A23: Obtain the pothole depth based on the distance between the pothole and the vehicle, the angle between the direction of the millimeter-wave radar electromagnetic wave and the vertical horizontal plane, and the height of the millimeter-wave radar from the ground, and determine the pothole depth.

[0057] It should be noted that since the vehicle is moving, the depth of this pothole obtained each time is naturally different. However, during driving, this value will definitely tend to be stable (it can be understood that at the beginning, it may not hit the deepest part of the pothole, but as the vehicle gets closer to the pothole during driving, it will definitely hit the deepest part of the pothole). As long as the obtained pothole depth is greater than the preset value, even if it does not hit the deepest part, it can naturally be judged that this pothole is relatively deep and requires deceleration and other treatments.

[0058] It should be understood that after calculating the pothole depth, the calculation result can be verified and confirmed. This may involve further calibration of the millimeter-wave radar data, considering the influence of surrounding environmental factors on the measurement result, and comparing the data of other sensors to comprehensively determine the accuracy of the pothole depth, so as to finally determine the pothole depth and provide a basis for the driving safety decision of the vehicle.

[0059] As shown Figure 6 in the figure Figure 6 is a schematic diagram for estimating the depth of a pothole in a new energy vehicle. Among them is a point where the electromagnetic wave emitted by the millimeter-wave radar of the vehicle hits the pothole is the angle between the direction of this beam of electromagnetic wave and the horizontal plane perpendicular to the ground is the height between the millimeter-wave radar and the ground is the longitudinal distance between the millimeter-wave radar and the point inside the pothole is the distance between the vehicle and the point inside the pothole

[0060] In a specific implementation, it is known that the horizontal width of the pothole can be ignored compared to the distance between the vehicle and the pothole. It is known that the distance between the vehicle and the point where the electromagnetic wave hits the pothole is , the angle between the direction of this beam of electromagnetic wave and the horizontal plane perpendicular to the ground is , and it is known that the height between the millimeter-wave radar and the ground is .

[0061] Then through the mathematical formula:

[0062]

[0063] the estimated depth of the pothole at this time is obtained

[0064] It should be noted that as the vehicle moves forward continuously the estimated value of also changes continuously. When the value of tends to be stable after a period of time, the final estimated value of the pothole depth

[0065] It can be assumed that a new energy vehicle is already equipped with an ADAS system and two millimeter-wave radars, left and right, are installed. When driving in the autonomous driving mode and encountering a pothole ahead, the time from the emission to the return of the electromagnetic waves of the left and right millimeter-wave radars is 10 microseconds each. Through calculation: The distance between the pothole and the vehicle is: 5 microseconds × the speed of electromagnetic wave emission (the speed of electromagnetic wave emission is about 8 m / s) = 150 m

[0066] It is known that the interval between the left and right millimeter-wave radars is 1 m, and the distances , from the left and right millimeter-wave radars to the left and right front ends of the vehicle are both 30 cm, and the maximum emission angles of the left and right millimeter-wave radars are both 70°

[0067] Derived from a mathematical formula obtained is approximately 50°, less than , Similarly, it is thus determined that the pothole is on the vehicle driving road.

[0068] The height of the millimeter-wave radar from the ground is 80 cm, and the angle between the direction of the electromagnetic wave and the horizontal plane perpendicular to it is 88°. When it is estimated that the depth tends to be stable, the distance between the vehicle and the pothole is 50 m.

[0069] Then the final estimated value of the depth is obtained as .

[0070] The vehicle end compares it with the system preset safety threshold of 0.5 m, and obtains the conclusion that the depth is greater than 0.5 m. It is considered that there is a high driving risk for autonomous driving at this time. The system sends an alarm message to the driver through the CAN bus, and reminds the driver to pay attention to the pothole ahead through the instrument panel or the sound and light alarm device. After that, the system will also send an instruction to the motor control system of the vehicle through the CAN bus to appropriately reduce the motor speed to achieve self-deceleration of the vehicle.

[0071] This embodiment provides a road pothole warning method, which realizes road pothole recognition by installing two millimeter-wave radars on a new energy vehicle. When it is estimated that the pothole depth is greater than the preset threshold, the vehicle end sends a message to the CAN bus, and realizes the alarm through the alarm on the CAN bus. There is no need to carry an additional lidar, and only two millimeter-wave radars need to be additionally installed on the basis of the original ADAS. The millimeter-wave radar emits electromagnetic waves with little influence from the environment, and its vertical field of view is large. By using the method of installing two millimeter-wave radars, irrelevant road data on both sides of the vehicle can be effectively filtered out, which is convenient for data modeling. The millimeter-wave radar has a very short transceiver time and strong real-time performance.

[0072] Based on the first embodiment and the second embodiment of the present application, in the third embodiment of the present application, the same or similar content as the above embodiments can be referred to the above introduction and will not be repeated hereinafter. On this basis, please refer to Figure 7 , step S30 includes steps S301~S306: Step S301, determine whether the pothole depth is less than the first preset pothole depth threshold; It should be noted that the first preset pothole depth threshold can be set, such as 20 cm; Step S302, when the pothole depth is less than the first preset pothole depth threshold, control the motor to reduce the speed to control the vehicle to decelerate; In a specific implementation, when the pothole depth is less than 20 cm, the pothole information does not pass through the alarm. Instead, a message is transmitted to the motor via the CAN bus to issue a command to reduce the motor speed, achieving the purpose of deceleration.

[0073] Step S303: When the pothole depth is greater than or equal to the first preset pothole depth threshold, determine whether the pothole depth is less than the second preset pothole depth threshold. It should be noted that the second preset pothole depth can be set, such as 50 cm. At this time, the second preset pothole depth can also be referred to as the depth danger value.

[0074] Step S304: When the pothole depth is less than the second preset pothole depth threshold, give a road pothole warning to prompt the driver and control the motor to reduce the speed to control the vehicle to decelerate. It should be noted that when the pothole depth is less than 50 cm and greater than 20 cm, the pothole information will pass through the alarm. The alarm will handle the alarm, which is reflected to the vehicle end to remind the driver, and a message will be transmitted to the motor via the CAN bus to issue a command to reduce the motor speed, achieving the purpose of deceleration.

[0075] Step S305: When the pothole depth is greater than or equal to the second preset pothole depth threshold, identify the pothole and determine the avoidance trajectory. It should be noted that when the pothole depth is greater than 50 cm, the pothole can be identified through the high-precision map at this time, and an avoidance instruction is sent to the in-vehicle computer, and a reasonable avoidance trajectory is specified.

[0076] Step S306: Control the vehicle to avoid based on the avoidance trajectory. In a specific implementation, that is, the vehicle is controlled to avoid the pothole along the avoidance trajectory through the autonomous driving mode to ensure the safety of the vehicle and the driver.

[0077] This embodiment provides a road pothole warning method. By comparing the estimated pothole depth value with the preset value at the vehicle end, it is possible to avoid the vehicle end frequently alarming the driver. The screened alarm can remind the driver at critical moments while not disturbing the driver's daily entertainment life. By sending a message via the CAN bus to achieve the vehicle deceleration function, it is possible to avoid the problem that the driver does not have time to switch modes and brake to decelerate by themselves after receiving the alarm, reducing potential safety hazards.

[0078] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the road pothole warning method of this application. Based on this technical concept, more forms of simple transformations are within the protection scope of this application.

[0079] This application also provides a road pothole warning device. Please refer to Figure 8 The road pothole warning device includes: The road information acquisition module 10 is configured to acquire road information through the first millimeter-wave radar and the second millimeter-wave radar; The pothole depth determination module 20 is configured to acquire the pothole depth when it is determined that there is a pothole on the road according to the road information; The processing module 30 is configured to perform road pothole warning processing according to the pothole depth information and a preset pothole depth threshold.

[0080] The road pothole warning device provided by this application adopts the road pothole warning method in the above-mentioned embodiment, and can solve the technical problem of how to accurately identify potholes. Compared with the prior art, the beneficial effects of the road pothole warning device provided by this application are the same as those of the road pothole warning method provided by the above-mentioned embodiment, and other technical features in the road pothole warning device are the same as the features disclosed in the method of the above-mentioned embodiment, and will not be elaborated here.

[0081] In one embodiment, the pothole depth determination module 20 is further configured to determine whether the pothole is located on the vehicle driving trajectory when it is determined that there is a pothole on the road according to the road information; When it is determined that the pothole is not located on the vehicle driving trajectory, control the vehicle to decelerate; When it is determined that the pothole is located on the vehicle driving trajectory, acquire the pothole depth.

[0082] In one embodiment, the pothole depth determination module 20 is further configured to determine the distance between the first millimeter-wave radar and the pothole and the distance between the second millimeter-wave radar and the pothole based on the electromagnetic wave emission speed of the millimeter-wave radar, the time from the first millimeter-wave radar's emission to return, and the time from the second millimeter-wave radar's emission to return; Acquire the distance between the first millimeter-wave radar and the left front of the vehicle and the distance between the second millimeter-wave radar and the right front of the vehicle; Based on the distance between the first millimeter-wave radar and the left front of the vehicle, the distance between the second millimeter-wave radar and the right front of the vehicle, the distance between the first millimeter-wave radar and the pothole, the maximum emission angle of the millimeter-wave radar, and the distance between the second millimeter-wave radar and the pothole, determine the first angle between the line connecting the pothole and the first millimeter-wave radar and the left boundary of the vehicle head and the second angle between the line connecting the pothole and the second millimeter-wave radar and the right boundary of the vehicle head; When the first angle is greater than a preset angle threshold and the second angle is greater than the preset angle threshold, determine that the pothole is located on the vehicle driving trajectory.

[0083] In one embodiment, the pothole depth determination module 20 is further configured to, when determining that the pothole is located on the vehicle driving trajectory, obtain the pitch angle between the direction of the millimeter-wave radar electromagnetic wave return and the horizontal plane and the height of the millimeter-wave radar from the ground, where the pitch angle is the vertical angle between the direction of the millimeter-wave radar electromagnetic wave return and the horizontal plane; Determine the distance between the pothole and the vehicle based on the distance between the first millimeter-wave radar and the pothole and the distance between the second millimeter-wave radar and the pothole; Obtain the pothole depth based on the distance between the pothole and the vehicle, the pitch angle, and the height of the millimeter-wave radar from the ground, and determine the pothole depth In one embodiment, the processing module 30 is further configured to determine whether the pothole depth is less than a first preset pothole depth threshold; When the pothole depth is less than the first preset pothole depth threshold, control the motor to reduce the speed to control the vehicle to decelerate.

[0084] In one embodiment, the processing module 30 is further configured to, when the pothole depth is greater than or equal to the first preset pothole depth threshold, determine whether the pothole depth is less than a second preset pothole depth threshold; When the pothole depth is less than the second preset pothole depth threshold, give a road pothole warning to prompt the driver, and control the motor to reduce the speed to control the vehicle to decelerate.

[0085] In one embodiment, the processing module 30 is further configured to, when the pothole depth is greater than or equal to the second preset pothole depth threshold, identify the pothole and determine an avoidance trajectory; Control the vehicle to avoid based on the avoidance trajectory.

[0086] The present application provides a road pothole warning device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the road pothole warning method in the first embodiment above.

[0087] Next, refer to Figure 9, which shows a schematic structural diagram of a road pothole warning device suitable for implementing the embodiments of the present application. The road pothole warning device in the embodiments of the present application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 9 The road pothole warning device shown is only an example and should not impose any limitations on the functions and usage scope of the embodiments of the present application.

[0088] As Figure 9 shown, the road pothole warning device may include a processing device 1001 (such as a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM: Read Only Memory) 1002 or the program loaded from the storage device 1003 into the random access memory (RAM: Random Access Memory) 1004. In the RAM 1004, various programs and data required for the operation of the road pothole warning device are also stored. The processing device 1001, the ROM 1002, and the RAM 1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems may be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the road pothole warning device to communicate with other devices wirelessly or wiredly to exchange data. Although the figure shows a road pothole warning device with various systems, it should be understood that it is not required to implement or have all the systems shown. Instead, more or fewer systems may be implemented or had.

[0089] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product that includes a computer program carried on a computer-readable medium, and the computer program contains program codes for executing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by a processing device 1001, the above functions defined in the methods of the embodiments disclosed in the present application are executed.

[0090] The road pothole warning device provided by the present application adopts the road pothole warning method in the above embodiment, and can solve the technical problem of how to accurately identify potholes. Compared with the prior art, the beneficial effects of the road pothole warning device provided by the present application are the same as those of the road pothole warning method provided by the above embodiment, and other technical features in the road pothole warning device are the same as the features disclosed in the method of the previous embodiment, and will not be elaborated here.

[0091] It should be understood that the various parts disclosed in the present application can be implemented by hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in a suitable manner in any one or more embodiments or examples.

[0092] As described above, only the specific embodiments of the present application are provided, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present application, and all should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

[0093] The present application provides a computer-readable storage medium having computer-readable program instructions (i.e., computer programs) stored thereon, and the computer-readable program instructions are used to execute the road pothole warning method in the above embodiment.

[0094] The computer-readable storage medium provided by the present application can be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or components, or any combination of the above. More specific examples of the computer-readable storage medium may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM) or flash memory, optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this embodiment, the computer-readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, device, or component. The program code contained on the computer-readable storage medium can be transmitted by any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0095] The above computer-readable storage medium can be included in the road pothole warning device; it can also exist independently without being assembled into the road pothole warning device.

[0096] The above computer-readable storage medium carries one or more programs. When the one or more programs are executed by the road pothole warning device, the road pothole warning device is caused to: obtain road information through the first millimeter-wave radar and the second millimeter-wave radar; obtain the pothole depth when it is determined according to the road information that there is a pothole on the road; and perform road pothole warning processing according to the pothole depth information and a preset pothole depth threshold.

[0097] Computer program code for performing the operations of this application can be written in one or more programming languages or combinations thereof. The above-mentioned programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN: Local Area Network) or a wide area network (WAN: Wide Area Network), or it can be connected to an external computer (for example, by connecting through an Internet service provider via the Internet).

[0098] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of the code, and this module, program segment, or part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order from that marked in the accompanying drawings. For example, two consecutively represented blocks may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, as well as the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0099] The modules described in the embodiments of this application can be implemented in software or in hardware. Among them, the name of the module does not constitute a limitation on the unit itself in some cases.

[0100] The readable storage medium provided by this application is a computer-readable storage medium. The computer-readable storage medium stores computer-readable program instructions (i.e., computer programs) for performing the above-mentioned road pothole warning method, and can solve the technical problem of how to accurately identify potholes. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by this application are the same as those of the road pothole warning method provided in the above embodiments, and will not be elaborated here.

[0101] The present application also provides a computer program product, including a computer program which, when executed by a processor, implements the steps of the road pothole warning method as described above.

[0102] The computer program product provided by the present application can solve the technical problem of how to accurately identify potholes. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as those of the road pothole warning method provided by the above embodiments, and will not be elaborated herein.

[0103] The above are only partial embodiments of the present application, and do not limit the patent scope of the present application accordingly. Any equivalent structural transformation made under the technical concept of the present application by using the content of the specification and drawings of the present application, or direct / indirect application in other related technical fields, is included in the patent protection scope of the present application.

Claims

1. A road pothole warning method, characterized in that: Applied to a road pothole warning system, the road pothole warning system includes a first millimeter wave radar and a second millimeter wave radar, and the road pothole warning method includes the following steps: Acquiring road information by using the first millimeter-wave radar and the second millimeter-wave radar; When it is determined according to the road information that there is a pothole on the road, obtaining the depth of the pothole; Performing road pothole warning processing according to the pothole depth information and a preset pothole depth threshold; When it is determined according to the road information that there is a pothole on the road, the step of obtaining the pothole depth comprises: When it is determined according to the road information that there is a pothole on the road, determining whether the pothole is located on the vehicle's driving track; When it is determined that the pothole is not located on the vehicle's driving track, controlling the vehicle to decelerate; When it is determined that the pothole is located on the vehicle's driving track, obtaining the pothole depth; When it is determined according to the road information that there is a pothole on the road, the step of determining whether the pothole is located on the vehicle driving track includes: Determine the distance between the first millimeter wave radar and the pit and the distance between the second millimeter wave radar and the pit based on the millimeter wave radar electromagnetic wave transmission speed, the time from the first millimeter wave radar to the return, and the time from the second millimeter wave radar to the return; Acquire the distance between the first millimeter-wave radar and the left front of the vehicle and the distance between the second millimeter-wave radar and the right front of the vehicle; Based on the distance between the first millimeter-wave radar and the left front of the vehicle, the distance between the second millimeter-wave radar and the right front of the vehicle, the distance between the first millimeter-wave radar and the pothole, the maximum emission angle of the millimeter-wave radar, and the distance between the second millimeter-wave radar and the pothole, determine a first angle between the line connecting the pothole and the first millimeter-wave radar and the left side boundary of the front of the vehicle, and a second angle between the line connecting the pothole and the second millimeter-wave radar and the right side boundary of the front of the vehicle; When the first angle is greater than a preset angle threshold and the second angle is greater than the preset angle threshold, it is determined that the pothole is located on the vehicle driving trajectory.

2. The road pothole warning method according to claim 1, characterized in that: The step of obtaining the depth of the pothole when determining that the pothole is located on the vehicle driving track includes: When it is determined that the pothole is located on the vehicle's driving track, the pitch angle between the direction in which the millimeter-wave radar electromagnetic wave returns and the horizontal plane and the height of the millimeter-wave radar and the ground are obtained, wherein the pitch angle is the vertical angle between the direction in which the millimeter-wave radar electromagnetic wave returns and the horizontal plane; determining a distance between the pothole and the vehicle based on a distance between the first millimeter-wave radar and the pothole and a distance between the second millimeter-wave radar and the pothole; The depth of the pothole is obtained based on the distance between the pothole and the vehicle, the pitch angle, and the height between the millimeter wave radar and the ground, and the depth of the pothole is determined.

3. The road pothole warning method according to claim 1, characterized in that: The step of performing road pothole warning processing according to the pothole depth and a preset pothole depth threshold comprises: Determining whether the pit depth is less than a first preset pit depth threshold; When the pothole depth is less than a first preset pothole depth threshold, the motor is controlled to reduce the rotation speed to control the vehicle to decelerate.

4. The road pothole warning method according to claim 3, characterized in that: After the step of determining whether the pit depth is less than a first preset pit depth threshold, the method further includes: When the pit depth is greater than or equal to the first preset pit depth threshold, determining whether the pit depth is less than a second preset pit depth threshold; When the pothole depth is less than a second preset pothole depth threshold, a pothole warning is issued to prompt the driver, and the motor is controlled to reduce the rotation speed to control the vehicle to decelerate.

5. The road pothole warning method according to claim 4, characterized in that: After the step of determining whether the pit depth is less than a second preset pit depth threshold, the method further includes: When the depth of the pit is greater than or equal to a second preset pit depth threshold, identifying the pit and determining an avoidance trajectory; The vehicle is controlled to perform avoidance based on the avoidance trajectory.

6. A road pothole warning device, characterized in that: The device comprises: A road information acquisition module, used to acquire road information through the first millimeter wave radar and the second millimeter wave radar; A pothole depth determination module, configured to obtain the pothole depth when determining that a pothole exists on the road according to the road information; The pothole depth determination module is further used to determine whether the pothole is located on the vehicle driving track when it is determined that there is a pothole on the road according to the road information; control the vehicle to decelerate when it is determined that the pothole is not located on the vehicle driving track; and obtain the pothole depth when it is determined that the pothole is located on the vehicle driving track; The pothole depth determination module is further used to determine the distance between the first millimeter-wave radar and the pothole and the distance between the second millimeter-wave radar and the pothole based on the millimeter-wave radar electromagnetic wave transmission speed, the time from the first millimeter-wave radar to the return, and the time from the second millimeter-wave radar to the return; obtain the distance between the first millimeter-wave radar and the left front of the vehicle and the distance between the second millimeter-wave radar and the right front of the vehicle; based on the distance between the first millimeter-wave radar and the left front of the vehicle, the distance between the second millimeter-wave radar and the pothole, the maximum transmission angle of the millimeter-wave radar, and the distance between the second millimeter-wave radar and the pothole, determine a first angle between the line connecting the pothole and the first millimeter-wave radar and the left side boundary of the front of the vehicle and a second angle between the line connecting the pothole and the second millimeter-wave radar and the right side boundary of the front of the vehicle; when the first angle is greater than a preset angle threshold and the second angle is greater than the preset angle threshold, determine that the pothole is located on the vehicle driving track; The processing module is used to perform road pothole warning processing according to the pothole depth information and a preset pothole depth threshold.

7. A road pothole warning device, characterized in that: The device comprises: a memory, a processor, and a pothole warning program stored in the memory and executable on the processor, wherein the pothole warning program is configured to implement the steps of the pothole warning method according to any one of claims 1 to 5.

8. A storage medium, characterized in that: The storage medium stores a pothole warning program, and when the pothole warning program is executed by the processor, the steps of the pothole warning method according to any one of claims 1 to 5 are implemented.

Images