Method, device, electronic device and readable medium for predicting road surface water depth
Through the combination of monocular cameras and binocular cameras, the problem that autonomous vehicles find it difficult to perceive the water depth of road areas is solved, and fast and accurate depth analysis and real-time perception are achieved to ensure safe driving.
Patent Information
- Application Number
- CN202310645105.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-06-01
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2043-06-01
AI Technical Summary
It is difficult for autonomous vehicles to perceive the depth of the road area during driving, which may cause engine shutdown, short-circuiting of the electronic control system, and threaten the safety of passengers.
A monocular camera and binocular camera collect the front vehicle image and point cloud information, mark the wheel area through the target recognition model, match the point cloud information, determine the wheel size and water surface height, calculate the road area and water depth, and predict the depth through curve fitting change.
It realizes rapid and accurate analysis of the water depth of the road area, provides real-time depth information, improves the vehicle's perception of water accumulation, and ensures safe driving.
Smart Images

Figure CN116758134B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer vision technology, and in particular to a method for predicting the depth of road waterlogging, a device for predicting the depth of road waterlogging, an electronic device and a computer-readable medium. Background Art
[0002] Generally speaking, autonomous vehicles often encounter many complex road conditions during driving. One of the most common is water accumulation on the road. Water accumulation on the road is usually caused by puddles formed in low-lying areas of the road after rainfall, or floods caused by poor drainage and extreme weather such as heavy rain and rainstorms. Since underwater conditions are usually difficult for vehicles and drivers to perceive, if the water level exceeds the safe wading depth of the vehicle, it is very likely to cause engine stalling, short circuit of the electronic control system, leakage of high-voltage components and other faults, seriously threatening the personal and property safety of passengers. Summary of the invention
[0003] The embodiments of the present invention provide a method, device, electronic device and computer-readable storage medium for predicting the depth of waterlogging on a road surface, so as to solve the problem that the depth of waterlogging on a road surface is difficult to perceive.
[0004] An embodiment of the present invention discloses a method for predicting the depth of water on a road surface, which is applied to a vehicle, wherein the vehicle includes a monocular camera and a binocular camera, and the method includes:
[0005] Obtain the image of the vehicle ahead captured by the monocular camera in the vehicle and the point cloud information of the vehicle ahead captured by the binocular camera in the vehicle;
[0006] Determine, based on the front vehicle image and the front vehicle point cloud information, size information of the front vehicle wheels and water surface height information of the front vehicle wheels exposed above the accumulated water surface;
[0007] The depth of water on the road surface is determined based on the water surface height information and the size information of the wheels of the preceding vehicle.
[0008] Optionally, the method further comprises:
[0009] The front vehicle image is input into a preset target recognition model to obtain wheel marking information output by the target recognition model; wherein the wheel marking information is used to mark the area where the front vehicle wheels are located in the front vehicle image.
[0010] Optionally, the step of determining the size information of the front vehicle wheels and the water surface height information of the front vehicle wheels exposed above the accumulated water surface based on the front vehicle image and the front vehicle point cloud information includes:
[0011] Based on the wheel annotation information, matching the area where the front vehicle wheel is located in the front vehicle image with the front vehicle point cloud information, and determining the point cloud information corresponding to the front vehicle wheel in the front vehicle point cloud information;
[0012] Based on the point cloud information corresponding to the front wheel, the size information of the front wheel and the water surface height information of the front wheel exposed above the accumulated water surface are determined.
[0013] Optionally, the step of matching the area where the front vehicle wheel is located in the front vehicle image with the front vehicle point cloud information based on the wheel annotation information, and determining the point cloud information corresponding to the front vehicle wheel in the front vehicle point cloud information includes:
[0014] Based on the wheel annotation information, matching the area where the front vehicle wheel is located in the front vehicle image with the front vehicle point cloud information, and determining the to-be-confirmed point cloud information corresponding to the front vehicle wheel in the front vehicle point cloud information;
[0015] Outliers are removed from the point cloud information to be confirmed to obtain target point cloud information corresponding to the wheels of the front vehicle.
[0016] Optionally, the method further comprises:
[0017] Obtaining several road surface water depths predicted within a preset time period;
[0018] Based on the plurality of road surface water depths within a preset time period, a change in road surface water depth is predicted.
[0019] Optionally, the step of predicting the change of road surface water depth based on the plurality of road surface water depths within a preset time period includes:
[0020] Taking time as the independent variable and road surface water depth as the dependent variable, curve fitting is performed to obtain the water depth variation curve;
[0021] Based on the water depth change curve, predict the change of road water depth.
[0022] Optionally, the step of obtaining a plurality of road surface water depths predicted within a preset time period includes:
[0023] For the road surface water depth obtained at the current moment, the road surface water depth obtained at the current moment is compared with the road surface water depth obtained at the previous moment;
[0024] If the deviation between the road surface water depth obtained at the current moment and the road surface water depth obtained at the previous moment is greater than a preset threshold, the road surface water depth obtained at the current moment is regarded as abnormal data;
[0025] If the deviation between the road surface water depth obtained at the current moment and the road surface water depth obtained at the previous moment is not greater than the preset threshold, the road surface water depth obtained at the current moment is regarded as normal data;
[0026] Normal data within a preset time period is retained to obtain a number of road surface water depths preset within the preset time period.
[0027] The embodiment of the present invention further discloses a device for predicting the depth of water on a road surface, which is applied to a vehicle, wherein the vehicle includes a monocular camera and a binocular camera, and the device includes:
[0028] An acquisition module is used to acquire the image of the preceding vehicle acquired by the monocular camera in the vehicle and the point cloud information of the preceding vehicle acquired by the binocular camera in the vehicle;
[0029] A size determination module, used to determine the size information of the front vehicle wheel and the water surface height information of the front vehicle wheel exposed on the surface of the accumulated water based on the front vehicle image and the front vehicle point cloud information;
[0030] The depth determination module is used to determine the depth of water on the road based on the water surface height information and the size information of the wheels of the front vehicle.
[0031] Optionally, the device further comprises:
[0032] The labeling module is used to input the front vehicle image into a preset target recognition model to obtain wheel labeling information output by the target recognition model; wherein the wheel labeling information is used to label the area where the front vehicle wheels are located in the front vehicle image.
[0033] Optionally, the size determination module includes:
[0034] A matching submodule, for matching the area where the front vehicle wheel is located in the front vehicle image with the front vehicle point cloud information based on the wheel annotation information, and determining the point cloud information corresponding to the front vehicle wheel in the front vehicle point cloud information;
[0035] The size determination submodule is used to determine the size information of the front wheel and the water surface height information of the front wheel exposed above the accumulated water surface based on the point cloud information corresponding to the front wheel.
[0036] Optionally, the matching submodule includes:
[0037] a matching unit, configured to match the area where the wheels of the preceding vehicle are located in the preceding vehicle image with the preceding vehicle point cloud information based on the wheel annotation information, and determine the to-be-confirmed point cloud information corresponding to the wheels of the preceding vehicle in the preceding vehicle point cloud information;
[0038] The abnormality removal unit is used to remove abnormal values in the point cloud information to be confirmed to obtain the target point cloud information corresponding to the wheel of the front vehicle.
[0039] Optionally, the device further comprises:
[0040] A depth acquisition module, used to acquire a number of road surface water depths predicted within a preset time period;
[0041] The change prediction module is used to predict the change of road surface water depth based on the plurality of road surface water depths within a preset time period.
[0042] Optionally, the depth acquisition module includes:
[0043] The fitting submodule is used to perform curve fitting with time as the independent variable and road surface water depth as the dependent variable to obtain a water depth variation curve;
[0044] The depth acquisition submodule is used to predict the change of road surface water depth based on the water depth change curve.
[0045] Optionally, the change prediction module includes:
[0046] A comparison submodule, for comparing the road surface water depth obtained at the current moment with the road surface water depth obtained at the previous moment;
[0047] The abnormal data determination submodule is used to determine the road surface water depth obtained at the current moment as abnormal data if the deviation between the road surface water depth obtained at the current moment and the road surface water depth obtained at the previous moment is greater than a preset threshold;
[0048] A normal data determination submodule, for determining the road surface water depth obtained at the current moment as normal data if the deviation between the road surface water depth obtained at the current moment and the road surface water depth obtained at the previous moment is not greater than a preset threshold;
[0049] The retaining submodule is used to retain normal data within a preset time period to obtain a plurality of road surface water depths preset within the preset time period.
[0050] The embodiment of the present invention further discloses an electronic device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;
[0051] The memory is used to store computer programs;
[0052] The processor is used to implement the method described in the embodiment of the present invention when executing the program stored in the memory.
[0053] The embodiment of the present invention further discloses one or more computer-readable media on which instructions are stored. When executed by one or more processors, the processors are enabled to execute the method described in the embodiment of the present invention.
[0054] The embodiments of the present invention include the following advantages:
[0055] Through the method for predicting the depth of waterlogging on the road in the embodiment of the present invention, the image of the front vehicle captured by the monocular camera in the vehicle and the point cloud information of the front vehicle captured by the binocular camera in the vehicle are obtained; based on the image of the front vehicle and the point cloud information of the front vehicle, the size information of the front vehicle wheel and the water surface height information of the front vehicle wheel exposed on the surface of the accumulated water are determined; based on the water surface height information and the size information of the front vehicle wheel, the depth of waterlogging on the road is determined. Therefore, the depth of waterlogging on the road can be quickly analyzed by collecting the image of the front vehicle for analysis, and it can have high accuracy and real-time performance. BRIEF DESCRIPTION OF THE DRAWINGS
[0056] Figure 1 is a schematic diagram of a vehicle detecting road surface water depth provided in an embodiment of the present invention;
[0057] Figure 2 is a flowchart of a method for predicting road surface water depth provided in an embodiment of the present invention;
[0058] Figure 3 is a flowchart of another method for predicting road surface water depth provided in an embodiment of the present invention;
[0059] Figure 4 It is a structural block diagram of a device for predicting road surface water depth provided in an embodiment of the present invention;
[0060] Figure 5 is a block diagram of an electronic device provided in an embodiment of the present invention;
[0061] Figure 6 is a schematic diagram of a computer-readable medium provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0062] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.
[0063] Reference Figure 1, showing a schematic diagram of a vehicle detecting the depth of water on a road provided in an embodiment of the present invention. As shown in the figure, the vehicle in the embodiment of the present invention may be provided with a monocular camera and a binocular camera, and the monocular camera and the binocular camera may be used to collect images in front of the vehicle, so that when water accumulates in front of the vehicle and the vehicle in front is wading, an image of the vehicle in front that is wading may be collected.
[0064] Reference Figure 2 , shows a flowchart of a method for predicting road surface water depth provided in an embodiment of the present invention, which may specifically include the following steps:
[0065] Step 201, obtaining a front vehicle image captured by a monocular camera in the vehicle, and a front vehicle point cloud information captured by a binocular camera in the vehicle;
[0066] In an embodiment of the present invention, the vehicle can collect information in front of the vehicle through its equipped monocular camera and binocular camera. Therefore, when water accumulates in front of the vehicle and the vehicle in front is wading, the vehicle obtains the image of the vehicle in front collected by its monocular camera and the point cloud information of the vehicle in front collected by the binocular camera. The vehicle in front collected by the vehicle can be the first vehicle in front of the vehicle or several vehicles in front of the vehicle, and the present invention does not limit this.
[0067] The vehicle's binocular camera may be an RGB binocular depth camera, so that the vehicle can obtain depth information in a single frame image, so as to calculate the depth of road surface water in real time and efficiently.
[0068] Step 202, based on the front vehicle image and the front vehicle point cloud information, determine the size information of the front vehicle wheel and the water surface height information of the front vehicle wheel exposed above the accumulated water surface;
[0069] After obtaining the image of the front vehicle and the point cloud information of the front vehicle, the area where the wheels are located in the image of the front vehicle can be determined based on the image of the front vehicle. Thereafter, the point cloud information related to the wheels of the front vehicle in the point cloud information of the front vehicle can be further determined based on the area where the vehicle is located in the image of the front vehicle. After obtaining the point cloud information related to the wheels of the front vehicle, the size of the wheels of the front vehicle can be calculated based on the point cloud information to obtain the size information. At the same time, the height of the wheels of the front vehicle exposed above the surface of the accumulated water can be analyzed to obtain the water surface height information of the wheels of the front vehicle exposed above the surface of the accumulated water.
[0070] Step 203: Determine the depth of water on the road based on the water surface height information and the size information of the wheels of the front vehicle.
[0071] When the water surface height information and the wheel size information of the front vehicle are known, since the wheels are usually round, the wheel diameter and the height of the wheel exposed above the water surface are known, the height of the wheel covered by the water, that is, the water depth of the road surface, can be calculated.
[0072] Through the method for predicting the depth of waterlogging on the road in the embodiment of the present invention, the image of the front vehicle captured by the monocular camera in the vehicle and the point cloud information of the front vehicle captured by the binocular camera in the vehicle are obtained; based on the image of the front vehicle and the point cloud information of the front vehicle, the size information of the front vehicle wheel and the water surface height information of the front vehicle wheel exposed on the surface of the accumulated water are determined; based on the water surface height information and the size information of the front vehicle wheel, the depth of waterlogging on the road is determined. Therefore, the depth of waterlogging on the road can be quickly analyzed by collecting the image of the front vehicle for analysis, and it can have high accuracy and real-time performance.
[0073] Reference Figure 3 , shows a flowchart of a method for predicting road surface water depth provided in an embodiment of the present invention, which may specifically include the following steps:
[0074] Step 301, obtaining a front vehicle image captured by a monocular camera in the vehicle, and a front vehicle point cloud information captured by a binocular camera in the vehicle;
[0075] In an embodiment of the present invention, the vehicle can collect information in front of the vehicle through its equipped monocular camera and binocular camera. Therefore, when water accumulates in front of the vehicle and the vehicle in front is wading, the vehicle obtains the image of the vehicle in front collected by its monocular camera and the point cloud information of the vehicle in front collected by the binocular camera. The vehicle in front collected by the vehicle can be the first vehicle in front of the vehicle or several vehicles in front of the vehicle, and the present invention does not limit this.
[0076] The vehicle's binocular camera may be an RGB binocular depth camera, so that the vehicle can obtain depth information in a single frame image, so as to calculate the depth of road surface water in real time and efficiently.
[0077] In the specific implementation, in order to improve the accuracy of the point cloud information, after obtaining the left and right images captured by the binocular camera, the images can be corrected based on the known camera distortion parameters. Afterwards, stereo matching is performed so that the same pixel in the left and right images is located on the same horizontal line. The depth information of the point is calculated by the difference between pixels, and the actual coordinates of the point in three-dimensional space are further calculated in combination with the known camera calibration parameters to obtain the point cloud information. Afterwards, the point cloud information can also be denoised, and then the depth distance between points can be interpolated by an interpolation algorithm to improve the accuracy.
[0078] Step 302: input the preceding vehicle image into a preset target recognition model to obtain wheel labeling information output by the target recognition model; wherein the wheel labeling information is used to label the area where the preceding vehicle wheels are located in the preceding vehicle image.
[0079] Specifically, in order to determine the area where the wheels of the front vehicle are located in the front vehicle image captured by the monocular camera, a pre-trained target recognition model can be used to recognize the wheels in the front vehicle image, so as to obtain the wheel annotation information output by the target recognition model. The wheel annotation information can be used to annotate the area where the wheels of the front vehicle are located in the front vehicle image. For example, a box can be used to annotate the area where the wheels of the front vehicle are located, and the coordinate range of the area where the wheels of the front vehicle are located in the front vehicle image can be annotated.
[0080] In a specific implementation, the front vehicle image can be preprocessed according to actual needs. For example, in order to improve the processing efficiency of the target recognition model, the front vehicle image can be cropped, the resolution can be adjusted, grayed, filtered, etc. The target recognition model can use semantic segmentation or instance segmentation to identify the area where the front vehicle wheels are located in the front vehicle image, and the present invention does not limit this.
[0081] Step 303, based on the wheel annotation information, matching the area where the front vehicle wheel is located in the front vehicle image with the front vehicle point cloud information, and determining the point cloud information corresponding to the front vehicle wheel in the front vehicle point cloud information;
[0082] Specifically, after obtaining the wheel annotation information, the area where the front vehicle wheel is located in the front vehicle image can be matched with the front vehicle point cloud information based on the wheel annotation information to identify the point cloud information belonging to the front vehicle wheel in the front vehicle point cloud information. Specifically, based on the wheel annotation information, it can be determined that part of the point cloud information of the front vehicle point cloud information includes the point cloud information of the front vehicle wheel, and in the part of the point cloud information, further analysis can be performed to determine the point cloud information corresponding to the front vehicle wheel.
[0083] In one embodiment of the present invention, the step of matching the area where the front vehicle wheel is located in the front vehicle image with the front vehicle point cloud information based on the wheel annotation information, and determining the point cloud information corresponding to the front vehicle wheel in the front vehicle point cloud information includes:
[0084] S11, based on the wheel annotation information, matching the area where the front vehicle wheel is located in the front vehicle image with the front vehicle point cloud information, and determining the to-be-confirmed point cloud information corresponding to the front vehicle wheel in the front vehicle point cloud information;
[0085] In a specific implementation, since the point cloud information of the front vehicle wheels obtained by matching the area where the front vehicle wheels in the front vehicle image are located with the point cloud information of the front vehicle may contain abnormal points, the point cloud information obtained by matching the area where the front vehicle wheels in the front vehicle image are located with the point cloud information of the front vehicle can be first used as the point cloud information to be confirmed corresponding to the front vehicle wheels, so as to further eliminate abnormalities.
[0086] S12, removing abnormal values from the point cloud information to be confirmed, and obtaining target point cloud information corresponding to the wheels of the front vehicle.
[0087] Specifically, after obtaining the point cloud information to be confirmed corresponding to the front vehicle wheel, for a certain point, if it is obviously deviated from other points, the point can be considered as an outlier. In order to prevent the outlier from affecting the subsequent vehicle wheel size calculation and water surface height information calculation, the outlier can be removed from the point cloud information to be confirmed to obtain the target point cloud information corresponding to the front vehicle wheel.
[0088] Step 304, determining the size information of the front wheel and the water surface height information of the front wheel exposed above the accumulated water surface based on the point cloud information corresponding to the front wheel of the vehicle in front;
[0089] After obtaining the point cloud information related to the front vehicle wheel, since the point cloud information records the three-dimensional spatial coordinates of multiple points on the front vehicle wheel, the size of the front vehicle wheel can be calculated based on the point cloud information to obtain the size information. At the same time, the height of the front vehicle wheel exposed above the water surface can be analyzed to obtain the water surface height information of the front vehicle wheel exposed above the water surface.
[0090] Step 305: Determine the depth of water on the road based on the water surface height information and the size information of the wheels of the front vehicle.
[0091] When the water surface height information and the wheel size information of the front vehicle are known, since the wheels are usually round, the wheel diameter and the height of the wheel exposed above the water surface are known, the height of the wheel covered by the water, that is, the water depth of the road surface, can be calculated.
[0092] In the specific implementation, since the road surface water depth is estimated based on the water surface height information and the size information of the front vehicle's wheels, there may be a certain deviation. Therefore, after the road surface water depth is calculated, the calculated road surface water depth can be corrected using a preset water depth correction coefficient to obtain a target road surface water depth with higher accuracy.
[0093] The water accumulation depth correction coefficient may be calculated based on the deviation between the actual road water accumulation depth previously tested and the road water accumulation depth calculated using the method of the embodiment of the present invention.
[0094] In one embodiment of the present invention, the method further comprises:
[0095] S21, obtaining a number of road surface water depths predicted within a preset time period;
[0096] In the embodiment of the present invention, since the depth of water on the road is usually variable, it may first gradually deepen and then gradually become shallower, or it may first deepen gently and then deepen sharply and then become shallower, etc. Therefore, the depth of water on the road is obtained based only on the analysis of the front vehicle image and the front vehicle point cloud information obtained once, and it is impossible to fully understand the water accumulation situation ahead. Therefore, in order to facilitate the user to better perceive the depth of water on the road ahead, the monocular camera and the binocular camera can continuously collect the front vehicle image and point cloud information in real time, and analyze the front vehicle image and point cloud information obtained at each moment, and calculate the depth of water on the road, so that several road surface water depths predicted within a preset time length can be obtained.
[0097] In an embodiment of the present invention, the step of obtaining a plurality of road surface water depths predicted within a preset time period includes:
[0098] S211, comparing the road surface water depth obtained at the current moment with the road surface water depth obtained at the previous moment;
[0099] Specifically, since the road waterlogging depth is predicted, there may be abnormal predictions. Therefore, in order to avoid adverse effects on the prediction of road waterlogging depth changes, after obtaining the road waterlogging depth at the current moment, the road waterlogging depth obtained at the current moment can be compared with the road waterlogging depth obtained at the previous moment to determine whether the current road waterlogging depth is abnormal.
[0100] S212, if the deviation between the road surface water depth obtained at the current moment and the road surface water depth obtained at the previous moment is greater than a preset threshold, the road surface water depth obtained at the current moment is regarded as abnormal data;
[0101] S213, if the deviation between the road surface water depth obtained at the current moment and the road surface water depth obtained at the previous moment is not greater than a preset threshold, the road surface water depth obtained at the current moment is regarded as normal data;
[0102] Specifically, it is possible to determine whether the road surface water depth obtained at the current moment is normal data by analyzing whether the deviation between the road surface water depth obtained at the current moment and the road surface water depth obtained at the previous moment is greater than a preset threshold. In the case where the deviation between the road surface water depth obtained at the current moment and the road surface water depth obtained at the previous moment is greater than the preset threshold, it can be considered that the road surface water depth obtained at the current moment may not be the accurate road surface water depth, and the road surface water depth obtained at the current moment is regarded as abnormal data. If the deviation between the road surface water depth obtained at the current moment and the road surface water depth obtained at the previous moment is not greater than the preset threshold, the road surface water depth obtained at the current moment can be regarded as normal data.
[0103] S214, retaining normal data within a preset time period to obtain a plurality of road surface water depths preset within the preset time period.
[0104] Specifically, the vehicle can store normal data to obtain a number of road surface water depths preset within a preset time period. Abnormal data can be discarded immediately, and normal data exceeding the preset time period can be deleted. In this way, only normal data within the preset time period can be retained to avoid occupying the vehicle's storage space.
[0105] S22: predicting a change in road surface water depth based on the plurality of road surface water depths within a preset time period.
[0106] Afterwards, the depths of several road floodings obtained within a preset time period can be analyzed to predict the change in road flooding depths, thereby predicting the maximum possible depth of road flooding and promptly reminding users to avoid being affected by water accumulation.
[0107] In an embodiment of the present invention, the step of predicting the change of road surface water depth based on the plurality of road surface water depths within a preset time period includes:
[0108] S31, taking time as an independent variable and road surface water depth as a dependent variable, curve fitting is performed to obtain a water depth variation curve;
[0109] Specifically, time can be used as the independent variable and road waterlogging depth as the dependent variable. Several road waterlogging depths obtained within a preset time period can be displayed using a two-dimensional coordinate system. Subsequently, curve fitting is performed in the two-dimensional coordinate system to obtain a waterlogging depth change curve.
[0110] In a specific implementation, based on the changes in water depth obtained from historical tests, a regression model type suitable for analyzing water depth changes can be selected, and regression parameters can be estimated based on the water depths of several roads within a preset time period. A regression model can be established and the regression model can be verified. When the verification error meets the requirements, the regression model can be used as the target regression model.
[0111] S32: predicting the change of road surface water depth based on the water depth change curve.
[0112] After the water accumulation depth change curve is obtained, the subsequent change of the water accumulation depth can be predicted based on the water accumulation depth change curve, thereby realizing the prediction of the change of the road surface water accumulation depth.
[0113] In the specific implementation, after the regression model is established, the change of road surface water depth can be predicted based on the regression model. Afterwards, a data interface can be configured to encode the image data collected by the vehicle, high-precision point cloud information, predicted road surface water depth and road surface water depth change curve data and send them to the vehicle or the user's mobile client for real-time display. At the same time, the predicted road surface water depth and road surface water depth change curve can also be sent to the planning decision module in the vehicle system to assist the vehicle's automatic driving decision.
[0114] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.
[0115] Reference Figure 4 , shows a structural block diagram of a road surface water depth device provided in an embodiment of the present invention, which is applied to a vehicle, the vehicle includes a monocular camera and a binocular camera, and specifically may include the following modules:
[0116] An acquisition module 401 is used to acquire the image of the preceding vehicle acquired by the monocular camera in the vehicle and the point cloud information of the preceding vehicle acquired by the binocular camera in the vehicle;
[0117] A size determination module 402, configured to determine size information of the front vehicle wheels and water surface height information of the front vehicle wheels exposed above the accumulated water surface based on the front vehicle image and the front vehicle point cloud information;
[0118] The depth determination module 403 is used to determine the depth of the road surface water based on the water surface height information and the size information of the wheels of the front vehicle.
[0119] Optionally, the device further comprises:
[0120] The labeling module is used to input the front vehicle image into a preset target recognition model to obtain wheel labeling information output by the target recognition model; wherein the wheel labeling information is used to label the area where the front vehicle wheels are located in the front vehicle image.
[0121] Optionally, the size determination module includes:
[0122] A matching submodule, for matching the area where the front vehicle wheel is located in the front vehicle image with the front vehicle point cloud information based on the wheel annotation information, and determining the point cloud information corresponding to the front vehicle wheel in the front vehicle point cloud information;
[0123] The size determination submodule is used to determine the size information of the front wheel and the water surface height information of the front wheel exposed above the accumulated water surface based on the point cloud information corresponding to the front wheel.
[0124] Optionally, the matching submodule includes:
[0125] a matching unit, configured to match the area where the wheels of the preceding vehicle are located in the preceding vehicle image with the preceding vehicle point cloud information based on the wheel annotation information, and determine the to-be-confirmed point cloud information corresponding to the wheels of the preceding vehicle in the preceding vehicle point cloud information;
[0126] The abnormality removal unit is used to remove abnormal values in the point cloud information to be confirmed to obtain the target point cloud information corresponding to the wheel of the front vehicle.
[0127] Optionally, the device further comprises:
[0128] A depth acquisition module, used to acquire a number of road surface water depths predicted within a preset time period;
[0129] The change prediction module is used to predict the change of road surface water depth based on the plurality of road surface water depths within a preset time period.
[0130] Optionally, the depth acquisition module includes:
[0131] The fitting submodule is used to perform curve fitting with time as the independent variable and road surface water depth as the dependent variable to obtain a water depth variation curve;
[0132] The depth acquisition submodule is used to predict the change of road surface water depth based on the water depth change curve.
[0133] Optionally, the change prediction module includes:
[0134] A comparison submodule, for comparing the road surface water depth obtained at the current moment with the road surface water depth obtained at the previous moment;
[0135] The abnormal data determination submodule is used to determine the road surface water depth obtained at the current moment as abnormal data if the deviation between the road surface water depth obtained at the current moment and the road surface water depth obtained at the previous moment is greater than a preset threshold;
[0136] A normal data determination submodule, for determining the road surface water depth obtained at the current moment as normal data if the deviation between the road surface water depth obtained at the current moment and the road surface water depth obtained at the previous moment is not greater than a preset threshold;
[0137] The retaining submodule is used to retain normal data within a preset time period to obtain a plurality of road surface water depths preset within the preset time period.
[0138] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.
[0139] In addition, an embodiment of the present invention further provides an electronic device, such as Figure 5 As shown, it includes a processor 501, a communication interface 502, a memory 503 and a communication bus 504, wherein the processor 501, the communication interface 502, and the memory 503 communicate with each other through the communication bus 504.
[0140] Memory 503, used for storing computer programs;
[0141] The processor 501 is used to execute the program stored in the memory 503, and implements the following steps:
[0142] Obtain the image of the vehicle ahead captured by the monocular camera in the vehicle and the point cloud information of the vehicle ahead captured by the binocular camera in the vehicle;
[0143] Determine, based on the front vehicle image and the front vehicle point cloud information, size information of the front vehicle wheels and water surface height information of the front vehicle wheels exposed above the accumulated water surface;
[0144] The depth of water on the road surface is determined based on the water surface height information and the size information of the wheels of the preceding vehicle.
[0145] Optionally, the method further comprises:
[0146] The front vehicle image is input into a preset target recognition model to obtain wheel marking information output by the target recognition model; wherein the wheel marking information is used to mark the area where the front vehicle wheels are located in the front vehicle image.
[0147] Optionally, the step of determining the size information of the front vehicle wheels and the water surface height information of the front vehicle wheels exposed above the accumulated water surface based on the front vehicle image and the front vehicle point cloud information includes:
[0148] Based on the wheel annotation information, matching the area where the front vehicle wheel is located in the front vehicle image with the front vehicle point cloud information, and determining the point cloud information corresponding to the front vehicle wheel in the front vehicle point cloud information;
[0149] Based on the point cloud information corresponding to the front wheel, the size information of the front wheel and the water surface height information of the front wheel exposed above the accumulated water surface are determined.
[0150] Optionally, the step of matching the area where the front vehicle wheel is located in the front vehicle image with the front vehicle point cloud information based on the wheel annotation information, and determining the point cloud information corresponding to the front vehicle wheel in the front vehicle point cloud information includes:
[0151] Based on the wheel annotation information, matching the area where the front vehicle wheel is located in the front vehicle image with the front vehicle point cloud information, and determining the to-be-confirmed point cloud information corresponding to the front vehicle wheel in the front vehicle point cloud information;
[0152] Outliers are removed from the point cloud information to be confirmed to obtain target point cloud information corresponding to the wheels of the front vehicle.
[0153] Optionally, the method further comprises:
[0154] Obtaining several road surface water depths predicted within a preset time period;
[0155] Based on the plurality of road surface water depths within a preset time period, a change in road surface water depth is predicted.
[0156] Optionally, the step of predicting the change of road surface water depth based on the plurality of road surface water depths within a preset time period includes:
[0157] Taking time as the independent variable and road surface water depth as the dependent variable, curve fitting is performed to obtain the water depth variation curve;
[0158] Based on the water depth change curve, predict the change of road water depth.
[0159] Optionally, the step of obtaining a plurality of road surface water depths predicted within a preset time period includes:
[0160] For the road surface water depth obtained at the current moment, the road surface water depth obtained at the current moment is compared with the road surface water depth obtained at the previous moment;
[0161] If the deviation between the road surface water depth obtained at the current moment and the road surface water depth obtained at the previous moment is greater than a preset threshold, the road surface water depth obtained at the current moment is regarded as abnormal data;
[0162] If the deviation between the road surface water depth obtained at the current moment and the road surface water depth obtained at the previous moment is not greater than the preset threshold, the road surface water depth obtained at the current moment is regarded as normal data;
[0163] Normal data within a preset time period is retained to obtain a number of road surface water depths preset within the preset time period.
[0164] The communication bus mentioned in the above terminal can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.
[0165] The communication interface is used for communication between the above terminal and other devices.
[0166] The memory may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.
[0167] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
[0168] like Figure 6As shown, in another embodiment provided by the present invention, a computer-readable storage medium 601 is also provided, in which instructions are stored. When the computer-readable storage medium is run on a computer, the computer executes the method for predicting the road surface water depth described in the above embodiment.
[0169] In another embodiment of the present invention, a computer program product including instructions is provided. When the computer program product is run on a computer, the computer executes the method for predicting the road surface water depth described in the above embodiment.
[0170] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc.
[0171] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.
[0172] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.
[0173] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.
Claims
1. A method for predicting road surface water depth, It is characterized in that The method is applied to a vehicle, wherein the vehicle includes a monocular camera and a binocular camera, and the method includes: Obtain the image of the vehicle ahead captured by the monocular camera in the vehicle and the point cloud information of the vehicle ahead captured by the binocular camera in the vehicle; Determine, based on the front vehicle image and the front vehicle point cloud information, size information of the front vehicle wheels and water surface height information of the front vehicle wheels exposed above the accumulated water surface; Determining the depth of water on the road based on the water surface height information and the size information of the wheels of the front vehicle; Wherein, the method further comprises: Inputting the front vehicle image into a preset target recognition model to obtain wheel marking information output by the target recognition model; wherein the wheel marking information is used to mark the area where the front vehicle wheels are located in the front vehicle image; Wherein, the step of determining the size information of the front vehicle wheel and the water surface height information of the front vehicle wheel exposed on the surface of the accumulated water based on the front vehicle image and the front vehicle point cloud information includes: Based on the wheel annotation information, matching the area where the front vehicle wheel is located in the front vehicle image with the front vehicle point cloud information, and determining the point cloud information corresponding to the front vehicle wheel in the front vehicle point cloud information; Based on the point cloud information corresponding to the front wheel, the size information of the front wheel and the water surface height information of the front wheel exposed above the accumulated water surface are determined.
2. The method according to claim 1, It is characterized in that The step of matching the area where the front vehicle wheel is located in the front vehicle image with the front vehicle point cloud information based on the wheel annotation information, and determining the point cloud information corresponding to the front vehicle wheel in the front vehicle point cloud information includes: Based on the wheel annotation information, matching the area where the front vehicle wheel is located in the front vehicle image with the front vehicle point cloud information, and determining the to-be-confirmed point cloud information corresponding to the front vehicle wheel in the front vehicle point cloud information; Outliers are removed from the point cloud information to be confirmed to obtain target point cloud information corresponding to the wheels of the front vehicle.
3. The method according to claim 1, It is characterized in that The method further comprises: Obtaining several road surface water depths predicted within a preset time period; Based on the plurality of road surface water depths within a preset time period, a change in road surface water depth is predicted.
4. The method according to claim 3, It is characterized in that The step of predicting the change of road surface water depth based on the plurality of road surface water depths within a preset time period includes: Taking time as the independent variable and road surface water depth as the dependent variable, curve fitting is performed to obtain the water depth variation curve; Based on the water depth change curve, predict the change of road water depth.
5. The method according to claim 3, It is characterized in that The step of obtaining a plurality of road surface water depths predicted within a preset time period includes: For the road surface water depth obtained at the current moment, the road surface water depth obtained at the current moment is compared with the road surface water depth obtained at the previous moment; If the deviation between the road surface water depth obtained at the current moment and the road surface water depth obtained at the previous moment is greater than a preset threshold, the road surface water depth obtained at the current moment is regarded as abnormal data; If the deviation between the road surface water depth obtained at the current moment and the road surface water depth obtained at the previous moment is not greater than the preset threshold, the road surface water depth obtained at the current moment is regarded as normal data; Normal data within a preset time period is retained to obtain a number of road surface water depths preset within the preset time period.
6. A device for predicting the depth of road waterlogging, It is characterized in that The device is applied to a vehicle, the vehicle includes a monocular camera and a binocular camera, and the device includes: An acquisition module is used to acquire the image of the preceding vehicle acquired by the monocular camera in the vehicle and the point cloud information of the preceding vehicle acquired by the binocular camera in the vehicle; A size determination module, used to determine the size information of the front vehicle wheel and the water surface height information of the front vehicle wheel exposed on the surface of the accumulated water based on the front vehicle image and the front vehicle point cloud information; A depth determination module, used to determine the depth of water on the road based on the water surface height information and the size information of the wheels of the front vehicle; Wherein, the device further comprises: A labeling module, used for inputting the front vehicle image into a preset target recognition model to obtain wheel labeling information output by the target recognition model; wherein the wheel labeling information is used for labeling the area where the front vehicle wheels are located in the front vehicle image; Wherein, the size determination module includes: A matching submodule, for matching the area where the front vehicle wheel is located in the front vehicle image with the front vehicle point cloud information based on the wheel annotation information, and determining the point cloud information corresponding to the front vehicle wheel in the front vehicle point cloud information; The size determination submodule is used to determine the size information of the front wheel and the water surface height information of the front wheel exposed above the accumulated water surface based on the point cloud information corresponding to the front wheel.
7. An electronic device, It is characterized in that It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; The memory is used to store computer programs; The processor is used to implement the method according to any one of claims 1 to 5 when executing the program stored in the memory.
8. One or more computer-readable media having instructions stored thereon, which, when executed by one or more processors, cause the processors to perform the method according to any one of claims 1 to 7.
