Ponding depth estimation method and apparatus, electronic device, and storage medium
By capturing road surface images with a binocular camera and using 3D point cloud information to determine the slope equation, the problem of water depth estimation relying on high-precision maps and 3D data in existing technologies has been solved, achieving high-precision water depth estimation without prior data.
Patent Information
- Application Number
- PCT/CN2024/086650
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-11
- Filing Date
- 2024-04-08
- Publication Date
- 2024-11-14
AI Technical Summary
Existing technologies rely on high-precision maps or predicted 3D data for water depth estimation, which is quite difficult and cannot effectively estimate water depth without prior data.
By acquiring road surface images captured by binocular cameras on the target vehicle, water accumulation areas are identified, and the slope equation is determined using 3D point cloud information outside the road surface images. The water depth is estimated based on the slope equation, and the water depth is calculated by combining semantic segmentation and elevation fitting methods.
It eliminates the need for high-precision maps and 3D data, reducing the difficulty of estimating water depth and improving the accuracy of estimation. It can accurately estimate water depth even without prior data.
Smart Images

Figure CN2024086650_14112024_PF_FP_ABST
Abstract
Description
Water depth estimation method, device, electronic device and storage medium
[0001] CROSS-REFERENCE TO RELATED APPLICATIONS
[0002] This application claims priority to the Chinese patent application filed with the China Patent Office on May 11, 2023, with application number 202310526317.8, and application name “Water Depth Estimation Method, Device, Electronic Device and Storage Medium”, all contents of which are incorporated by reference into this application. Technical Field
[0003] The present application relates to the field of image processing technology, and in particular to a method, device, electronic device and storage medium for estimating water depth. Background Art
[0004] Water on the road is important for vehicle passage, safe operation and courteous driving.
[0005] Due to the reflection of light, the depth of the water surface on the road cannot be directly obtained by general sensors. In current practice, other auxiliary information is often relied on to judge the depth of water and the thickness of water. For example, the depth of water is judged by the water draft of the tires, or high-precision map information of the road surface where the water is located is obtained by relying on precise positioning information, and the depth of water is estimated based on the map information. This method requires high-precision map information of the corresponding road section in advance, and requires precise positioning function during detection. Alternatively, it is necessary to estimate the depth of water with the help of pre-shot 3D data of the road section without water accumulation, and compare it with the 3D data when there is water accumulation to obtain the water depth information. This method requires the 3D information of the road section in advance, which is very restrictive and cannot solve the problem of estimating the depth of water when there is no prior 3D data of water accumulation.
[0006] The above two methods need to rely on information such as high-precision maps or predicted three-dimensional data to estimate the depth of water accumulation, which is difficult to estimate the depth of water accumulation.
[0007] Summary of the Invention
[0008] The embodiments of the present application provide a method, device, electronic device and storage medium for estimating the depth of water accumulation to solve the problem that the existing technology needs to rely on information such as high-precision maps or predicted three-dimensional data to estimate the depth of water accumulation, which makes the estimation of the depth of water accumulation difficult.
[0009] In order to solve the above technical problems, the embodiments of the present application are implemented as follows:
[0010] In a first aspect, an embodiment of the present application provides a method for estimating water depth, the method comprising:
[0011] Obtaining a road surface image of the target vehicle captured by a binocular camera;
[0012] Identify waterlogged areas within road surface images;
[0013] Determine the slope equation corresponding to the road surface based on the three-dimensional point cloud information of the image areas other than the waterlogged area in the road surface image;
[0014] Based on the slope equation, the water depth corresponding to the waterlogged area is estimated.
[0015] Optionally, identifying a waterlogged area in a road surface image includes:
[0016] The semantic segmentation algorithm is used to perform semantic segmentation processing on the road surface image to obtain the boundary information of the waterlogged area;
[0017] Based on the boundary information, the water accumulation area in the road surface image is determined.
[0018] Optionally, determining a slope equation corresponding to the road surface based on three-dimensional point cloud information of image areas other than the waterlogged area in the road surface image includes:
[0019] Obtain the initial three-dimensional point cloud information of each pixel in the road image;
[0020] Eliminate the point cloud information located in the water accumulation area from the initial three-dimensional point cloud information to obtain the three-dimensional point cloud information of other image areas;
[0021] Determine multiple elevation curves of the road surface in the direction of the target vehicle based on the three-dimensional point cloud information;
[0022] The multiple elevation curves are fitted respectively to obtain multiple slope equations corresponding to the multiple elevation curves.
[0023] Optionally, based on the slope equation, the water depth corresponding to the waterlogged area is estimated, including:
[0024] According to the boundary points and slope equation of the waterlogged area, the water depth corresponding to the waterlogged area is estimated.
[0025] Optionally, the water depth corresponding to the waterlogged area is estimated based on the boundary points and the slope equation of the waterlogged area, including:
[0026] Acquire a first height of a reference object in a non-waterlogged area and a second height of a reference object in a waterlogged area within the road surface image;
[0027] Get the height difference between the first height and the second height;
[0028] Based on the height difference, boundary points and slope equation, the water depth corresponding to the waterlogged area is estimated.
[0029] Optionally, after estimating the water depth corresponding to the waterlogged area based on the slope equation, the method further includes:
[0030] When the depth of the accumulated water is greater than the first depth threshold, outputting driving prompt information for the target vehicle to take a detour; or
[0031] When the water depth is less than or equal to the first depth threshold and greater than the second depth threshold, outputting driving prompt information for the target vehicle to decelerate; or
[0032] When the water depth is less than or equal to the second depth threshold, driving prompt information corresponding to the target vehicle for safely passing the waterlogged area is output.
[0033] In a second aspect, an embodiment of the present application provides a device for estimating water depth, the device comprising:
[0034] A road surface image acquisition module is used to acquire a road surface image of the target vehicle captured by a binocular camera.
[0035] A water accumulation area recognition module is used to identify water accumulation areas in road surface images;
[0036] A slope equation determination module is used to determine the slope equation corresponding to the road surface based on the three-dimensional point cloud information of other image areas in the road surface image except the waterlogged area;
[0037] The water depth estimation module is used to estimate the water depth corresponding to the waterlogged area based on the slope equation.
[0038] Optionally, the waterlogged area identification module includes:
[0039] A boundary information acquisition unit is used to perform semantic segmentation processing on the road surface image using a semantic segmentation algorithm to obtain boundary information of the waterlogged area;
[0040] The water accumulation area determination unit is used to determine the water accumulation area in the road surface image based on the boundary information.
[0041] Optionally, the slope equation determination module includes:
[0042] An initial point cloud acquisition unit, used to acquire initial three-dimensional point cloud information of each pixel point in the road surface image;
[0043] A three-dimensional point cloud acquisition unit is used to eliminate point cloud information located in the water accumulation area from the initial three-dimensional point cloud information to obtain three-dimensional point cloud information of other image areas;
[0044] an elevation curve determination unit, configured to determine a plurality of elevation curves of the road surface in the direction of the target vehicle according to the three-dimensional point cloud information;
[0045] The slope equation obtaining unit is used to perform fitting processing on multiple elevation curves respectively to obtain multiple slope equations corresponding to the multiple elevation curves.
[0046] Optionally, the water depth estimation module includes:
[0047] The water depth estimation unit is used to estimate the water depth corresponding to the water accumulation area according to the boundary points and slope equation of the water accumulation area.
[0048] Optionally, the water depth estimation unit includes:
[0049] A reference object height acquisition subunit, configured to acquire a first height of a reference object in a non-waterlogged area within the road surface image, and a second height of a reference object in a waterlogged area;
[0050] A height difference obtaining subunit, configured to obtain a height difference between a first height and a second height;
[0051] The water depth estimation subunit is used to estimate the water depth corresponding to the waterlogged area based on the height difference, boundary points and slope equation.
[0052] Optionally, the device further comprises:
[0053] A first prompt information output module is used to output driving prompt information corresponding to the target vehicle for detour driving when the depth of the accumulated water is greater than a first depth threshold;
[0054] The second prompt information output module is used to output driving prompt information corresponding to the target vehicle to slow down when the water depth is less than or equal to the first depth threshold and greater than the second depth threshold;
[0055] The third prompt information output module is used to output driving prompt information corresponding to the target vehicle for safely passing the flooded area when the water depth is less than or equal to the second depth threshold.
[0056] In a third aspect, an embodiment of the present application provides an electronic device, including:
[0057] A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, any of the above methods for estimating water depth is implemented.
[0058] In a fourth aspect, an embodiment of the present application provides a readable storage medium. When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device can execute any of the above-mentioned methods for estimating water depth.
[0059] In an embodiment of the present application, a road surface image of the target vehicle is captured by a binocular camera, and the waterlogged area within the road surface image is identified. Based on the three-dimensional point cloud information of the image areas other than the waterlogged area within the road surface image, the slope equation corresponding to the road surface is determined. Based on the slope equation, the water depth corresponding to the waterlogged area is estimated. The embodiment of the present application obtains accurate point cloud information of the current road surface without waterlogging through the RGB image captured by the binocular camera, and estimates the water depth through the elevation fitting method, without relying on high-precision maps, predicted three-dimensional data, and other information, thereby reducing the difficulty of estimating the water depth.
[0060] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS
[0061] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0062] FIG1 is a flowchart of a method for estimating water depth according to an embodiment of the present application;
[0063] FIG2 is a flowchart of a method for determining a waterlogged area according to an embodiment of the present application;
[0064] FIG3 is a flowchart of a method for obtaining a slope equation according to an embodiment of the present application;
[0065] FIG4 is a flowchart of a method for estimating water depth provided in an embodiment of the present application;
[0066] FIG5 is a flowchart of another method for estimating water depth provided in an embodiment of the present application;
[0067] FIG6 is a flowchart of a method for outputting driving prompt information provided by an embodiment of the present application;
[0068] FIG7 is a schematic diagram of a water accumulation area provided in an embodiment of the present application;
[0069] FIG8 is a schematic diagram of an elevation curve provided in an embodiment of the present application;
[0070] FIG9 is a schematic diagram of a water depth estimation method provided by an embodiment of the present application;
[0071] FIG10 is a schematic diagram of an auxiliary water depth estimation method provided by an embodiment of the present application;
[0072] FIG11 is a schematic structural diagram of a device for estimating water depth provided in an embodiment of the present application;
[0073] FIG12 is a schematic structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0074] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.
[0075] 1 , a flowchart of a method for estimating water depth according to an embodiment of the present invention is shown. As shown in FIG1 , the method for estimating water depth may include: step 101 , step 102 , step 103 , and step 104 .
[0076] Step 101: Obtain a road surface image of the target vehicle captured by a binocular camera.
[0077] The embodiments of the present application can be applied to scenarios where the depth of road surface water is estimated by combining RGB images of the road surface taken by a binocular camera and the elevation fitting method.
[0078] In this embodiment, the binocular cameras are all RGB cameras. R, G, and B are abbreviations for red, green, and blue, respectively.
[0079] It is understandable that, generally speaking, water accumulation on roads is mainly caused by the presence of a road slope, which is characterized by a generally gradual and continuous road slope. Estimating the depth of water accumulation due to abnormal factors such as potholes is not within the scope of discussion of the embodiments of this application.
[0080] When the target vehicle encounters water in front of it while driving, the binocular camera on the target vehicle can be activated to capture the road image of the road where the target vehicle is located. The road image is an RGB image.
[0081] After acquiring the road surface image of the target vehicle captured by the binocular camera on the target vehicle, step 102 is executed.
[0082] Step 102: Identify the waterlogged area in the road surface image.
[0083] After acquiring a road image of the target vehicle's road surface captured by a binocular camera, waterlogged areas within the road image, i.e., waterlogged image regions, can be identified. In this example, deep learning semantic segmentation algorithms can be used to perform semantic segmentation on the road image to identify the boundaries of waterlogged areas within the road image. The waterlogged areas within the road image can then be determined based on this boundary information. This implementation process is described in detail below with reference to Figure 2.
[0084] 2 , which shows a flow chart of steps of a method for determining a waterlogged area provided by an embodiment of the present application. As shown in FIG2 , the method for determining a waterlogged area may include: step 201 and step 202 .
[0085] Step 201: Using a semantic segmentation algorithm to perform semantic segmentation processing on the road surface image to obtain boundary information of the waterlogged area.
[0086] In this embodiment, after the road surface image is acquired, a semantic segmentation algorithm may be used to perform semantic segmentation processing on the road surface image to obtain boundary information of the waterlogged area.
[0087] After semantic segmentation processing is performed on the road surface image using a semantic segmentation algorithm to obtain boundary information of the waterlogged area, step 202 is executed.
[0088] Step 202: Determine the water accumulation area in the road surface image based on the boundary information.
[0089] After obtaining the boundary information of the waterlogged area, the waterlogged area within the road image can be determined based on this boundary information. As shown in Figure 7, the water surface is perceived based on the binocular RGB image. The perception algorithm uses semantic segmentation to obtain the boundary of the waterlogged area in the image.
[0090] The embodiment of the present application uses a deep learning semantic segmentation algorithm to identify waterlogged areas in road surface images, which can improve the accuracy of identifying waterlogged areas.
[0091] After the water accumulation area in the road surface image is identified, step 103 is executed.
[0092] Step 103: Determine the slope equation corresponding to the road surface based on the three-dimensional point cloud information of the image areas other than the waterlogged area in the road surface image.
[0093] In practice, the left and right RGB views (i.e., road images) captured by a binocular camera can be used to exploit the principle of parallax. Through deep learning and other methods, a disparity map of each pixel within the image field of view of the left and right RGB views can be obtained. Furthermore, the 3D point cloud information within the image field of view can be calculated based on the intrinsic parameters of the binocular camera.
[0094] After identifying the waterlogged area within the road surface image, 3D point cloud information can be obtained for other image regions within the road surface image, excluding the waterlogged area. Based on this 3D point cloud information, the slope equation for the road surface corresponding to the road surface image can be determined. Specifically, multiple elevation curves for the road surface can be determined based on the 3D point cloud information for the other image regions. Combining these multiple elevation curves can yield multiple slope equations. This implementation process is described in detail below with reference to Figure 3.
[0095] 3 , a flowchart of a method for obtaining a slope equation according to an embodiment of the present application is shown. As shown in FIG3 , the method for obtaining a slope equation may include: step 301 , step 302 , step 303 , and step 304 .
[0096] Step 301: Obtain initial three-dimensional point cloud information of each pixel in the road surface image.
[0097] In this embodiment, after the waterlogged area in the road surface image is identified, initial three-dimensional point cloud information of each pixel in the road surface image may be obtained.
[0098] After obtaining the initial three-dimensional point cloud information of each pixel in the road surface image, step 302 is executed.
[0099] Step 302: Eliminate the point cloud information located in the water accumulation area from the initial three-dimensional point cloud information to obtain three-dimensional point cloud information of other image areas.
[0100] After obtaining the initial three-dimensional point cloud information of each pixel in the road surface image, the point cloud information located in the waterlogged area in the initial three-dimensional point cloud information can be eliminated to obtain the three-dimensional point cloud information of other image areas.
[0101] After acquiring the three-dimensional point cloud information in other image areas, step 303 is executed.
[0102] Step 303: Determine multiple elevation curves of the road surface in the direction of the target vehicle based on the three-dimensional point cloud information.
[0103] After acquiring the 3D point cloud information within other image regions, multiple elevation curves of the road surface in the direction of the target vehicle can be determined based on the 3D point cloud information within the other image regions. Specifically, the 3D point cloud information can be evaluated along the direction of the target vehicle to obtain multiple elevation curves along the vehicle's direction. As shown in Figures 7 and 8 , evaluating the 3D point cloud information along line 1 in Figure 7 yields elevation curve 1 in Figure 8 , evaluating the 3D point cloud information along line 2 in Figure 7 yields elevation curve 2 in Figure 8 , and so on.
[0104] It can be understood that the above examples are merely examples listed for a better understanding of the technical solutions of the embodiments of the present application, and are not intended to be the sole limitation on the embodiments.
[0105] After a plurality of elevation curves of the road surface in the direction of the target vehicle are determined based on the three-dimensional point cloud information, step 304 is executed.
[0106] Step 304: performing fitting processing on the multiple elevation curves respectively to obtain multiple slope equations corresponding to the multiple elevation curves.
[0107] After determining a plurality of elevation curves of the road surface in the body direction of the target vehicle based on the three-dimensional point cloud information, the plurality of elevation curves may be fitted to obtain a plurality of slope equations corresponding to the plurality of elevation curves.
[0108] Understandably, due to the presence of accumulated water, the elevation curve contains discrete elevation points. Once an elevation curve is obtained, the discrete elevation points on the elevation curve can be fitted to obtain the slope equation corresponding to the elevation curve. In this example, the slope equation can be expressed using a polynomial such as Z = ay^3 + by^2 + cy + d, where Z is the height and y is the y-axis value corresponding to the vehicle body direction.
[0109] After determining the slope equation corresponding to the road surface based on the three-dimensional point cloud information of the image areas other than the waterlogged area in the road surface image, step 104 is executed.
[0110] Step 104: Based on the slope equation, estimate the water depth corresponding to the waterlogged area.
[0111] After determining the slope equation corresponding to the road surface, the water depth corresponding to the waterlogged area can be estimated based on the slope equation. The water depth estimation process can be described in detail below with reference to FIG4 .
[0112] 4 , there is shown a flowchart of a method for estimating water depth provided by an embodiment of the present application. As shown in FIG4 , the method for estimating water depth may include: step 401 .
[0113] Step 401: Estimate the water depth corresponding to the waterlogged area based on the boundary points and slope equation of the waterlogged area.
[0114] In this embodiment, after obtaining the slope equation, the water depth of the waterlogged area can be estimated based on the boundary points of the waterlogged area and the slope equation. As shown in Figure 9, the heights of the two boundaries of the waterlogged area are 200 and 300 respectively. The slope equation can be used to determine the height of each point in the waterlogged area, thereby estimating the water depth.
[0115] The embodiment of the present application obtains accurate point cloud information of the current road surface without water accumulation through RGB images taken by a binocular camera, and estimates the depth of water accumulation through the elevation fitting method. It does not need to rely on high-precision maps and predicted three-dimensional data and other information, thereby reducing the difficulty of estimating the depth of water accumulation.
[0116] In the embodiment of the present application, the water depth can be estimated by combining information of other objects on the road, such as curbs, fences, etc. The implementation process of the auxiliary water depth estimation can be described in detail below with reference to FIG5 .
[0117] 5 , there is shown a flowchart of another method for estimating water depth provided by an embodiment of the present application. As shown in FIG5 , the method for estimating water depth may include: step 501 , step 502 , and step 503 .
[0118] Step 501: Acquire a first height of a reference object in a non-waterlogged area and a second height of a reference object in a waterlogged area in a road surface image.
[0119] In this embodiment, the reference object may be a curb, a fence, or other reference object. The type of the reference object may be determined according to actual conditions and is not limited in this embodiment.
[0120] The first height refers to the height of the reference object in a non-waterlogged area in the road surface image.
[0121] The second height refers to the height of a reference object located in a flooded area within the road surface image.
[0122] In a specific implementation, when estimating the depth of water in a flooded area, a first height of a reference object in a non-waterlogged area within a road image and a second height of a reference object in a flooded area can be obtained. For example, taking a curb as an example, the height of the curb in the non-waterlogged area within the road image can be obtained based on the road image, i.e., the first height, which is the height between the curb and the road surface. The height of the curb in the flooded area within the road image can also be obtained, i.e., the second height, which is the height between the curb and the water surface, and so on.
[0123] It can be understood that the above examples are merely examples listed for a better understanding of the technical solutions of the embodiments of the present application, and are not intended to be the sole limitation on the embodiments.
[0124] After obtaining the first height of the reference object in the non-waterlogged area and the second height of the reference object in the waterlogged area in the road surface image, step 502 is executed.
[0125] Step 502: Obtain the height difference between the first height and the second height.
[0126] After obtaining the first height of the reference object in the non-waterlogged area and the second height of the reference object in the waterlogged area within the road image, the height difference between the first and second heights can be calculated. As shown in Figure 10, taking a curb as an example, the first height of the curb in the non-waterlogged area is 15 cm (centimeter), while the second height of the curb in the waterlogged area is 10 cm. In this case, the calculated height difference is 5 cm, etc.
[0127] After the height difference between the first height and the second height is acquired, step 503 is executed.
[0128] Step 503: Based on the height difference, the boundary points and the slope equation, the water depth corresponding to the waterlogged area is estimated.
[0129] After obtaining the height difference between the first height and the second height, the water depth corresponding to the waterlogged area can be estimated based on the height difference, the boundary point, and the slope equation.
[0130] In one specific implementation of the present application, the initial predicted water depth of the flooded area can be estimated using boundary points and a slope equation. The initial predicted water depth can then be calibrated based on the height difference to obtain a final estimated water depth. For example, the average of the initial predicted water depth and the height difference can be calculated as the final estimated water depth, or the weighted average of the initial predicted water depth and the height difference can be calculated as the final estimated water depth.
[0131] In another specific implementation of the present application, the initial predicted water depth of the waterlogged area can be estimated first by using the boundary points and the slope equation, and then the accuracy of the initial predicted water depth can be verified based on the height difference. For example, when the difference between the height difference and the initial predicted water depth is within a preset range, the accuracy of the initial predicted water depth can be considered to be high, and the initial predicted water depth can be used as the final estimated water depth. When the difference between the height difference and the initial predicted water depth is not within the preset range, the accuracy of the initial predicted water depth can be considered to be low, and at this time, the RGB image of the road surface at the waterlogged area can be re-photographed to re-estimate the water depth, etc.
[0132] It can be understood that the above two implementation methods are merely examples listed to better understand the technical solutions of the embodiments of the present application, and are not intended to be the sole limitations of this embodiment.
[0133] The embodiment of the present application can improve the accuracy of the estimation of the depth of water accumulation by combining reference objects (such as curbs, fences, etc.) and road facilities to assist in determining the depth of water accumulation.
[0134] In the embodiment of the present application, after estimating the depth of the water, a corresponding driving strategy can be executed according to the depth of the water. Specifically, the implementation process can be described in detail below with reference to FIG6 .
[0135] 6 , a flowchart of a method for outputting driving prompt information according to an embodiment of the present invention is shown. As shown in FIG6 , the method for outputting driving prompt information may include: step 601 , step 602 , and step 603 .
[0136] Step 601: When the depth of the accumulated water is greater than a first depth threshold, outputting driving prompt information for detour driving corresponding to the target vehicle.
[0137] Step 602: When the water depth is less than or equal to the first depth threshold and greater than the second depth threshold, output driving prompt information corresponding to the target vehicle to slow down.
[0138] Step 603: When the water depth is less than or equal to the second depth threshold, output driving prompt information corresponding to the target vehicle for safely passing through the water area.
[0139] In the embodiment of the present application, the first depth threshold and the second water depth threshold are both pre-set water depth thresholds that provide a user with guidance on how to navigate a flooded road section. The first depth threshold is greater than the second depth threshold. The specific values of the first and second depth thresholds can be determined based on business needs and are not limited in this embodiment.
[0140] When the depth of the accumulated water is greater than the first depth threshold, a driving prompt message for the target vehicle to take a detour may be output, that is, the depth of the accumulated water on the road is too deep, driving is dangerous, and the user is prompted to take a detour.
[0141] If the water depth is less than or equal to the first depth threshold and greater than the second depth threshold, a driving prompt message can be outputted for the target vehicle to slow down. This means that the water depth is appropriate and it is advisable to slow down and drive through to avoid water entering the vehicle and causing it to stall.
[0142] When the depth of the water is less than or equal to the second depth threshold, a driving prompt message corresponding to the target vehicle for safely passing through the waterlogged area can be output, that is, the water on the road is relatively shallow and the target vehicle can pass through the flooded road section normally.
[0143] The embodiment of the present application assists users in driving by pre-setting a water depth threshold, thereby improving driving safety.
[0144] The method for estimating the depth of water accumulation provided in the embodiment of the present application obtains a road surface image of the target vehicle captured by a binocular camera, and identifies the water accumulation area within the road surface image. Based on the three-dimensional point cloud information of other image areas in the road surface image except the water accumulation area, the slope equation corresponding to the road surface is determined. Based on the slope equation, the water accumulation depth corresponding to the water accumulation area is estimated. The embodiment of the present application obtains accurate point cloud information of the current road surface without water accumulation through the RGB image captured by the binocular camera, and calculates the water accumulation depth through the elevation fitting method. It does not need to rely on high-precision maps, predicted three-dimensional data and other information, thereby reducing the difficulty of estimating the depth of water accumulation.
[0145] 11, which shows a schematic diagram of the structure of a water depth estimation device provided by an embodiment of the present application. As shown in FIG11, the water depth estimation device 1100 may include the following modules:
[0146] The road surface image acquisition module 1101 is used to acquire a road surface image of the target vehicle captured by a binocular camera.
[0147] A water accumulation area recognition module 1102 is used to recognize water accumulation areas in a road surface image;
[0148] The slope equation determination module 1103 is configured to determine the slope equation corresponding to the road surface based on the three-dimensional point cloud information of the image areas other than the waterlogged area in the road surface image;
[0149] The water depth estimation module 1104 is used to estimate the water depth corresponding to the water accumulation area based on the slope equation.
[0150] Optionally, the waterlogged area identification module includes: a boundary information acquisition unit, configured to perform semantic segmentation processing on the road surface image using a semantic segmentation algorithm to obtain boundary information of the waterlogged area;
[0151] The water accumulation area determination unit is used to determine the water accumulation area in the road surface image based on the boundary information.
[0152] Optionally, the slope equation determination module includes:
[0153] An initial point cloud acquisition unit, used to acquire initial three-dimensional point cloud information of each pixel point in the road surface image;
[0154] A three-dimensional point cloud acquisition unit is used to eliminate point cloud information located in the water accumulation area from the initial three-dimensional point cloud information to obtain three-dimensional point cloud information of other image areas;
[0155] an elevation curve determination unit, configured to determine a plurality of elevation curves of the road surface in the direction of the target vehicle according to the three-dimensional point cloud information;
[0156] The slope equation obtaining unit is used to perform fitting processing on multiple elevation curves respectively to obtain multiple slope equations corresponding to the multiple elevation curves.
[0157] Optionally, the water depth estimation module includes:
[0158] The water depth estimation unit is used to estimate the water depth corresponding to the water accumulation area according to the boundary points and slope equation of the water accumulation area.
[0159] Optionally, the water depth estimation unit includes:
[0160] A reference object height acquisition subunit, configured to acquire a first height of a reference object in a non-waterlogged area within the road surface image, and a second height of a reference object in a waterlogged area;
[0161] A height difference obtaining subunit, configured to obtain a height difference between a first height and a second height;
[0162] The water depth estimation subunit is used to estimate the water depth corresponding to the waterlogged area based on the height difference, boundary points and slope equation.
[0163] Optionally, the device further comprises:
[0164] A first prompt information output module is used to output driving prompt information corresponding to the target vehicle for detour driving when the depth of the accumulated water is greater than a first depth threshold;
[0165] The second prompt information output module is used to output driving prompt information corresponding to the target vehicle to slow down when the water depth is less than or equal to the first depth threshold and greater than the second depth threshold;
[0166] The third prompt information output module is used to output driving prompt information corresponding to the target vehicle for safely passing the flooded area when the water depth is less than or equal to the second depth threshold.
[0167] The device for estimating the depth of water accumulation provided in the embodiment of the present application obtains a road surface image of the target vehicle captured by a binocular camera, and identifies the water accumulation area within the road surface image. Based on the three-dimensional point cloud information of other image areas in the road surface image except the water accumulation area, the slope equation corresponding to the road surface is determined. Based on the slope equation, the water accumulation depth corresponding to the water accumulation area is estimated. The embodiment of the present application obtains accurate point cloud information of the non-waterlogged road surface of the current road through the RGB image captured by the binocular camera, and calculates the water accumulation depth through the elevation fitting method. It does not need to rely on high-precision maps, predicted three-dimensional data and other information, thereby reducing the difficulty of estimating the depth of water accumulation.
[0168] An embodiment of the present application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the above-mentioned method for estimating water depth is implemented.
[0169] Figure 12 shows a schematic structural diagram of an electronic device 1200 according to an embodiment of the present application. As shown in Figure 12, the binocular camera 1300 can provide the electronic device 1200 with a captured road surface image (i.e., an RGB image). The electronic device 1200 includes a central processing unit (CPU) 1201, which can perform various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) 1202 or computer program instructions loaded from a storage unit 1208 into a random access memory (RAM) 1203. Various programs and data required for the operation of the electronic device 1200 can also be stored in RAM 1203. CPU 1201, ROM 1202, and RAM 1203 are connected to each other via a bus 1204. An input / output (I / O) interface 1205 is also connected to the bus 1204.
[0170] Multiple components in the electronic device 1200 are connected to the I / O interface 1205, including an input unit 1206, such as a keyboard, a mouse, a microphone, etc.; an output unit 1207, such as various types of displays, speakers, etc.; a storage unit 1208, such as a magnetic disk, an optical disk, etc.; and a communication unit 1209, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1209 allows the electronic device 1200 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.
[0171] The various processes and procedures described above may be executed by the processing unit 1201. For example, the method of any of the above embodiments may be implemented as a computer software program, which is tangibly contained in a computer-readable medium, such as the storage unit 1208. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 1200 via the ROM 1202 and / or the communication unit 1209. When the computer program is loaded into the RAM 1203 and executed by the CPU 1201, one or more actions in the method described above may be performed.
[0172] The present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the various processes of the above-described method for estimating water depth are implemented, and the same technical effects are achieved. To avoid repetition, the details are not described here. The computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.
[0173] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or apparatus comprising the element.
[0174] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods of each embodiment of the present application.
[0175] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of this application, ordinary technicians in this field can also make many forms without departing from the purpose of this application and the scope of protection of the claims, all of which are within the protection of this application.
[0176] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in the embodiments of this application can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0177] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0178] In the embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or groups can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0179] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.
[0180] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
[0181] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a ROM, a RAM, a magnetic disk, or an optical disk.
[0182] The above are only specific embodiments of the present application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.
Claims
1. A method for estimating water depth, characterized in that: The method comprises: Obtaining a road surface image of the target vehicle taken by a binocular camera; Identifying a waterlogged area within the road surface image; Determining a slope equation corresponding to the road surface according to three-dimensional point cloud information of other image areas in the road surface image except the waterlogged area; Based on the slope equation, the water depth corresponding to the waterlogged area is estimated.
2. The method according to claim 1, characterized in that: The identifying of the waterlogged area in the road surface image includes: Using a semantic segmentation algorithm to perform semantic segmentation processing on the road surface image to obtain boundary information of the waterlogged area; Based on the boundary information, a water accumulation area in the road surface image is determined.
3. The method according to claim 1, characterized in that The step of determining the slope equation corresponding to the road surface according to the three-dimensional point cloud information of other image areas in the road surface image except the waterlogged area comprises: Acquire initial three-dimensional point cloud information of each pixel in the road surface image; Eliminating the point cloud information located in the water accumulation area from the initial three-dimensional point cloud information to obtain the three-dimensional point cloud information of the other image areas; Determining, based on the three-dimensional point cloud information, a plurality of elevation curves of the road surface in a body direction of the target vehicle; The plurality of elevation curves are respectively fitted to obtain a plurality of slope equations corresponding to the plurality of elevation curves.
4. The method according to claim 1, characterized in that: The estimating the water depth corresponding to the waterlogged area based on the slope equation includes: The water depth corresponding to the water accumulation area is estimated based on the boundary points of the water accumulation area and the slope equation.
5. The method according to claim 4, characterized in that The estimating the water depth corresponding to the waterlogged area according to the boundary point of the waterlogged area and the slope equation includes: Acquire a first height of a reference object in a non-waterlogged area in the road surface image, and a second height of the reference object in the waterlogged area; Obtaining a height difference between the first height and the second height; Based on the height difference, the boundary point and the slope equation, the water depth corresponding to the waterlogged area is estimated.
6. The method according to claim 1, characterized in that After estimating the water depth corresponding to the waterlogged area based on the slope equation, the method further includes: When the water depth is greater than a first depth threshold, outputting driving prompt information for detour driving corresponding to the target vehicle; or When the water depth is less than or equal to the first depth threshold and greater than the second depth threshold, outputting driving prompt information for decelerating driving corresponding to the target vehicle; or When the water depth is less than or equal to the second depth threshold, driving prompt information corresponding to the target vehicle for safely passing through the water area is output.
7. A device for estimating water depth, characterized in that: The device comprises: A road surface image acquisition module is used to acquire a road surface image of the road surface taken by a binocular camera on the target vehicle; A water accumulation area recognition module, used to recognize the water accumulation area in the road surface image; A slope equation determination module, used to determine the slope equation corresponding to the road surface according to the three-dimensional point cloud information of other image areas in the road surface image except the waterlogged area; The water accumulation depth estimation module is used to estimate the water accumulation depth corresponding to the water accumulation area based on the slope equation.
8. The device according to claim 7, characterized in that The waterlogging area identification module includes: A boundary information acquisition unit, used to perform semantic segmentation processing on the road surface image using a semantic segmentation algorithm to obtain boundary information of the waterlogged area; The water accumulation area determination unit is used to determine the water accumulation area in the road surface image based on the boundary information.
9. The device according to claim 7, characterized in that The slope equation determination module includes: An initial point cloud acquisition unit, used to acquire initial three-dimensional point cloud information of each pixel point in the road surface image; A three-dimensional point cloud acquisition unit, used to remove the point cloud information located in the water accumulation area from the initial three-dimensional point cloud information, and obtain the three-dimensional point cloud information of the other image areas; an elevation curve determining unit, configured to determine a plurality of elevation curves of the road surface in a body direction of the target vehicle according to the three-dimensional point cloud information; The slope equation acquisition unit is used to perform fitting processing on the multiple elevation curves respectively to obtain multiple slope equations corresponding to the multiple elevation curves.
10. The device according to claim 7, characterized in that The water depth estimation module comprises: The water accumulation depth estimation unit is used to estimate the water accumulation depth corresponding to the water accumulation area according to the boundary points of the water accumulation area and the slope equation.
11. The device according to claim 10, characterized in that The water depth estimation unit comprises: A reference object height acquisition subunit, used to acquire a first height of a reference object in a non-waterlogged area in the road surface image, and a second height of the reference object in the waterlogged area; A height difference acquisition subunit, used to acquire a height difference between the first height and the second height; The water accumulation depth estimation subunit is used to estimate the water accumulation depth corresponding to the water accumulation area based on the height difference, the boundary point and the slope equation.
12. The device according to claim 7, characterized in that The device also includes: A first prompt information output module, configured to output driving prompt information for detour driving corresponding to the target vehicle when the water depth is greater than a first depth threshold; A second prompt information output module is used to output driving prompt information for decelerating driving corresponding to the target vehicle when the water depth is less than or equal to the first depth threshold and greater than a second depth threshold; The third prompt information output module is used to output driving prompt information corresponding to the target vehicle for safely passing through the flooded area when the water depth is less than or equal to the second depth threshold.
13. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the method for estimating water depth according to any one of claims 1 to 6.
14. A readable storage medium, characterized in that: When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the water depth estimation method according to any one of claims 1 to 6.
Citation Information
Patent Citations
Power supply model prediction method and system for unmanned vehicle in ice and snow environment
CN108573545A
Ponding water depth determination method and device
CN113168535A
Accumulated water depth estimation method and device, electronic equipment and storage medium
CN116665165A
Fording depth estimation
US20200317206A1
Cited By
Three-dimensional road surface maximum ponding section area detection method based on line scanning
CN120008519A
Urban ponding intelligent identification method and device based on unmanned aerial vehicle group multispectral imaging
CN120356126A
Substation accumulated water detection method and device, terminal equipment and computer program product
CN121010922A
A substation water accumulation detection method, device, terminal equipment and computer program product
CN121010922B
New energy sanitation vehicle sweeping control method and system fusing vision and radar, electronic equipment and storage medium
CN121050455A
