Water depth estimation method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application provide a water accumulation depth estimation method and device, electronic equipment and storage medium. The method comprises: acquiring a road surface image of a road surface on which a target vehicle is located, which is captured by a binocular camera on the target vehicle; identifying a water accumulation area in the road surface image; determining a slope equation corresponding to the road surface according to three-dimensional point cloud information of other image areas in the road surface image except the water accumulation area; and obtaining a water accumulation depth corresponding to the water accumulation area based on the slope equation. The embodiments of the present application can estimate the water accumulation depth of the road surface without relying on high-precision maps and predicted three-dimensional data and other information, thereby reducing the difficulty of water accumulation depth estimation.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method, apparatus, electronic device and storage medium for estimating water depth. Background Technology

[0002] Road flooding is crucial for vehicle passage, safe driving, and courteous driving.

[0003] Due to reflections and glare, the depth of water on roads is generally difficult for sensors to directly determine. Current practices often rely on auxiliary information to assess water depth and thickness. For example, tire pressure can be used to judge water depth, or high-precision maps of the affected road surface can be obtained using precise location data to estimate the depth. This method requires prior high-precision maps of the corresponding road segment and precise location capabilities during detection. Alternatively, 3D data of previously captured, waterless sections can be used to estimate water depth, which is then compared with 3D data of water-covered sections. This method requires prior 3D data of the road segment, is highly limiting, and cannot address water depth estimation without prior 3D data on the water level.

[0004] Both of the above methods rely on information such as high-precision maps or predicted 3D data to estimate water depth, which is quite difficult. Summary of the Invention

[0005] This application provides a method, apparatus, electronic device, and storage medium for estimating water depth, in order to solve the problem that the prior art requires relying on information such as high-precision maps or predicted three-dimensional data for water depth estimation, which makes water depth estimation quite difficult.

[0006] To solve the above-mentioned technical problems, the embodiments of this application are implemented as follows:

[0007] In a first aspect, embodiments of this application provide a method for estimating water depth, the method comprising:

[0008] Acquire road surface images of the target vehicle captured by its binocular camera.

[0009] Identify water accumulation areas within the road surface image;

[0010] Based on the three-dimensional point cloud information of other image regions in the road surface image besides the water accumulation area, the slope equation corresponding to the road surface is determined;

[0011] Based on the slope equation, the water depth corresponding to the water accumulation area is estimated.

[0012] Optionally, identifying water accumulation areas within the road surface image includes:

[0013] The road surface image is semantically segmented using a semantic segmentation algorithm to obtain the boundary information of the water accumulation area.

[0014] Based on the boundary information, the water accumulation area within the road surface image is determined.

[0015] Optionally, determining the slope equation corresponding to the road surface based on the three-dimensional point cloud information of other image regions in the road surface image besides the waterlogged area includes:

[0016] Obtain the initial three-dimensional point cloud information of each pixel in the road surface image;

[0017] Remove the point cloud information located within the water accumulation area from the initial three-dimensional point cloud information to obtain the three-dimensional point cloud information of the other image areas;

[0018] Based on the three-dimensional point cloud information, multiple elevation curves of the road surface in the direction of the target vehicle are determined;

[0019] The multiple elevation curves are fitted to obtain multiple slope equations corresponding to the multiple elevation curves.

[0020] Optionally, estimating the water depth corresponding to the water accumulation area based on the slope equation includes:

[0021] Based on the boundary points of the water accumulation area and the slope equation, the water depth corresponding to the water accumulation area is estimated.

[0022] Optionally, estimating the water depth corresponding to the water accumulation area based on the boundary points of the water accumulation area and the slope equation includes:

[0023] Obtain the first height of the reference object in the non-waterlogged area of ​​the road surface image, and the second height of the reference object in the waterlogged area;

[0024] Obtain the height difference between the first height and the second height;

[0025] Based on the height difference, the boundary point, and the slope equation, the water depth corresponding to the water accumulation area is estimated.

[0026] Optionally, after estimating the water depth corresponding to the water accumulation area based on the slope equation, the method further includes:

[0027] If the water depth exceeds a first depth threshold, output driving prompts for the target vehicle to detour; or

[0028] If the water depth is less than or equal to the first depth threshold but greater than the second depth threshold, output a driving prompt message for the target vehicle indicating deceleration; or

[0029] If the water depth is less than or equal to the second depth threshold, output driving prompt information for the target vehicle to safely pass through the water area.

[0030] Secondly, embodiments of this application provide a water depth estimation device, the device comprising:

[0031] The road surface image acquisition module is used to acquire road surface images of the target vehicle captured by the binocular camera on the road surface.

[0032] A water accumulation area identification module is used to identify water accumulation areas within the road surface image;

[0033] The slope equation determination module is used to determine the slope equation corresponding to the road surface based on the three-dimensional point cloud information of other image regions in the road surface image, excluding the water accumulation area.

[0034] The water depth estimation module is used to estimate the water depth corresponding to the water accumulation area based on the slope equation.

[0035] Optionally, the water accumulation area identification module includes:

[0036] The boundary information acquisition unit is used to perform semantic segmentation processing on the road surface image using a semantic segmentation algorithm to obtain the boundary information of the water accumulation area.

[0037] The water accumulation area determination unit is used to determine the water accumulation area within the road surface image based on the boundary information.

[0038] Optionally, the slope equation determination module includes:

[0039] An initial point cloud acquisition unit is used to acquire the initial three-dimensional point cloud information of each pixel in the road surface image;

[0040] A three-dimensional point cloud acquisition unit is used to remove point cloud information located in the water accumulation area from the initial three-dimensional point cloud information to obtain three-dimensional point cloud information of other image areas;

[0041] The elevation curve determination unit is used to determine multiple elevation curves of the road surface in the direction of the target vehicle based on the three-dimensional point cloud information.

[0042] The slope equation acquisition unit is used to perform fitting processing on the multiple elevation curves respectively to obtain multiple slope equations corresponding to the multiple elevation curves.

[0043] Optionally, the water depth estimation module includes:

[0044] The water depth estimation unit is used to estimate the water depth corresponding to the water accumulation area based on the boundary points of the water accumulation area and the slope equation.

[0045] Optionally, the water depth estimation unit includes:

[0046] The reference height acquisition subunit is used to acquire the first height of the reference object in the non-waterlogged area of ​​the road surface image, and the second height of the reference object in the waterlogged area.

[0047] The height difference acquisition subunit is used to acquire the height difference between the first height and the second height;

[0048] The water depth estimation subunit is used to estimate the water depth corresponding to the water accumulation area based on the height difference, the boundary point, and the slope equation.

[0049] Optionally, the device further includes:

[0050] The first prompt information output module is used to output driving prompt information for the target vehicle to detour when the water depth is greater than the first depth threshold.

[0051] The second prompt information output module is used to output driving prompt information for the target vehicle to decelerate when the water depth is less than or equal to the first depth threshold and greater than the second depth threshold.

[0052] The third prompt information output module is used to output driving prompt information for the target vehicle to safely pass through the waterlogged area when the water depth is less than or equal to the second depth threshold.

[0053] Thirdly, embodiments of this application provide an electronic device, including:

[0054] A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the water depth estimation method described above.

[0055] Fourthly, embodiments of this application provide a readable storage medium that, when the instructions in the storage medium are executed by the processor of an electronic device, enables the electronic device to perform the water depth estimation method described in any of the preceding claims.

[0056] In this embodiment, road surface images captured by a binocular camera on the target vehicle are obtained, and water accumulation areas within the road surface images are identified. Based on the 3D point cloud information of other image regions within the road surface image besides the water accumulation areas, the slope equation corresponding to the road surface is determined. Based on the slope equation, the water depth corresponding to the water accumulation areas is estimated. This embodiment obtains accurate point cloud information of the current road surface without water accumulation from RGB images captured by a binocular camera, and calculates the water depth using an elevation fitting method. This eliminates the need for high-precision maps and predictive 3D data, reducing the difficulty of water depth estimation.

[0057] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0058] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0059] Figure 1 A flowchart illustrating the steps of a water depth estimation method provided in this application embodiment;

[0060] Figure 2 A flowchart illustrating the steps of a method for determining a water accumulation area provided in this application embodiment;

[0061] Figure 3 A flowchart illustrating the steps of a method for obtaining a slope equation provided in this application embodiment;

[0062] Figure 4 A flowchart illustrating the steps of a method for estimating water depth provided in this application embodiment;

[0063] Figure 5 A flowchart illustrating the steps of another method for estimating water depth provided in this application embodiment;

[0064] Figure 6 A flowchart illustrating the steps of a driving prompt information output method provided in this application embodiment;

[0065] Figure 7 This is a schematic diagram of a water accumulation area provided in an embodiment of this application;

[0066] Figure 8A schematic diagram of an elevation curve provided for an embodiment of this application;

[0067] Figure 9 A schematic diagram illustrating a water depth estimation method provided in an embodiment of this application;

[0068] Figure 10 A schematic diagram illustrating an auxiliary method for estimating water depth, as provided in an embodiment of this application;

[0069] Figure 11 A schematic diagram of a water depth estimation device provided in an embodiment of this application;

[0070] Figure 12 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0071] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0072] Reference Figure 1 The diagram illustrates a flowchart of a water depth estimation method provided in an embodiment of this application. Figure 1 As shown, the water depth estimation method may include steps 101, 102, 103 and 104.

[0073] Step 101: Obtain the road surface image of the target vehicle taken by the binocular camera.

[0074] The embodiments of this application can be applied to scenarios where the road surface RGB image captured by a binocular camera is combined with the elevation fitting method to estimate the road surface water depth.

[0075] In this embodiment, both binocular cameras are RGB cameras.

[0076] Understandably, road flooding is usually caused by the slope of the road surface, which is characterized by a gradual and continuous gradient. Estimation of water depth due to abnormal factors such as potholes is beyond the scope of this application's embodiments.

[0077] If the target vehicle encounters standing water while driving, the binocular camera on the target vehicle can be activated to capture an image of the road surface where the target vehicle is located. This road surface image is an RGB image.

[0078] After obtaining the road surface image of the target vehicle taken by the binocular camera, proceed to step 102.

[0079] Step 102: Identify the water accumulation areas within the road surface image.

[0080] After acquiring the road surface image captured by the binocular camera on the target vehicle, the water accumulation areas within the road surface image can be identified, i.e., the water accumulation image areas. In this example, algorithms such as deep learning semantic segmentation can be used to perform semantic segmentation processing on the road surface image to identify the boundary information of the water accumulation areas within the road surface image. Then, the water accumulation areas within the road surface image can be determined based on the boundary information. This implementation process can be combined with... Figure 2 The following is a detailed description.

[0081] Reference Figure 2 The diagram illustrates a flowchart of a method for determining a waterlogged area according to an embodiment of this application. Figure 2 As shown, the method for determining the waterlogged area may include steps 201 and 202.

[0082] Step 201: Use a semantic segmentation algorithm to perform semantic segmentation processing on the road surface image to obtain the boundary information of the water accumulation area.

[0083] In this embodiment, after obtaining the road surface image, a semantic segmentation algorithm can be used to perform semantic segmentation processing on the road surface image to obtain the boundary information of the water accumulation area.

[0084] After using a semantic segmentation algorithm to perform semantic segmentation processing on the road surface image to obtain the boundary information of the water accumulation area, step 202 is executed.

[0085] Step 202: Based on the boundary information, determine the water accumulation area within the road surface image.

[0086] After obtaining the boundary information of the waterlogged area, the waterlogged area within the road surface image can be determined based on this boundary information. For example... Figure 7 As shown, the water surface is perceived based on a binocular RGB image. The perception algorithm can obtain the boundary of the water accumulation area on the image through semantic segmentation.

[0087] This application uses a deep learning semantic segmentation algorithm to identify water accumulation areas in road surface images, which can improve the accuracy of water accumulation area identification.

[0088] After identifying the waterlogged areas in the road surface image, proceed to step 103.

[0089] Step 103: Determine the slope equation corresponding to the road surface based on the three-dimensional point cloud information of other image regions in the road surface image, excluding the water accumulation area.

[0090] In practical implementation, the left and right RGB views (i.e., road surface images) captured by a binocular camera can utilize the principle of parallax and employ methods such as deep learning to obtain the disparity maps of each pixel within the image field of view of the left and right RGB views. Furthermore, the 3D point cloud information within the image field of view can be calculated based on the intrinsic parameters of the binocular camera.

[0091] After identifying the waterlogged areas in the road image, 3D point cloud information of other image regions besides the waterlogged areas can be obtained. Based on this 3D point cloud information, the slope equation corresponding to the road surface can be determined. Specifically, multiple elevation curves for the road surface can be determined first based on the 3D point cloud information of other image regions. Combining these multiple elevation curves yields multiple slope equations. This implementation process can be further described in conjunction with... Figure 3 The following is a detailed description.

[0092] Reference Figure 3 The diagram illustrates a flowchart of a method for obtaining a slope equation according to an embodiment of this application. Figure 3 As shown, the method for obtaining the slope equation may include steps 301, 302, 303, and 304.

[0093] Step 301: Obtain the initial three-dimensional point cloud information of each pixel in the road surface image.

[0094] In this embodiment, after identifying the water accumulation area in the road surface image, the initial three-dimensional point cloud information of each pixel in the road surface image can be obtained.

[0095] After obtaining the initial 3D point cloud information of each pixel in the road surface image, step 302 is executed.

[0096] Step 302: Remove the point cloud information located within the water accumulation area from the initial three-dimensional point cloud information to obtain the three-dimensional point cloud information of the other image areas.

[0097] After obtaining the initial 3D point cloud information of each pixel in the road surface image, the point cloud information located in the water accumulation area can be removed from the initial 3D point cloud information to obtain the 3D point cloud information of other image areas.

[0098] After obtaining the 3D point cloud information of other image regions, proceed to step 303.

[0099] Step 303: Based on the three-dimensional point cloud information, determine multiple elevation curves of the road surface in the direction of the target vehicle.

[0100] After acquiring the 3D point cloud information from other image regions, multiple elevation curves of the road surface along the vehicle's body direction can be determined based on this information. Specifically, the 3D point cloud information can be sampled along the vehicle's body direction to obtain multiple elevation curves along that direction, such as... Figure 7 and Figure 8 As shown, along Figure 7 By taking the value of the 3D point cloud information of line 1 in the image, we can obtain... Figure 8 Elevation curve 1 in the middle, along Figure 7 By taking the values ​​of the 3D point cloud information from line 2 in the image, we can obtain... Figure 8 Elevation curve 2, etc.

[0101] It is understood that the above examples are merely examples listed to better understand the technical solutions of the embodiments of this application, and are not intended to be the only limitation on the embodiments.

[0102] After determining multiple elevation curves of the road surface in the direction of the target vehicle based on the 3D point cloud information, step 304 is executed.

[0103] Step 304: Fit the multiple elevation curves respectively to obtain multiple slope equations corresponding to the multiple elevation curves.

[0104] After determining multiple elevation curves of the road surface in the direction of the target vehicle based on the 3D point cloud information, the multiple elevation curves can be fitted separately to obtain multiple slope equations corresponding to the multiple elevation curves.

[0105] Understandably, due to the presence of water accumulation, the elevation curve contains discrete elevation points. After obtaining an elevation curve, these discrete elevation points can be fitted to obtain the corresponding slope equation. In this example, the slope equation can be represented by a polynomial such as Z = ay^3 + by^2 + cy + d, where Z is the height, y is the y-axis value corresponding to the vehicle's direction, etc.

[0106] After determining the slope equation corresponding to the road surface based on the three-dimensional point cloud information of other image regions in the road surface image, excluding the water accumulation area, step 104 is executed.

[0107] Step 104: Based on the slope equation, estimate the water depth corresponding to the water accumulation area.

[0108] After determining the slope equation corresponding to the road surface, the water depth of the water accumulation area can be estimated based on the slope equation. The process of estimating water depth can be combined with... Figure 4 The following is a detailed description.

[0109] Reference Figure 4 The diagram illustrates a flowchart of a method for estimating water depth according to an embodiment of this application. Figure 4 As shown, the water depth estimation method may include: step 401.

[0110] Step 401: Based on the boundary points of the water accumulation area and the slope equation, estimate the water depth corresponding to the water accumulation area.

[0111] In this embodiment, after obtaining the slope equation, the water depth corresponding to the water accumulation area can be estimated based on the boundary points of the water accumulation area and the slope equation. For example... Figure 9 As shown, the heights of the two sides of the water accumulation area are 200 and 300 respectively. The height of each point in the water accumulation area can be obtained through the slope equation, thereby enabling the estimation of the water depth, etc.

[0112] This application embodiment obtains accurate point cloud information of the current road surface without water accumulation by capturing RGB images with a binocular camera, and calculates the water depth by using an elevation fitting method. It does not require high-precision maps and predictive 3D data, thus reducing the difficulty of water depth estimation.

[0113] In this embodiment, information about other objects on the road, such as curbs and fences, can also be used to assist in estimating water depth. The implementation process for assisting in water depth estimation can be combined with... Figure 5 The following is a detailed description.

[0114] Reference Figure 5 The diagram illustrates a flowchart of another method for estimating water depth provided in an embodiment of this application. Figure 5 As shown, the water depth estimation method may include steps 501, 502 and 503.

[0115] Step 501: Obtain the first height of the reference object in the non-waterlogged area of ​​the road surface image, and the second height of the reference object in the waterlogged area.

[0116] In this embodiment, the reference object can be a curb, fence, or other similar object. The type of reference object can be determined according to the actual situation, and this embodiment does not impose any restrictions on it.

[0117] The first height refers to the height of the reference point in the area without water accumulation within the road surface image.

[0118] The second height refers to the height of the reference point located in the waterlogged area within the road surface image.

[0119] In practical implementation, when estimating the water depth in a flooded area, a first height of a reference object in a water-free area within the road surface image and a second height of the reference object located in the flooded area can be obtained. For example, taking a curb as an example, the height of the curb in a water-free area within the road surface image can be obtained, i.e., the first height, which is the height between the curb and the road surface. The height of the curb in the flooded area within the road surface image can also be obtained, i.e., the second height, which is the height between the curb and the water surface, etc.

[0120] It is understood that the above examples are merely examples listed to better understand the technical solutions of the embodiments of this application, and are not intended to be the only limitation on the embodiments.

[0121] After obtaining the first height of the reference object in the non-waterlogged area of ​​the road surface image and the second height of the reference object in the waterlogged area, step 502 is executed.

[0122] Step 502: Obtain the height difference between the first height and the second height.

[0123] After obtaining the first height of the reference object in the water-free area of ​​the road surface image and the second height of the reference object in the water-filled area, the height difference between the first and second heights can be calculated. For example... Figure 10 As shown, taking the curb as an example, the first height of the curb is 15cm in areas without water accumulation, and the second height of the curb is 10cm in areas with water accumulation. The calculated height difference is 5cm, etc.

[0124] After obtaining the height difference between the first height and the second height, proceed to step 503.

[0125] Step 503: Based on the height difference, the boundary point, and the slope equation, the water depth corresponding to the water accumulation area is estimated.

[0126] After obtaining the height difference between the first and second heights, the water depth corresponding to the water accumulation area can be estimated based on the height difference, boundary points, and slope equation.

[0127] In one specific implementation of this application, the initial predicted water depth of the water accumulation area can be estimated first using boundary points and slope equations. Then, the initial predicted water depth is calibrated based on the height difference to obtain the final predicted water depth. For example, the average of the initial predicted water depth and the height difference can be used as the final predicted water depth, or a weighted average of the initial predicted water depth and the height difference can be used as the final predicted water depth, and so on.

[0128] In another specific implementation of this application, the initial predicted water depth of the waterlogged area can be estimated first using boundary points and slope equations. Then, the accuracy of the initial predicted water depth is verified based on the height difference. For example, if the difference between the height difference and the initial predicted water depth is within a preset range, the accuracy of the initial predicted water depth can be considered high, and in this case, the initial predicted water depth can be used as the final estimated water depth. However, if the difference between the height difference and the initial predicted water depth is not within the preset range, the accuracy of the initial predicted water depth can be considered low. In this case, a new RGB image of the road surface at the waterlogged area can be taken to re-estimate the water depth.

[0129] It is understood that the above two implementation methods are merely examples listed to better understand the technical solutions of the embodiments of this application, and are not intended to be the only limitation of this embodiment.

[0130] The embodiments of this application improve the accuracy of water depth estimation by using reference objects (such as curbs, fences, etc.) to assist in determining the depth of water accumulation.

[0131] In this embodiment, after estimating the water depth, a corresponding driving strategy can be executed based on the water depth. Specifically, this implementation process can be combined with... Figure 6 The following is a detailed description.

[0132] Reference Figure 6 The diagram illustrates a flowchart of a driving prompt information output method provided in an embodiment of this application. Figure 6 As shown, the driving prompt information output method may include steps 601, 602 and 603.

[0133] Step 601: If the water depth is greater than the first depth threshold, output driving prompt information for the target vehicle to detour.

[0134] Step 602: If the water depth is less than or equal to the first depth threshold and greater than the second depth threshold, output the driving prompt information for deceleration of the target vehicle.

[0135] Step 603: If the water depth is less than or equal to the second depth threshold, output driving prompt information for the target vehicle to safely pass through the water area.

[0136] In this embodiment, both the first depth threshold and the second water depth threshold are pre-set water depth thresholds used to guide users on how to drive through waterlogged sections. The first depth threshold is greater than the second depth threshold. The specific values ​​of the first and second depth thresholds can be determined according to business needs; this embodiment does not impose any restrictions on the specific values ​​of the first and second depth thresholds.

[0137] If the water depth exceeds a first depth threshold, driving instructions can be output for the target vehicle to take an alternate route. In other words, if the road surface is too flooded and driving is dangerous, the user is advised to take an alternate route.

[0138] When the water depth is less than or equal to the first depth threshold but greater than the second depth threshold, a driving prompt message indicating deceleration can be output for the target vehicle. In other words, if the water depth is appropriate, the vehicle can slow down and drive through to avoid water entering the vehicle and causing it to stall.

[0139] If the water depth is less than or equal to the second depth threshold, a driving prompt message indicating that the target vehicle can safely pass through the flooded area can be output. This means the water level is shallow and the vehicle can pass through the flooded section normally.

[0140] This application embodiment assists users in driving by pre-setting a water depth threshold, which can improve driving safety.

[0141] The water depth estimation method provided in this application acquires road surface images captured by a binocular camera on the target vehicle and identifies water-filled areas within the road surface images. Based on the 3D point cloud information of other image regions besides the water-filled areas, the slope equation corresponding to the road surface is determined. Based on the slope equation, the water depth corresponding to the water-filled areas is estimated. This application embodiment obtains accurate point cloud information of the current road surface without water accumulation from RGB images captured by a binocular camera, and calculates the water depth using an elevation fitting method. This method does not rely on high-precision maps or predictive 3D data, thus reducing the difficulty of water depth estimation.

[0142] Reference Figure 11 The diagram shows a structural schematic of a water depth estimation device provided in an embodiment of this application. Figure 11 As shown, the water depth estimation device 1100 may include the following modules:

[0143] The road surface image acquisition module 1101 is used to acquire road surface images of the target vehicle captured by the binocular camera on the road surface.

[0144] The water accumulation area identification module 1102 is used to identify water accumulation areas within the road surface image;

[0145] The slope equation determination module 1103 is used to determine the slope equation corresponding to the road surface based on the three-dimensional point cloud information of other image regions in the road surface image, excluding the water accumulation area.

[0146] The water depth estimation module 1104 is used to estimate the water depth corresponding to the water accumulation area based on the slope equation.

[0147] Optionally, the water accumulation area identification module includes:

[0148] The boundary information acquisition unit is used to perform semantic segmentation processing on the road surface image using a semantic segmentation algorithm to obtain the boundary information of the water accumulation area.

[0149] The water accumulation area determination unit is used to determine the water accumulation area within the road surface image based on the boundary information.

[0150] Optionally, the slope equation determination module includes:

[0151] An initial point cloud acquisition unit is used to acquire the initial three-dimensional point cloud information of each pixel in the road surface image;

[0152] A three-dimensional point cloud acquisition unit is used to remove point cloud information located in the water accumulation area from the initial three-dimensional point cloud information to obtain three-dimensional point cloud information of other image areas;

[0153] The elevation curve determination unit is used to determine multiple elevation curves of the road surface in the direction of the target vehicle based on the three-dimensional point cloud information.

[0154] The slope equation acquisition unit is used to perform fitting processing on the multiple elevation curves respectively to obtain multiple slope equations corresponding to the multiple elevation curves.

[0155] Optionally, the water depth estimation module includes:

[0156] The water depth estimation unit is used to estimate the water depth corresponding to the water accumulation area based on the boundary points of the water accumulation area and the slope equation.

[0157] Optionally, the water depth estimation unit includes:

[0158] The reference height acquisition subunit is used to acquire the first height of the reference object in the non-waterlogged area of ​​the road surface image, and the second height of the reference object in the waterlogged area.

[0159] The height difference acquisition subunit is used to acquire the height difference between the first height and the second height;

[0160] The water depth estimation subunit is used to estimate the water depth corresponding to the water accumulation area based on the height difference, the boundary point, and the slope equation.

[0161] Optionally, the device further includes:

[0162] The first prompt information output module is used to output driving prompt information for the target vehicle to detour when the water depth is greater than the first depth threshold.

[0163] The second prompt information output module is used to output driving prompt information for the target vehicle to decelerate when the water depth is less than or equal to the first depth threshold and greater than the second depth threshold.

[0164] The third prompt information output module is used to output driving prompt information for the target vehicle to safely pass through the waterlogged area when the water depth is less than or equal to the second depth threshold.

[0165] The water depth estimation device provided in this application acquires road surface images of the target vehicle taken by a binocular camera, and identifies water-filled areas within the road surface images. Based on the 3D point cloud information of other image regions besides the water-filled areas, the slope equation corresponding to the road surface is determined. Based on the slope equation, the water depth corresponding to the water-filled areas is estimated. This application embodiment obtains accurate point cloud information of the current road surface without water accumulation from RGB images captured by a binocular camera, and calculates the water depth using an elevation fitting method. This eliminates the need for high-precision maps and predictive 3D data, reducing the difficulty of water depth estimation.

[0166] This application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, it implements the above-described method for estimating water depth.

[0167] Figure 12 A schematic diagram of the structure of an electronic device 1200 according to an embodiment of the present invention is shown. Figure 12As shown, the binocular camera 1300 can provide the electronic device 1200 with captured road images (i.e., RGB images). The electronic device 1200 includes a central processing unit (CPU) 1201, which can perform various appropriate actions and processes according to computer program instructions stored in read-only memory (ROM) 1202 or loaded from storage unit 1208 into random access memory (RAM) 1203. The RAM 1203 can also store various programs and data required for the operation of the electronic device 1200. The CPU 1201, ROM 1202, and RAM 1203 are interconnected via bus 1204. An input / output (I / O) interface 1205 is also connected to bus 1204.

[0168] Multiple components in electronic device 1200 are connected to I / O interface 1205, including: input unit 1206, such as keyboard, mouse, microphone, etc.; output unit 1207, such as various types of monitors, speakers, etc.; storage unit 1208, such as disk, optical disk, etc.; and communication unit 1209, such as network card, modem, wireless transceiver, etc. Communication unit 1209 allows electronic device 1200 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0169] The various processes and handling described above can be executed by processing unit 1201. For example, the methods of any of the above embodiments can be implemented as computer software programs tangibly contained in a computer-readable medium, such as storage unit 1208. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 1200 via ROM 1202 and / or communication unit 1209. When the computer program is loaded into RAM 1203 and executed by CPU 1201, one or more actions of the methods described above can be performed.

[0170] This application provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the various processes of the above-described water depth estimation method embodiments and achieves the same technical effects. To avoid repetition, further details are omitted here. The computer-readable storage medium may include read-only memory (ROM), random access memory (RAM), a magnetic disk, or an optical disk.

[0171] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0172] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0173] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

[0174] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed in this application can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0175] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0176] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or groups may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0177] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0178] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0179] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0180] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A water depth estimation method characterized by comprising: The method comprises: acquiring a road surface image of a road surface on which a target vehicle is located, the road surface image being captured by a binocular camera on the target vehicle; identifying a water accumulation area in the road surface image; determining a slope equation corresponding to the road surface according to three-dimensional point cloud information of other image areas in the road surface image except the water accumulation area; estimating water depth corresponding to the water accumulation area based on the slope equation; the estimating water depth corresponding to the water accumulation area based on the slope equation comprises: estimating water depth corresponding to the water accumulation area according to boundary points of the water accumulation area and the slope equation; the estimating water depth corresponding to the water accumulation area according to the boundary points of the water accumulation area and the slope equation comprises: acquiring a first height of a reference object at a non-water accumulation area in the road surface image and a second height of the reference object at the water accumulation area; acquiring a height difference between the first height and the second height; estimating water depth corresponding to the water accumulation area based on the height difference, the boundary points and the slope equation.

2. The method of claim 1, wherein, The identifying the water accumulation area in the road surface image comprises: performing semantic segmentation processing on the road surface image by using a semantic segmentation algorithm to obtain boundary information of the water accumulation area; determining the water accumulation area in the road surface image based on the boundary information.

3. The method of claim 1, wherein, The determining a slope equation corresponding to the road surface according to three-dimensional point cloud information of other image areas in the road surface image except the water accumulation area comprises: acquiring initial three-dimensional point cloud information of each pixel point in the road surface image; eliminating point cloud information in the initial three-dimensional point cloud information located in the water accumulation area to obtain three-dimensional point cloud information of the other image areas; determining a plurality of elevation curves of the road surface in a vehicle body direction of the target vehicle according to the three-dimensional point cloud information; respectively performing fitting processing on the plurality of elevation curves to obtain a plurality of slope equations corresponding to the plurality of elevation curves.

4. The method of claim 1, wherein, After the estimating water depth corresponding to the water accumulation area based on the slope equation, the method further comprises: in a case where the water depth is greater than a first depth threshold, outputting driving prompt information corresponding to detour driving of the target vehicle; or in a case where the water depth is less than or equal to the first depth threshold and greater than a second depth threshold, outputting driving prompt information corresponding to speed reduction driving of the target vehicle; or in a case where the water depth is less than or equal to the second depth threshold, outputting driving prompt information corresponding to safe driving through the water accumulation area of the target vehicle.

5. An accumulated water depth estimation device characterized by comprising: The device comprises: a road surface image acquisition module configured to acquire a road surface image of a road surface on which a target vehicle is located, the road surface image being captured by a binocular camera on the target vehicle; a water accumulation area identification module configured to identify a water accumulation area in the road surface image; a slope equation determination module configured to determine a slope equation corresponding to the road surface according to three-dimensional point cloud information of other image areas in the road surface image except the water accumulation area; a water depth estimation module configured to estimate water depth corresponding to the water accumulation area based on the slope equation; the water depth estimation module comprises: The water accumulation depth estimation unit is configured to estimate the water accumulation depth corresponding to the water accumulation region according to the boundary point of the water accumulation region and the slope equation; The water accumulation depth estimation unit comprises: The reference object height acquisition subunit is configured to acquire a first height of a reference object in a non-water accumulation region in the road surface image and a second height of the reference object in the water accumulation region; The height difference value acquisition subunit is configured to acquire a height difference value between the first height and the second height; The water accumulation depth estimation subunit is configured to estimate the water accumulation depth corresponding to the water accumulation region based on the height difference value, the boundary point, and the slope equation.

6. The apparatus of claim 5, wherein, The water accumulation region identification module comprises: The boundary information acquisition unit is configured to perform semantic segmentation processing on the road surface image by using a semantic segmentation algorithm to obtain boundary information of a water accumulation region; The water accumulation region determination unit is configured to determine a water accumulation region in the road surface image based on the boundary information.

7. The apparatus of claim 5, wherein, The slope equation determination module comprises: The initial point cloud acquisition unit is configured to acquire initial three-dimensional point cloud information of each pixel point in the road surface image; The three-dimensional point cloud acquisition unit is configured to remove point cloud information in the initial three-dimensional point cloud information located in the water accumulation region to obtain three-dimensional point cloud information of the other image regions; The elevation curve determination unit is configured to determine a plurality of elevation curves of the road surface in the direction of the vehicle body of the target vehicle according to the three-dimensional point cloud information; The slope equation acquisition unit is configured to perform fitting processing on the plurality of elevation curves to obtain a plurality of slope equations corresponding to the plurality of elevation curves.

8. The apparatus of claim 5, wherein, The device further comprises: The first prompt information output module is configured to output driving prompt information corresponding to detour driving of the target vehicle when the water accumulation depth is greater than a first depth threshold; The second prompt information output module is configured to output driving prompt information corresponding to speed reduction driving of the target vehicle when the water accumulation depth is less than or equal to the first depth threshold and greater than a second depth threshold; The third prompt information output module is configured to output driving prompt information corresponding to safe driving through the water accumulation region of the target vehicle when the water accumulation depth is less than or equal to the second depth threshold.

9. An electronic device, comprising: The device further comprises: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is executed by the processor to implement the water accumulation depth estimation method in any one of claims 1 to 4.

10. A readable storage medium, characterized by, When the instructions in the storage medium are executed by the processor of the electronic device, the electronic device can execute the water accumulation depth estimation method in any one of claims 1 to 4.

Images