Water depth detection method and device, electronic equipment and storage medium

Abstract

The invention provides a water depth detection method and device, electronic equipment and a storage medium, and relates to the technical field of detection. The water depth detection method comprises the following steps: generating an initial parallax image according to obtained original images of different visual angles of a target water area; determining a first edge feature of the original image and a second edge feature of the initial parallax image; obtaining a first parallax value of each pixel in a first preset range around a second edge feature in the initial parallax image, and performing optimization processing on each first parallax value to align the first edge feature and the second edge feature to obtain a middle parallax image; obtaining a second parallax value of each pixel in a first preset range around the second edge feature in the middle parallax image, and carrying out filtering processing on each second parallax value to obtain a target parallax image; and obtaining a third parallax value of the preset target point according to the target parallax image, and determining the actual water depth of the preset target point based on the third parallax value. The accuracy of water depth detection is improved.

Description

Technical Field

[0001] The present invention relates to the field of detection technologies, and in particular, to a water depth detection method, device, electronic device, and storage medium. Background Art

[0002] Detecting water depth is not only a basic task in ocean science research, but also an important means to ensure marine economic activities, protect the ecological environment, and respond to climate change. In recent years, with the progress of computer vision technology and image processing algorithms, water depth detection methods based on optical images have gradually attracted attention. This method analyzes stereo images taken from different angles, uses parallax information to reconstruct the three-dimensional terrain, and then calculates the water depth data. This non-contact detection method not only has a lower cost, but can also be combined with existing remote sensing platforms (such as drones, satellites, etc.), and has broad application prospects.

[0003] In related technologies, due to the low visibility underwater, combined with the influence of factors such as the atmosphere, solar flares, water turbidity, and bottom sediment types, the texture of the obtained underwater stereo images is not clear and radiation distortion is likely to occur, resulting in poor accuracy of water depth detection. Summary of the Invention

[0004] The problem solved by the present invention is how to improve the accuracy of water depth detection.

[0005] To solve the above problems, the present invention provides a water depth detection method, device, electronic device, and storage medium.

[0006] In a first aspect, the present invention provides a water depth detection method, including: Generating an initial parallax image according to the original images of a target water area from different perspectives that have been acquired; Determining the first edge feature of the original image of any perspective, and determining the second edge feature of the initial parallax image; Obtaining the first parallax values of each pixel within a first preset range around the second edge feature in the initial parallax image, and performing optimization processing on each of the first parallax values to align the first edge feature and the second edge feature, obtaining an intermediate parallax image; Obtaining the second parallax values of each pixel within the first preset range around the second edge feature in the intermediate parallax image, and performing filtering processing on each of the second parallax values to obtain a target parallax image; Obtaining the third parallax value of a preset target point according to the target parallax image, and determining the actual water depth corresponding to the preset target point based on the third parallax value.

[0007] Optionally, the determining the second edge feature of the initial parallax image includes: Detect the line features of the initial disparity image. When the first disparity values of the pixels within the first preset range around the line features are all greater than the preset threshold, the line features are used as the second edge features.

[0008] Optionally, the optimizing the first disparity values includes: Construct a global energy function based on the first disparity values of the pixels within the first preset range around the second edge features in the initial disparity image. Taking the minimization of the global energy function as the optimization objective, solve the global energy function to obtain the second disparity values; wherein, the global energy function includes a cost term and a regularization term. The cost term is used to measure the distance between the second disparity values and the first disparity values of the pixels, and the regularization term is used to smooth and constrain the second disparity values of the first target pixel and its neighboring pixels. The first target pixel and its neighboring pixels refer to the pixels located on the same side of the second edge feature.

[0009] Optionally, the taking the minimization of the global energy function as the optimization objective, solving the global energy function to obtain the second disparity values includes: Based on the pixels within the first preset range around the second edge feature, construct a graphical model; Determine the minimum cut of the graphical model, and obtain the minimum value of the global energy function and the second disparity values according to the minimum cut.

[0010] Optionally, the filtering the second disparity values to obtain the target disparity image includes: Based on the gray values and distances of the second target pixels and their neighboring pixels, determine the weights of the second target pixels; wherein, the second target pixels refer to the pixels located within the first preset range around the second edge feature; the neighboring pixels refer to the pixels located within the second preset range around the second edge feature, and the second preset range includes the first preset range; According to the weights of the second target pixels, perform weighted averaging on the second disparity values corresponding to the second target pixels to obtain the target disparity image.

[0011] Optionally, the determining the actual water depth corresponding to the preset target point based on the third disparity values includes: Obtain the shooting parameters corresponding to each of the original images, and based on the third disparity values and the shooting parameters, determine the preset water depth corresponding to the preset target point; Perform refraction correction on the preset water depth of the target point to obtain the actual water depth corresponding to the preset target point.

[0012] Optionally, generating an initial disparity image based on the original images of different perspectives of the acquired target water area includes: Performing epipolar rectification on the original images and determining the matching cost of each pixel in the original images; Aggregating the matching cost in multiple directions to generate the initial disparity image, where the first disparity value of the pixel in the initial disparity image corresponds to the minimum value of the aggregated matching cost of the pixel.

[0013] In a second aspect, the present invention provides a water depth detection device, including: A generation module, configured to generate an initial disparity image based on the original images of different perspectives of the acquired target water area; A first determination module, configured to determine the first edge feature of the original image of any perspective and determine the second edge feature of the initial disparity image; An optimization module, configured to obtain the first disparity values of the pixels within a first preset range around the second edge feature in the initial disparity image, and perform optimization processing on the first disparity values to align the first edge feature and the second edge feature to obtain an intermediate disparity image; A filtering module, configured to obtain the second disparity values of the pixels within the first preset range around the second edge feature in the intermediate disparity image, and perform filtering processing on the second disparity values to obtain a target disparity image; A second determination module, configured to obtain the third disparity value of a preset target point based on the target disparity image, and determine the actual water depth corresponding to the preset target point based on the third disparity value.

[0014] In a second aspect, the present invention provides a water depth detection device, including: In a third aspect, the present invention provides an electronic device, including a memory and a processor; The memory is configured to store a computer program; The processor is configured to, when executing the computer program, implement the water depth detection method as described in the first aspect.

[0015] In a fourth aspect, the present invention provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the water depth detection method as described in the first aspect is implemented.

[0016] The beneficial effects of the water depth detection method, device, electronic device and storage medium of the present invention are as follows: Obtaining multiple original images of the same target water area taken from different perspectives helps to provide rich perspective information and reduce the matching error caused by a single perspective. Through a stereo matching algorithm, an initial disparity image is generated based on the multiple original images. The first edge feature of any one of the original images and the second edge feature of the initial disparity image are respectively obtained, and by aligning the first edge feature with the second edge feature, the matching error caused by unclear texture is reduced, thereby improving the matching degree of corresponding points in the multiple original images. Among them, the first edge feature and the second edge feature can be the edge line of the underwater terrain, the contour line of underwater objects, and the boundary line between water and land, etc. In order to align the first edge feature with the second edge feature, the first disparity values of each pixel within a first preset range around the second edge feature are optimized to obtain an intermediate disparity image. In this way, it can not only reduce the mis-matched pixels caused by unclear texture and reduce the probability of the occurrence of the expansion phenomenon, but also retain the detailed information such as the terrain and landform near the second edge feature, avoiding the loss of details caused by forced smoothing. On this basis, there may be slight disparity breaks around the second edge feature in the intermediate disparity image, that is, for each pixel within the first preset range and each pixel outside the first preset range. It is necessary to perform filtering processing on the second disparity values of each pixel within the first preset range to obtain a target disparity image, making the disparity change around the second edge feature smoother, reducing the matching error caused by radiation distortion, and at the same time retaining the detailed information such as the terrain and landform near the second edge feature. In this way, when the first edge feature is aligned with the second edge feature, a target point is selected as a preset target point on the original image, and the third disparity value of the preset target point is obtained from the target disparity image, and then the actual water depth corresponding to the preset target point can be determined, which is beneficial to improving the accuracy of water depth detection. Description of the Drawings

[0017] Figure 1 It is a schematic flowchart of the water depth detection method according to an embodiment of the present invention; Figure 2 It is a schematic flowchart of the water depth detection method according to another embodiment of the present invention; Figure 3 It is a system architecture diagram of the water depth detection device according to an embodiment of the present invention; Figure 4 It is a system architecture diagram of the electronic device according to an embodiment of the present invention. Detailed Embodiments

[0018] To make the above objects, features, and advantages of the present invention more apparent and understandable, the following provides a detailed description of specific embodiments of the present invention with reference to the accompanying drawings. Although some embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be construed as limited to the embodiments described herein. Instead, these embodiments are provided to more thoroughly and completely understand the present invention. It should be understood that the drawings and embodiments of the present invention are only for illustrative purposes and are not used to limit the protection scope of the present invention.

[0019] It should be understood that the various steps described in the method embodiments of the present invention can be executed in different orders and / or executed in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present invention is not limited in this regard.

[0020] The term "comprising" and its variations used herein are open-ended, that is, "including but not limited to"; the term "based on" is "at least partially based on"; the term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments"; the term "optionally" means "optional embodiments". The relevant definitions of other terms will be given in the following description. It should be noted that the concepts such as "first" and "second" mentioned in the present invention are only used to distinguish different devices, modules, or units, and are not used to limit the order or interdependence relationship of the functions performed by these devices, modules, or units.

[0021] It should be noted that the modifications of "one" and "multiple" mentioned in the present invention are illustrative rather than restrictive. Those skilled in the art should understand that unless clearly stated otherwise in the context, it should be understood as "one or more".

[0022] The names of the messages or information exchanged between multiple devices in the embodiments of the present invention are only for illustrative purposes and are not used to limit the scope of these messages or information.

[0023] As Figure 1 shown, a water depth detection method provided by an embodiment of the present invention includes: S100: Generate an initial disparity image based on the original images of the target water area from different perspectives that have been acquired.

[0024] Specifically, in this embodiment, drones, satellites or other remote sensing platforms can be used to photograph the target water area from different angles to obtain multiple original images with different perspectives. These original images should have sufficient overlapping areas to ensure that the subsequent stereo matching algorithm can accurately find corresponding points. For example, a high-resolution camera is carried by a drone to photograph the original images of the target water area from multiple angles according to a preset flight path and altitude, ensuring that the resolution and coverage of the original images meet the requirements of detection accuracy. At the same time, information such as the lighting conditions and shooting parameters during shooting is recorded for necessary correction and adjustment in subsequent processing. The original images with different perspectives obtained are preprocessed, including radiometric correction and geometric correction, to eliminate the radiometric distortion and geometric distortion in the original images and ensure the accuracy and consistency of the original images. A dense matching algorithm, such as a block matching algorithm or a semi-global dense matching algorithm, is used to process the preprocessed original images. By comparing the feature similarities of corresponding pixels in the overlapping areas of the original images with different perspectives, the disparity value of each pixel in the overlapping area is calculated, thereby generating an initial disparity image of the overlapping area. This process can provide basic data for subsequent disparity optimization and water depth calculation.

[0025] S200: Determine the first edge feature of the original image of any perspective, and determine the second edge feature of the initial disparity image.

[0026] Specifically, the first edge feature and the second edge feature can be straight line features. A straight line detection algorithm, such as the LSD (Line Segment Detector) operator, is used to process the original image and the initial disparity image to extract the first edge feature and the second edge feature. The LSD operator can automatically detect and extract straight line segments in the initial disparity image. These straight line segments appear as regions of sudden change in disparity values in the initial disparity image, usually corresponding to the straight edges of the underwater terrain or the straight contours of underwater objects. By setting appropriate parameters, such as the minimum length and maximum gap of the straight line, the straight line segments that conform to the characteristics of the underwater terrain or underwater objects are screened out.

[0027] When the first edge feature and the second edge feature are non-straight line features, an edge detection algorithm, such as the Canny edge detection operator or the Sobel operator, is used to process the original image and the initial disparity image. These algorithms can detect regions with significant gray level changes in the image, that is, regions with obvious changes in disparity values, usually corresponding to non-straight edges of the underwater terrain or non-straight contours of underwater objects. Post-processing is performed on the detected edges, such as edge tracking and edge connection, to form complete non-straight line features.

[0028] In some embodiments, the first edge feature and the second edge feature can be the edges of reefs, the contours of sandbars, the shapes of sunken ships, etc.

[0029] S300: Obtain the first disparity values of each pixel within a first preset range around the second edge feature in the initial disparity image, and perform optimization processing on each of the first disparity values to align the first edge feature and the second edge feature, thereby obtaining an intermediate disparity image.

[0030] Specifically, with the second edge feature as the center, set a preset range, for example, a buffer zone with a radius of 10 pixels, as the first preset range. This range can be adjusted according to the actual application scenario and image resolution. In the initial disparity image, extract all pixel points within the first preset range and record their disparity values, that is, the first disparity values. Perform optimization processing on the first disparity values to obtain an intermediate disparity image. Among them, the optimization processing includes constructing a global energy function, taking the first disparity value as the initial disparity, and using the graph cut algorithm to solve the minimum value of the global energy function to obtain the estimated disparity, that is, the second disparity value. By replacing the first disparity values of each pixel within the first preset range around the second edge feature with the second disparity values, an intermediate disparity image is formed.

[0031] S400: Obtain the third disparity value of a preset target point according to the target disparity image, and based on the third disparity value, determine the actual water depth corresponding to the preset target point.

[0032] Specifically, in the intermediate disparity image, extract all pixel points within the first preset range and record their disparity values, that is, the second disparity values. Perform filtering processing on the second disparity values to obtain the third disparity value. By replacing the second disparity values of each pixel within the first preset range around the second edge feature with the third disparity values, a target disparity image is formed. Among them, the filtering processing includes Gaussian filtering, bilateral filtering, non-local mean filtering, etc. Considering that it is necessary to retain detailed information such as the terrain and landforms near the second edge feature, an image-guided filtering method can be selected.

[0033] S500: Obtain the third disparity value of a preset target point according to the target disparity image, and based on the third disparity value, determine the actual water depth corresponding to the preset target point.

[0034] Specifically, on the original image, select target points for which the water depth needs to be detected from the target water area according to the actual detection requirements as the preset target points. These preset target points can be specific positions of interest to users, such as the top of a reef in the target water area, key positions of underwater objects, etc., or multiple points distributed according to a certain grid or rule to obtain water depth data for a large area within the target water area. Map the position information of the preset target points from the original image to the target disparity image to obtain the third disparity value of the preset target points. Use the pre-established mapping relationship between disparity and water depth to obtain the actual water depth of the preset target points.

[0035] In this embodiment, obtaining multiple original images of the same target water area taken from different perspectives helps to provide rich perspective information and reduce the matching error caused by a single perspective. Through a stereo matching algorithm, an initial disparity image is generated based on the multiple original images. The first edge feature of the original image and the second edge feature of the initial disparity image are respectively obtained, and the matching error caused by unclear texture is reduced by aligning the first edge feature with the second edge feature, thereby improving the matching degree of corresponding points in the multiple original images. Among them, the first edge feature and the second edge feature can be the edge line of the underwater terrain, the contour line of underwater objects, the boundary line between water and land, etc. In order to align the first edge feature with the second edge feature, the first disparity values of each pixel within a first preset range around the second edge feature are optimized to obtain an intermediate disparity image. In this way, it can not only reduce the mis-matched pixels caused by unclear texture and reduce the probability of the appearance of the expansion phenomenon, but also retain the detailed information such as the terrain and landform near the second edge feature, avoiding the loss of details caused by forced smoothing. At this time, slight disparity breaks may occur around the second edge feature in the intermediate disparity image, that is, for each pixel within the first preset range and each pixel outside the first preset range. It is necessary to perform filtering processing on the second disparity values of each pixel within the first preset range to obtain a target disparity image, making the disparity change around the second edge feature smoother, reducing the matching error caused by radiation distortion, and at the same time retaining the detailed information such as the terrain and landform near the second edge feature. When the first edge feature is aligned with the second edge feature, a target point is selected on the original image as a preset target point, and the third disparity value of the preset target point is obtained from the target disparity image, and then the actual water depth corresponding to the preset target point can be determined, which is beneficial to improving the accuracy of water depth detection.

[0036] Optionally, as Figure 2 shown, determining the second edge feature of the initial disparity image includes: Detecting the line feature of the initial disparity image. When the first disparity values of each pixel within the first preset range around the line feature are all greater than a preset threshold, the line feature is used as the second edge feature.

[0037] Specifically, in the initial disparity image, a specific algorithm (such as the LSD operator or the Canny edge detection operator) is used to detect line features, that is, potential second edge features. These line features may be the edges of the underwater terrain, the contours of underwater objects, etc. For example, in the initial disparity image of a shallow sea area, line features such as the edges of reefs and the contours of sandbars may be detected. Suppose a straight line segment with a length of 50 pixels is detected as a potential second edge feature. For each detected line feature, the first disparity values of the pixels within the first preset range around it are statistically analyzed. If the first disparity values of these pixels are all less than or equal to the preset threshold, or the first disparity values of some of the pixels are all less than or equal to the preset threshold, it indicates that the disparity change in the first buffer is not obvious. If the first disparity values of these pixels are all greater than the preset threshold, it indicates that there is a significant disparity change in the first buffer, and this line feature is taken as the second edge feature. Since the dense matching window of the stereo image usually uses a 9×9 pixel window, the first preset range can be set as a buffer with a radius of 10 pixels.

[0038] Among them, the preset threshold is set according to the specific application scenario and detection requirements, and is used to distinguish which pixels belong to the edge feature area. In some embodiments, the preset threshold is set to 2.

[0039] In this optional embodiment, by detecting the line features in the initial disparity image and combining the first disparity values of the pixels within the first preset range around them for judgment, the true edge features can be more accurately identified. This method not only depends on the geometric shape of the line features, but also considers the change of the disparity values, thereby reducing the possibility of misjudgment and facilitating the subsequent optimization of the first disparity values of the pixels within the first preset range. Compared with global optimization of the entire initial disparity image, the number of pixels to be processed is reduced, and the computational complexity is reduced. In addition, by setting the preset threshold, the detection criteria for edge features can be adjusted according to different application scenarios, so that this method can maintain good performance in different environments and conditions.

[0040] Optionally, as Figure 2 shown, the optimization process of the first disparity values includes: Construct a global energy function based on the first disparity values of the pixels within the first preset range around the second edge feature in the initial disparity image. Taking the minimization of the global energy function as the optimization goal, solve the global energy function to obtain the second disparity value; among them, the global energy function includes a cost term and a regularization term. The cost term is used to measure the distance between the second disparity value of each pixel and the first disparity value, and the regularization term is used to smoothly constrain the second disparity values of the first target pixel and its neighboring pixels. The first target pixel and its neighboring pixels refer to the pixels located on the same side of the second edge feature.

[0041] Specifically, the global energy function satisfies the following formula: ; Wherein, is a set of second disparity values of all pixels within a first preset range, is the objective function value of the global energy function, is a set of all pixels within a first preset range, is any pixel within a first preset range, i.e., the first target pixel, is corresponding to the disparity value The cost is calculated based on the distance between the second disparity value and the first disparity value, is The confidence level, indicating The reliability of the cost calculation of, and is used to reduce the cost weight of mis-matched points, is a set of all neighboring pixels of the first target pixel, is The neighboring pixel of, is The second disparity value of, is The second disparity value of, is a preset threshold, is the penalty term coefficient, is based on , The weight calculated from the grayscale difference of, is based on , The weight defined by the position of.

[0042] The first term in the formula is the cost term of the global energy function. The cost term takes the distance between the second disparity value and the first disparity value as the basis for cost calculation, and combines the confidence level to retain details such as terrain and landforms near the second edge feature, while avoiding retaining mis-matched points. The cost term satisfies the following formula: ; Wherein, is any pixel within a first preset range, i.e., the first target pixel, is any first disparity value [p1], is The first disparity value of, is a preset threshold, is Corresponding to the disparity value The cost of.

[0043] The confidence level satisfies the following formula: ; Among them, is the distance between the first edge feature and the second edge feature, is the radius of the first preset range. The first preset range can be a rectangular area extending along the second edge feature, indicating half of the width of the rectangular area in the direction perpendicular to the second edge feature, is the first parallax of is the average value of the parallax of pixels with similar gray levels around is a preset threshold, is the confidence level of

[0044] The second term in the formula is the regularization term of the global energy function, which is used to perform smoothing constraints on the second parallax of all pixels within the first preset range. Specifically, it smooths the second parallax of the first target pixel and its neighboring pixels. The first target pixel represents any pixel within the first preset range, and the neighboring pixels of the first target pixel represent any pixel on the same side of the second edge feature as this pixel. If and have a smaller gray level difference, is larger. While being able to correct the mismatched points near the second edge feature, as many details in the buffer area as possible are retained. The weight of the gray level difference satisfies the following formula: ; Among them, represents the gray level of represents the gray level of is the Gaussian kernel function smoothing factor related to the gray level, is based on , the weight calculated from the gray level difference of

[0045] If , are on the same side of the second edge feature, then the weight is 1. If , are on both sides of the second edge feature, the weight is 0. is used to prevent the optimization results on both sides of the second edge feature from influencing each other, so that a sharper edge can be obtained.

[0046] In this optional embodiment, a global energy function is constructed. By means of a cost term, it is ensured that the optimized first disparity value (i.e., the second disparity value) is as close as possible to the first disparity value, thereby retaining the reliability of the original matching result. By means of a regularization term, it is ensured that the disparity change of adjacent pixels on the same side of the second edge feature is smooth, avoiding drastic disparity jumps, thereby improving the continuity and consistency of the matching result. By minimizing this global energy function, an optimal disparity value distribution within the first preset range, that is, the second disparity value, can be obtained, so as to obtain an accurate intermediate disparity image, aligning the first edge feature of the original image with the second edge feature of the intermediate disparity image, and reducing the matching error caused by unclear texture.

[0047] Optionally, taking the minimization of the global energy function as the optimization objective, solving the global energy function to obtain the second disparity value includes: Based on each pixel within the first preset range around the second edge feature, constructing a graph model; Determining the minimum cut of the graph model, and obtaining the minimum value of the global energy function and the second disparity value according to the minimum cut.

[0048] Specifically, constructing a graph model, the graph model includes a source point, a sink point, nodes, first edges, and second edges. The source point corresponds to the label for retaining the first disparity value, the sink point corresponds to the label for optimizing the first disparity value, one node corresponds to one pixel, the first edges connect adjacent pixels, the weight of the first edges is the regularization term, the second edges connect pixels to the source point and the sink point, and the weight of the second edges is the cost term; obtaining the maximum flow from the source point to the sink point, and determining the minimum cut based on the maximum flow; wherein, the weight of the minimum cut is the minimum value of the global energy function, and the result of the minimum cut is the second disparity value.

[0049] Further explanation, a graphical model is constructed. A node is created for each pixel within a first preset range around the second edge feature of the initial disparity image. Adjacent nodes are connected by a first edge, and the weight of the first edge is set as the regularization term of the global energy function. The nodes are connected to the source point and the sink point by a second edge, and the weight of the second edge is set as the cost term of the global energy function. Among them, the source point represents retaining the first disparity value, and the sink point represents optimizing the first disparity value. The maximum flow algorithm (such as the Ford-Fulkerson algorithm or the Edmonds-Karp algorithm, etc.) is used to calculate the maximum flow from the source point to the sink point. The core idea of the maximum flow algorithm is to continuously find augmenting paths to increase the flow until no more augmenting paths can be found. According to the maximum flow minimum cut theorem, that is, the maximum flow is equal to the weight of the minimum cut, the minimum value of the global energy function can be directly obtained. At the same time, after determining the maximum flow, the result of the minimum cut can be obtained, that is, which nodes are connected to the source point and which nodes are connected to the sink point. According to the result of the minimum cut, the second disparity value of each pixel can be determined. If the node of the pixel is assigned to the source point part, the first disparity value of the pixel is used as the second disparity value; if the node of the pixel is assigned to the sink point part, the optimized first disparity value of the pixel is used as the second disparity value.

[0050] In this optional embodiment, by constructing a graphical model and determining the minimum cut of the graphical model, the global energy function can be efficiently minimized to obtain the optimized first disparity value (the second disparity value), so as to obtain the intermediate disparity image, ensuring that the optimized first disparity value not only conforms to the distribution of the first disparity value but also satisfies the constraint condition that the disparity values of adjacent pixels on the same side of the second edge feature are similar, thereby improving the matching degree of corresponding points.

[0051] Optionally, as Figure 2 shown, the filtering the second disparity values to obtain a target disparity image includes: Determining the weight of the second target pixel based on the gray values and distances of the second target pixel and its neighboring pixels; wherein, the second target pixel represents the pixel within the first preset range around the second edge feature; the neighboring pixels represent the pixels within a second preset range around the second edge feature, and the second preset range includes the first preset range; Performing weighted averaging on the second disparity value corresponding to the second target pixel according to the weight of the second target pixel to obtain the target disparity image.

[0052] Specifically, the image-guided filtering method is to perform weighted averaging on the second disparity values of the pixels within a local window. The weight of each pixel within the window depends on the linear model between the local disparity and the gray value. The image-guided filtering operator satisfies the following formula: ; Among them, is any pixel within the first preset range, is the neighboring pixel of is the third disparity value of is the second disparity value of is the set of all pixels within the first preset range, is the set of all pixels within the second preset range, is centered on the filtering window, is and the weight between

[0053] On the basis of determining the second edge feature, the second preset range is determined. The second preset range is centered on the second edge feature and is obtained by further expanding on the basis of the first preset range. The second preset range includes the first preset range. The second preset range is used to provide more context information during the filtering process, which is beneficial to achieving smooth transition of the disparity values inside and outside the first preset range, thereby reducing mismatched pixels and improving the matching degree of corresponding points. When filtering the second disparity value of the pixels within the first preset range, the filtering window moves within the second preset range to constrain and optimize the first disparity (i.e., the second disparity value) after processing.

[0054] In some embodiments, the radius of the second preset range is set to half of the cost window plus half of the first preset range. For example, the cost window adopts a 6×6 pixel window, the first preset range adopts a buffer with a radius of 10 pixels, and the radius of the second buffer is set to 13 pixels.

[0055] In this alternative embodiment, since there may be slight disparity breaks within the second preset range, the second disparity of the pixels within the first preset range is weighted and averaged based on the weights determined by the gray values and distances of the pixels and their neighboring pixels, smoothing the disparity changes, thereby reducing the influence of mismatched pixels and retaining important topographic and geomorphic details. Finally, a more accurate target disparity image is generated, providing a solid foundation for subsequent actual water depth calculation.

[0056] Optionally, determining the actual water depth corresponding to the preset target point based on the third disparity value includes: Obtaining the shooting parameters corresponding to each of the original images, and determining the preset water depth corresponding to the preset target point based on the third disparity value and the shooting parameters; Performing refraction correction on the preset water depth of the target point to obtain the actual water depth corresponding to the preset target point.

[0057] Specifically, when using a drone equipped with a high-resolution camera to obtain the original images of the target water area from the left and right perspectives respectively, the preset water depth of the target point satisfies the following formula: ; Wherein, is the camera focal length, is the baseline distance between the left-view camera and the right-view camera, is the third parallax of the target point, is the preset water depth of the target point.

[0058] Since the propagation of light from air to water will cause refraction, the influence of the law of refraction on water depth detection needs to be considered. The law of refraction satisfies the following formula: ; Wherein, is the refractive index of air, is the refractive index of water, is the incident angle when light enters water from air, is the refraction angle when light enters water from air.

[0059] Determine the incident angle of light according to the shooting angle of the camera and the position of the target point, and use the law of refraction to determine the refraction angle of light. The actual water depth of the target point satisfies the following formula: ; Wherein, is the preset water depth of the target point, is the incident angle when light enters water from air, is the refraction angle when light enters water from air, is the actual water depth of the target point.

[0060] In this optional embodiment, by combining the third parallax value and the shooting parameters, the preset water depth of the target point can be calculated more accurately. The shooting parameters (such as focal length, baseline distance, etc.) provide the necessary geometric information, enabling the parallax information to be accurately converted into depth information. Further refraction correction takes into account the propagation characteristics of light in water, correcting the detection error caused by light refraction, so as to obtain a more accurate actual water depth closer to the true value.

[0061] Optionally, as Figure 2 shown, generating an initial parallax image based on the original images of different perspectives of the acquired target water area includes: Performing epipolar correction on the original image and determining the matching cost of each pixel in the original image; Aggregate the matching cost in multiple directions to generate the initial disparity image, where the first disparity value of the pixel in the initial disparity image corresponds to the minimum value of the matching cost aggregated for the pixel.

[0062] Specifically, multiple original images of the target water area are obtained from different perspectives (such as the left perspective and the right perspective). These original images can be obtained by drones, satellites, or other remote sensing platforms. Perform epipolar rectification on the obtained original images to ensure that corresponding pixels in the original images from different perspectives are located on the same epipolar line. Define a cost function to measure the matching similarity between corresponding pixels in the original images from different perspectives. Common cost functions include the sum of absolute differences (SAD), the sum of squared differences (SSD), and the cross-correlation coefficient, etc. Through this cost function, calculate the matching cost of each pixel in the original images from different perspectives to generate a cost volume. Aggregate the matching cost in multiple directions to optimize the cost space. Commonly used aggregation methods include the scan line method, the dynamic programming method, the path aggregation method in the SGM algorithm, etc. These methods reduce the influence of noise by considering the information of neighboring pixels and improve the accuracy of matching. In the aggregated cost space, find the disparity value corresponding to the minimum matching cost of each pixel as the first disparity value of the pixel. Integrate the first disparity values of all pixels into a two-dimensional image to generate the initial disparity image.

[0063] In this alternative embodiment, epipolar rectification ensures the geometric accuracy of the matching, the matching cost calculation provides a measure of the similarity between pixels, and cost aggregation further improves the reliability and accuracy of the matching, enabling the generated initial disparity image to better reflect the actual disparity distribution, thereby providing reliable data support for subsequent processing.

[0064] As Figure 3 shown, a water depth detection device 300 provided by an embodiment of the present invention includes: A generation module 310, configured to generate an initial disparity image according to original images of the target water area from different perspectives that have been acquired; A first determination module 320, configured to determine the first edge feature of the original image of any perspective, and determine the second edge feature of the initial disparity image; An optimization module 330, configured to obtain the first disparity values of the pixels within a first preset range around the second edge feature in the initial disparity image, and perform optimization processing on the first disparity values to align the first edge feature and the second edge feature to obtain an intermediate disparity image; A filtering module 340, configured to obtain the second disparity values of the pixels within the first preset range around the second edge feature in the intermediate disparity image, and perform filtering processing on the second disparity values to obtain a target disparity image; The second determination module 350 is configured to obtain a third parallax value of a preset target point according to the target parallax image, and determine an actual water depth corresponding to the preset target point based on the third parallax value.

[0065] Optionally, the first determination module 320 is specifically configured to detect a line feature of the initial parallax image, and when the first parallax values of the pixels within the first preset range around the line feature are all greater than a preset threshold, use the line feature as the second edge feature.

[0066] Optionally, the optimization module 330 is specifically configured to construct a global energy function according to the first parallax values of the pixels within the first preset range around the second edge feature in the initial parallax image, and solve the global energy function with the minimization of the global energy function as the optimization objective to obtain the second parallax value; wherein, the global energy function includes a cost term and a regularization term, the cost term is used to measure the distance between the second parallax value of each pixel and the first parallax value, and the regularization term is used to smooth and constrain the second parallax values of the first target pixel and its neighboring pixels, and the first target pixel and its neighboring pixels refer to the pixels located on the same side of the second edge feature.

[0067] Optionally, the optimization module 330 includes a processing module, and the processing module is specifically configured to construct a graphical model based on the pixels within the first preset range around the second edge feature; determine a minimum cut of the graphical model, and obtain the minimum value of the global energy function and the second parallax value according to the minimum cut.

[0068] Optionally, the filtering module 340 is specifically configured to determine the weight of a second target pixel based on the gray value and distance of the second target pixel and its neighboring pixels; wherein, the second target pixel refers to the pixel located within the first preset range around the second edge feature; the neighboring pixels refer to the pixels located within a second preset range around the second edge feature, and the second preset range includes the first preset range; perform weighted averaging on the second parallax value corresponding to the second target pixel according to the weight of the second target pixel to obtain the target parallax image.

[0069] Optionally, the second determination module 350 is specifically configured to obtain the shooting parameters corresponding to each of the original images, and determine a preset water depth corresponding to the preset target point based on the third parallax value and the shooting parameters; perform refraction correction on the preset water depth of the target point to obtain the actual water depth corresponding to the preset target point.

[0070] Optionally, the generating module 310 is specifically configured to perform epipolar correction on the original image and determine the matching cost of each pixel in the original image; aggregate the matching cost in multiple directions to generate the initial disparity image, where the first disparity value of the pixel in the initial disparity image corresponds to the minimum value of the aggregated matching cost of the pixel.

[0071] As Figure 4 shown, an electronic device 400 provided by an embodiment of the present invention includes a memory 410 and a processor 420; the memory 410 is used to store a computer program; the processor 420 is used to implement the water depth detection method as described above when executing the computer program.

[0072] Or, an electronic device 400 includes a memory 410 and a processor 420 coupled to the memory 410; the memory 410 is configured to store a computer program; the processor 420 is configured to perform the following operations when executing the computer program: Generate an initial disparity image based on the original images of different perspectives of the target water area that have been acquired; Determine the first edge feature of the original image of any perspective, and determine the second edge feature of the initial disparity image; Obtain the first disparity values of the pixels within a first preset range around the second edge feature in the initial disparity image, and perform optimization processing on each of the first disparity values to align the first edge feature and the second edge feature, obtaining an intermediate disparity image; Obtain the second disparity values of the pixels within the first preset range around the second edge feature in the intermediate disparity image, and perform filtering processing on each of the second disparity values to obtain a target disparity image; Obtain the third disparity value of a preset target point based on the target disparity image, and determine the actual water depth corresponding to the preset target point based on the third disparity value.

[0073] A computer-readable storage medium provided by an embodiment of the present invention has a computer program stored thereon. When the computer program is executed by a processor, the water depth detection method as described above is implemented.

[0074] Or, a non-volatile computer-readable storage medium has a computer program stored thereon. When the computer program is executed by a processor, the processor is caused to perform the following operations: Generate an initial disparity image based on the original images of different perspectives of the target water area that have been acquired; Determine the first edge feature of the original image of any perspective, and determine the second edge feature of the initial disparity image; Obtain the first disparity values of each pixel within a first preset range around the second edge feature in the initial disparity image, and perform optimization processing on each of the first disparity values to align the first edge feature and the second edge feature, obtaining an intermediate disparity image; Obtain the second disparity values of each pixel within the first preset range around the second edge feature in the intermediate disparity image, and perform filtering processing on each of the second disparity values to obtain a target disparity image; Obtain the third disparity value of a preset target point according to the target disparity image, and determine the actual water depth corresponding to the preset target point based on the third disparity value.

[0075] Now, an electronic device 400 that can be a server or a client of the present invention will be described. It is an example of a hardware device that can be applied to various aspects of the present invention. The electronic device 400 is intended to represent various forms of digital electronic computer devices, such as, a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device 400 can also represent various forms of mobile devices, such as, a personal digital processor, a cellular phone, a smart phone, a wearable device, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0076] The electronic device 400 includes a computing unit that can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) or a computer program loaded from a storage unit into a random access memory (RAM). In the RAM, various programs and data required for device operation can also be stored. The computing unit, the ROM, and the RAM are connected to each other via a bus. An input / output (I / O) interface is also connected to the bus.

[0077] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM), etc. In this application, the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiments of the present invention. In addition, the functional units in the various embodiments of the present invention can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above integrated units can be implemented in the form of hardware or in the form of software functional units.

[0078] Although the present invention is disclosed as above, the scope of protection of the present invention is not limited thereto. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention, and these changes and modifications will all fall within the scope of protection of the present invention.

Claims

1. A water depth detection method, characterized in that, Including: Generating an initial disparity image based on original images of different perspectives of the acquired target water area; Determining a first edge feature of the original image of any perspective and determining a second edge feature of the initial disparity image; Obtaining first disparity values of each pixel within a first preset range around the second edge feature in the initial disparity image, and performing an optimization process on each of the first disparity values to align the first edge feature and the second edge feature, thereby obtaining an intermediate disparity image; Obtaining second disparity values of each pixel within the first preset range around the second edge feature in the intermediate disparity image, and performing a filtering process on each of the second disparity values to obtain a target disparity image; Obtaining a third disparity value of a preset target point according to the target disparity image, and determining an actual water depth corresponding to the preset target point based on the third disparity value.

2. The water depth detection method according to claim 1, characterized in that The determining of the second edge feature of the initial disparity image includes: Detecting a line feature of the initial disparity image, and when the first disparity values of each pixel within the first preset range around the line feature are all greater than a preset threshold, using the line feature as the second edge feature.

3. The water depth detection method according to claim 1, characterized in that, The performing of the optimization process on each of the first disparity values includes: Constructing a global energy function according to the first disparity values of each pixel within the first preset range around the second edge feature in the initial disparity image, taking the minimization of the global energy function as an optimization objective, and solving the global energy function to obtain the second disparity value; wherein, the global energy function includes a cost term and a regularization term, the cost term is used to measure the distance between the second disparity value of each pixel and the first disparity value, and the regularization term is used to smoothly constrain the second disparity values of a first target pixel and its neighboring pixels, and the first target pixel and its neighboring pixels represent the pixels located on the same side of the second edge feature.

4. The water depth detection method according to claim 3, wherein The taking of the minimization of the global energy function as an optimization objective and solving the global energy function to obtain the second disparity value includes: Constructing a graph model based on each pixel within the first preset range around the second edge feature; Determining a minimum cut of the graph model, and obtaining the minimum value of the global energy function and the second disparity value according to the minimum cut.

5. The water depth detection method according to claim 1, characterized in that, The performing of the filtering process on each of the second disparity values to obtain a target disparity image includes: Determining a weight of a second target pixel based on gray values and distances of the second target pixel and its neighboring pixels; wherein, the second target pixel represents a pixel located within the first preset range around the second edge feature; the neighboring pixels represent pixels located within a second preset range around the second edge feature, and the second preset range includes the first preset range; Performing a weighted average on the second disparity value corresponding to the second target pixel according to the weight of the second target pixel to obtain the target disparity image.

6. The water depth detection method according to claim 1, wherein The determining of the actual water depth corresponding to the preset target point based on the third disparity value includes: Obtain the shooting parameters corresponding to each of the original images, and determine the preset water depth corresponding to the preset target point based on the third parallax value and the shooting parameters; Perform refraction correction on the preset water depth of the target point to obtain the actual water depth corresponding to the preset target point.

7. The water depth detection method according to claim 1, characterized in that The generating the initial parallax image according to the original images of different perspectives of the target water area that have been obtained includes: Perform epipolar correction on the original image and determine the matching cost of each pixel in the original image; Aggregate the matching costs in multiple directions to generate the initial parallax image, where the first parallax value of the pixel in the initial parallax image corresponds to the minimum value of the aggregated matching cost of the pixel.

8. A water depth detection device, characterized in that, Including: A generating module, configured to generate an initial parallax image according to the original images of different perspectives of the target water area that have been obtained; A first determining module, configured to determine the first edge feature of the original image of any perspective and determine the second edge feature of the initial parallax image; An optimizing module, configured to obtain the first parallax values of the pixels within a first preset range around the second edge feature in the initial parallax image, and perform optimization processing on each of the first parallax values to align the first edge feature and the second edge feature to obtain an intermediate parallax image; A filtering module, configured to obtain the second parallax values of the pixels within the first preset range around the second edge feature in the intermediate parallax image, and perform filtering processing on each of the second parallax values to obtain a target parallax image; A second determining module, configured to obtain the third parallax value of the preset target point according to the target parallax image, and determine the actual water depth corresponding to the preset target point based on the third parallax value.

9. An electronic device, characterized in that, Including a memory and a processor; The memory is used to store a computer program; The processor is configured to, when executing the computer program, implement the water depth detection method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, A computer program is stored on the storage medium, and when the computer program is executed by the processor, the water depth detection method according to any one of claims 1 to 7 is implemented.

Images