A water depth detection method and device, electronic equipment and storage medium

Abstract

The application provides a water depth detection method and device, electronic equipment and storage medium, and relates to the technical field of detection. The water depth detection method comprises the following steps: generating an initial parallax image according to an original image of a target water area at different perspectives; determining a first edge feature of the original image and a second edge feature of the initial parallax image; obtaining a first parallax value of each pixel within a first preset range around the second edge feature in the initial parallax image, and performing optimization processing on each first parallax value to align the first edge feature and the second edge feature, thereby obtaining an intermediate parallax image; obtaining a second parallax value of each pixel within the first preset range around the second edge feature in the intermediate parallax image, and performing filtering processing on each second parallax value, thereby obtaining a target parallax image; obtaining a third parallax value of a preset target point according to the target parallax image, and determining an actual water depth of the preset target point based on the third parallax value. The application improves the accuracy of water depth detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of detection, in particular to a water depth detection method and device, electronic equipment and storage medium. BACKGROUND

[0002] Detecting water depth is not only a basic work of marine scientific research, but also an important means to protect ecological environment and cope with climate change. In recent years, with the progress of computer vision technology and image processing algorithm, water depth detection methods based on optical images have gradually attracted attention. This method analyzes stereo images taken from different angles, uses parallax information to reconstruct three-dimensional terrain, and then calculates water depth data. This non-contact detection method not only has low cost, but also can be used with existing remote sensing platforms (such as unmanned aerial vehicles, satellites, etc.), and has broad application prospects.

[0003] In related technologies, due to the low visibility under water, combined with the influence of factors such as atmosphere, solar flare, water turbidity and bottom type, the texture of the obtained underwater stereo image is not clear and is prone to radiation deformation, resulting in poor accuracy of water depth detection. SUMMARY

[0004] The problem solved by the present application is how to improve the accuracy of water depth detection.

[0005] To solve the above problems, the present application provides a water depth detection method, device, electronic equipment and storage medium.

[0006] In a first aspect, the present application provides a water depth detection method, comprising:

[0007] Generating an initial parallax image according to the original images of different angles of view of the target water area obtained;

[0008] Determining the first edge feature of the original image of any angle of view, and determining the second edge feature of the initial parallax image;

[0009] Obtaining the first parallax value of each pixel within the first preset range around the second edge feature in the initial parallax image, and optimizing each first parallax value to align the first edge feature and the second edge feature, to obtain an intermediate parallax image;

[0010] Obtaining the second parallax value of each pixel within the first preset range around the second edge feature in the intermediate parallax image, and filtering each second parallax value to obtain a target parallax image;

[0011] Obtaining the third parallax value of the preset target point according to the target parallax image, and determining the actual water depth corresponding to the preset target point based on the third parallax value.

[0012] Optionally, the determining the second edge feature of the initial disparity image comprises:

[0013] detecting a line feature of the initial disparity image, and regarding the line feature as the second edge feature when the first disparity values of pixels within a first preset range around the line feature are all greater than a preset threshold.

[0014] Optionally, the optimizing each of the first disparity values comprises:

[0015] constructing a global energy function according to the first disparity values of pixels within the first preset range around the second edge feature in the initial disparity image, solving the global energy function with minimization of the global energy function as an optimization target, and obtaining the second disparity value; wherein the global energy function comprises a cost term and a regularization term, the cost term is used to measure distances between the second disparity value of each of the pixels and the first disparity value, and the regularization term is used to smooth the second disparity value of a first target pixel and its neighborhood pixels, the first target pixel and its neighborhood pixels represent the pixels on the same side of the second edge feature.

[0016] Optionally, the solving the global energy function with minimization of the global energy function as an optimization target, and obtaining the second disparity value comprises:

[0017] constructing a graph model based on the pixels within the first preset range around the second edge feature;

[0018] determining a minimum cut of the graph model, and obtaining the minimum value of the global energy function and the second disparity value according to the minimum cut.

[0019] Optionally, the filtering each of the second disparity values to obtain a target disparity image comprises:

[0020] determining a weight of a second target pixel based on a gray value and a distance of the second target pixel and its neighborhood pixels; wherein the second target pixel represents the pixels within the first preset range around the second edge feature, and the neighborhood pixels represent the pixels within a second preset range around the second edge feature, the second preset range comprises the first preset range.

[0021] performing weighted average on the second disparity value corresponding to the second target pixel according to the weight of the second target pixel, and obtaining the target disparity image.

[0022] Optionally, the determining the actual water depth corresponding to the preset target point based on the third disparity value comprises:

[0023] obtaining a shooting parameter corresponding to each of the original images, and determining a preset water depth corresponding to the preset target point based on the third disparity value and the shooting parameter;

[0024] performing refraction correction on the preset water depth of the target point to obtain the actual water depth corresponding to the preset target point.

[0025] Optionally, the generating an initial disparity image according to the original images of different perspectives of the target water area comprises:

[0026] performing epipolar correction on the original images and determining a matching cost of each pixel in the original images;

[0027] aggregating the matching cost in multiple directions to generate the initial disparity image, wherein the first disparity value of the pixel in the initial disparity image corresponds to a minimum value of the aggregated matching cost of the pixel.

[0028] In a second aspect, the present application provides a water depth detection device, comprising:

[0029] a generating module configured to generate an initial disparity image according to original images of different perspectives of a target water area;

[0030] a first determining module configured to determine a first edge feature of the original image of any perspective and determine a second edge feature of the initial disparity image;

[0031] an optimizing module configured to obtain a first disparity value of each pixel within a first preset range around the second edge feature in the initial disparity image, and perform optimization processing on each of the first disparity values to align the first edge feature and the second edge feature, thereby obtaining an intermediate disparity image;

[0032] a filtering module configured to obtain a second disparity value of each pixel within the first preset range around the second edge feature in the intermediate disparity image, and perform filtering processing on each of the second disparity values, thereby obtaining a target disparity image;

[0033] a second determining module configured to obtain a third disparity value of a preset target point according to the target disparity image, and determine an actual water depth corresponding to the preset target point based on the third disparity value.

[0034] In a second aspect, the present application provides a water depth detection device, comprising:

[0035] In a third aspect, the present application provides an electronic device comprising a memory and a processor.

[0036] The memory is configured to store a computer program.

[0037] The processor is configured to implement the water depth detection method according to the first aspect when executing the computer program.

[0038] In a fourth aspect, the present application provides a computer readable storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the water depth detection method according to the first aspect is implemented.

[0039] The water depth detection method, device, electronic equipment and storage medium have the following advantages: multiple original images of the same target water area are obtained from different perspectives, which helps to provide rich perspective information and reduce matching errors caused by a single perspective. An initial disparity image is generated from the multiple original images by using a stereo matching algorithm. First edge features of any original image and second edge features of the initial disparity image are obtained respectively. The first edge features are aligned with the second edge features, which reduces matching errors caused by unclear textures and improves the matching degree of corresponding points of the multiple original images. The first edge features and the second edge features can be edge lines of the underwater terrain, contour lines of underwater objects, and boundary lines between water and land, etc. In order to align the first edge features with the second edge features, the first disparity values of the pixels in a first preset range around the second edge features are optimized to obtain an intermediate disparity image. In this way, the number of mismatched pixels caused by unclear textures is reduced, the probability of the occurrence of the expansion phenomenon is reduced, and the details such as the terrain and topography near the second edge features are retained, thereby avoiding the loss of details caused by forced flattening. On this basis, the pixels in the first preset range and the pixels outside the first preset range around the second edge features in the intermediate disparity image may have slight disparity breaks, and the second disparity values of the pixels in the first preset range need to be filtered to obtain a target disparity image, so that the disparity around the second edge features changes more smoothly, the matching errors caused by radiation deformation are reduced, and the details such as the terrain and topography near the second edge features are retained. In this way, when the first edge features are aligned with the second edge features, a target point is selected as a preset target point on the original image, and a third disparity value of the preset target point is obtained from the target disparity image, so that the actual water depth corresponding to the preset target point is determined, which is beneficial to improving the accuracy of water depth detection. BRIEF DESCRIPTION OF DRAWINGS

[0040] Figure 1 A flowchart of a water depth detection method according to an embodiment of the present application is shown in FIG. 1.

[0041] Figure 2 A flowchart of a water depth detection method according to another embodiment of the present application is shown in FIG. 2.

[0042] Figure 3A system architecture diagram of the water depth detection device according to an embodiment of the present application;

[0043] Figure 4 A system architecture diagram of the electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0044] In order to make the above objectives, features and advantages of the present application more apparent, specific embodiments of the present application will be described in detail below with reference to the accompanying drawings. Although some embodiments of the present application are illustrated in the drawings, it is understood that the present application can be implemented in various forms, and should not be interpreted as being limited to the embodiments set forth herein, but rather, these embodiments are provided so as to more completely and thoroughly understand the present application. It is understood that the drawings and embodiments of the present application are for illustrative purposes only, and are not intended to limit the scope of the present application.

[0045] It should be understood that the various steps of the method embodiments of the present application can be performed in different orders, and / or in parallel. In addition, the method embodiments can include additional steps and / or omit the steps shown. The scope of the present application is not limited in this respect.

[0046] The term "comprising" and variations thereof as used herein are open-ended, that is, "comprising but not limited to"; the term "based on" is "based at least in part on"; the term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments"; the term "optional" means "optional in at least some embodiments". Related definitions are given throughout the description. It is noted that the concepts of "first", "second", etc. mentioned in the present application are only used to distinguish different devices, modules or units, and are not intended to limit the order or interdependence of the functions performed by these devices, modules or units.

[0047] It should be noted that the modification of "one" or "multiple" mentioned in the present application is illustrative rather than limiting, and those skilled in the art should understand that, unless otherwise explicitly indicated in the context, it should be understood as "one or more".

[0048] The names of the messages or information exchanged between the devices in the embodiments of the present application are only for illustrative purposes, and are not intended to limit the scope of the messages or information.

[0049] As Figure 1 shown, the water depth detection method according to an embodiment of the present application comprises:

[0050] S100: generating an initial parallax image according to the original images of different perspectives of the target water area that have been acquired.

[0051] Specifically, the present embodiment can utilize unmanned aerial vehicles, satellites, or other remote sensing platforms to capture multiple original images of the target water area from different angles. These original images should have sufficient overlapping areas to ensure that subsequent stereo matching algorithms can accurately find corresponding points. For example, using an unmanned aerial vehicle equipped with a high-resolution camera, multiple angle original images of the target water area are captured according to a predetermined flight path and height, ensuring that the resolution and coverage of the original images meet the requirements of detection accuracy. At the same time, information such as light conditions and shooting parameters during shooting is recorded for necessary correction and adjustment in subsequent processing. The original images obtained from different angles are preprocessed, including radiation correction and geometric correction, to eliminate radiation distortion and geometric distortion in the original images, ensuring the accuracy and consistency of the original images. A dense matching algorithm, such as a block matching algorithm or a semi-global dense matching algorithm, is used to process the preprocessed original images. By comparing the feature similarity of corresponding pixels in the overlapping areas of different angle original images, the disparity value of each pixel in the overlapping area is calculated, thereby generating an initial disparity image of the overlapping area. This process can provide basic data for subsequent disparity optimization and water depth calculation.

[0052] S200: Determine the first edge feature of any angle of the original image, and determine the second edge feature of the initial disparity image.

[0053] Specifically, the first edge feature and the second edge feature can be straight line features. A straight line detection algorithm, such as the LSD (Line Segment Detector) operator, is used to process the original image and the initial disparity image to extract the first edge feature and the second edge feature. The LSD operator can automatically detect and extract straight line segments in the initial disparity image. These straight line segments appear as abrupt changes in disparity values in the initial disparity image, and usually correspond to straight line edges of the water bottom terrain or straight line profiles of underwater objects. By setting appropriate parameters, such as the minimum length of the straight line and the maximum gap, straight line segments that meet the characteristics of the water bottom terrain or underwater objects are selected.

[0054] When the first edge feature and the second edge feature are non-straight line features, an edge detection algorithm, such as the Canny edge detection operator or the Sobel operator, is used to process the original image and the initial disparity image. These algorithms can detect areas in the image where the gray scale changes significantly, i.e., areas where the disparity value changes significantly, which usually correspond to non-straight line edges of the water bottom terrain or non-straight line profiles of underwater objects. Post-processing is performed on the detected edges, such as edge tracking and edge connection, to form complete non-straight line features.

[0055] In some embodiments, the first edge feature and the second edge feature can be the edges of reefs, the profiles of sandbars, the shapes of sunken ships, etc.

[0056] S300: Obtain first disparity values of pixels in a first preset range around the second edge feature in the initial disparity image, and perform optimization processing on each of the first disparity values to align the first edge feature and the second edge feature, to obtain an intermediate disparity image.

[0057] Specifically, a preset range, for example, a buffer area with a radius of 10 pixels, is set as the first preset range, with the second edge feature as the center. This range can be adjusted according to the actual application scenario and image resolution. In the initial disparity image, all pixel points in the first preset range are extracted, and their disparity values, i.e., first disparity values, are recorded. The first disparity values are optimized to obtain an intermediate disparity image. The optimization processing includes constructing a global energy function, using the first disparity values as initial disparities, and using a graph cut algorithm to solve the minimum value of the global energy function to obtain an estimated disparity, i.e., a second disparity value. The intermediate disparity image is constructed by replacing the first disparity values of the pixels in the first preset range around the second edge feature with the second disparity values.

[0058] S400: Obtain a third disparity value of a preset target point according to the target disparity image, and determine an actual water depth corresponding to the preset target point based on the third disparity value.

[0059] Specifically, in the intermediate disparity image, all pixel points in the first preset range are extracted, and their disparity values, i.e., second disparity values, are recorded. The second disparity values are filtered to obtain third disparity values. The target disparity image is constructed by replacing the second disparity values of the pixels in the first preset range around the second edge feature with the third disparity values. The filtering processing includes Gaussian filtering, bilateral filtering, non-local mean filtering, etc. Considering the need to preserve details such as topography near the second edge feature, an image-guided filtering method can be selected.

[0060] S500: Obtain a third disparity value of a preset target point according to the target disparity image, and determine an actual water depth corresponding to the preset target point based on the third disparity value.

[0061] Specifically, on the original image, according to the actual detection requirements, target points in the target water area that need to be detected for water depth are selected as preset target points. These preset target points can be specific locations of interest to the user, such as the top of a reef in the target water area, key positions of underwater objects, etc., or a plurality of points distributed according to a certain grid or rule to obtain water depth data in a large area of the target water area. The position information of the preset target points is mapped from the original image to the target disparity image to obtain third disparity values of the preset target points. Using a pre-established mapping relationship between disparity and water depth, the actual water depth of the preset target points is obtained.

[0062] In this embodiment, multiple original images of the same target water area are captured from different perspectives, which helps to provide rich perspective information and reduce matching errors caused by a single perspective. An initial disparity image is generated from the multiple original images by a stereo matching algorithm. First edge features of the original images and second edge features of the initial disparity image are obtained respectively. The matching errors caused by unclear textures are reduced by aligning the first edge features with the second edge features, thereby improving the matching degree of the corresponding points of the multiple original images. The first edge features and the second edge features can be edge lines of the underwater terrain, contour lines of underwater objects, and boundary lines between water and land, etc. In order to align the first edge features with the second edge features, the first disparity values of the pixels in a first preset range around the second edge features are optimized to obtain an intermediate disparity image. In this way, the mismatched pixels caused by unclear textures are reduced, the probability of the occurrence of the expansion phenomenon is reduced, and the detailed information of the terrain and topography around the second edge features is retained, thereby avoiding the loss of details caused by forced flattening. At this time, the second edge features around the intermediate disparity image, i.e. the pixels in the first preset range and the pixels outside the first preset range, may have slight disparity breaks. The second disparity values of the pixels in the first preset range need to be filtered to obtain a target disparity image, so that the disparity around the second edge features changes more smoothly, the matching errors caused by radiation deformation are reduced, and the detailed information of the terrain and topography around the second edge features is retained. When the first edge features are aligned with the second edge features, a target point is selected as a preset target point on the original image, and a third disparity value of the preset target point is obtained from the target disparity image, so as to determine the actual water depth corresponding to the preset target point, which is beneficial to improve the accuracy of water depth detection.

[0063] Optionally, as shown in Figure 2 the method further includes:

[0064] detecting a line feature of the initial disparity image, and when the first disparity values of the pixels in the first preset range around the line feature are all greater than a preset threshold, regarding the line feature as the second edge feature.

[0065] Specifically, in the initial disparity image, line features, i.e. potential second edge features, are detected using a specific algorithm, such as the LSD operator or the Canny edge detection operator. These line features can be edges of the underwater terrain, outlines of underwater objects, etc. For example, in the initial disparity image of a shallow sea area, line features such as edges of a reef, outlines of a sandbank, etc. can be detected. Suppose a straight line segment with a length of 50 pixels is detected as a potential second edge feature. For each detected line feature, the first disparity values of the pixels within a first preset range around the line feature are counted. If the first disparity values of all the pixels are less than or equal to a preset threshold, or the first disparity values of part of the pixels are less than or equal to the preset threshold, it indicates that the disparity change in the first buffer is not obvious. If the first disparity values of all the pixels are greater than the preset threshold, it indicates that there is a significant disparity change in the first buffer, and the line feature is taken as a second edge feature. Since the matching window of the stereo image is usually a 9x9 pixel window, the first preset range can be set as a buffer with a radius of 10 pixels.

[0066] wherein the preset threshold is set according to a specific application scenario and detection requirement, and is used to distinguish which pixels belong to the edge feature region. In some embodiments, the preset threshold is set to 2.

[0067] In this optional embodiment, by detecting the line features in the initial disparity image and combining the first disparity values of the pixels within a first preset range around the line features for judgment, the real edge features can be more accurately identified. This method not only depends on the geometric shape of the line features, but also considers the change of the disparity values, thereby reducing the possibility of misjudgment and facilitating the subsequent optimization of the first disparity values of the pixels within the first preset range. Compared with global optimization of the entire initial disparity image, the number of pixels to be processed is reduced, and the computational complexity is reduced. In addition, by setting the preset threshold, the detection standard of the edge features can be adjusted according to different application scenarios, so that the method can maintain good performance under different environments and conditions.

[0068] Optionally, as shown in Figure 2 the optimization processing of the first disparity values comprises:

[0069] The global energy function is constructed according to the first disparity value of each pixel in the first preset range around the second edge feature in the initial disparity image, and the second disparity value is obtained by solving the global energy function with the minimization of the global energy function as the optimization objective; wherein the global energy function comprises a cost term and a regularization term, the cost term is used to measure the distance between the second disparity value of each pixel and the first disparity value, and the regularization term is used to smooth the second disparity value of the first target pixel and its neighborhood pixels, and the first target pixel and its neighborhood pixels represent the pixels located on the same side of the second edge feature.

[0070] Specifically, the global energy function satisfies the following formula:

[0071] ;

[0072] wherein, is a set of second disparity values of all pixels in the first preset range, is a target function value of the global energy function, is a set of all pixels in the first preset range, is any one pixel in the first preset range, i.e. the first target pixel, is corresponding to the disparity value , which is calculated according to the distance between the second disparity value and the first disparity value, is the confidence of , which represents the reliability of the cost calculation of , and is used to reduce the cost weight of the mismatched point, is a set of all neighborhood pixels of the first target pixel, is the neighborhood pixel of , is the second disparity value of , is the second disparity value of , is a preset threshold, is a penalty term coefficient, is a weight calculated according to the gray difference of , is a weight defined according to the position of , , .

[0073] The first term in the formula is the cost term of the global energy function. The cost term takes the distance between the second disparity value and the first disparity value as the basis for cost calculation, and combines the confidence to retain the detail information such as terrain and topography near the second edge feature, while avoiding retaining the mismatched point. The cost term satisfies the following formula:

[0074] ;

[0075] wherein, is any one pixel in the first preset range, i.e., a first target pixel, is any one first disparity value [p1], is the first disparity value of , and is a preset threshold, is the cost corresponding to the disparity value of .

[0076] The confidence satisfies the following formula:

[0077] ;

[0078] wherein, is the distance between to the second edge feature, is the radius of the first preset range, and the first preset range can be a rectangular area extending along the second edge feature, and represents half of the width of the rectangular area perpendicular to the direction of the second edge feature, is the first disparity value of , and is the average of the disparity values of the surrounding pixels of with similar gray levels, is a preset threshold, is the confidence of .

[0079] The second term in the formula is a regularization term of the global energy function, which is used to perform smoothing constraint on the second disparity values of all pixels in the first preset range. Specifically, the second disparity values of the first target pixel and its neighborhood pixels are smoothed, wherein the first target pixel represents any one pixel in the first preset range, and the neighborhood pixels of the first target pixel represent any one pixel located on the same side of the second edge feature as the first target pixel. If and the smaller the gray level difference is, the larger the weight is. While being able to correct the mismatched points near the second edge feature, as much detail information in the buffer area as possible is retained. The weight of the gray level difference satisfies the following formula:

[0080] ;

[0081] wherein, represents the gray level of , and represents the gray level of a gray scale difference between the first edge feature and the second edge feature, is a Gaussian kernel function smoothing factor related to the gray scale, is a cost term according to , is a weight calculated according to the gray scale difference.

[0082] If the second edge feature is located on the same side of the first edge feature, the weight , is 1. If the second edge feature is located on both sides of the first edge feature, the weight is 0. , is 0. The optional embodiment is used to prohibit the optimization results on both sides of the second edge feature from affecting each other, so that a more sharp edge can be obtained.

[0083] In the optional embodiment, a global energy function is constructed, a cost term is used to ensure that the first disparity value after optimization (i.e. the second disparity value) is as close as possible to the first disparity value, so as to retain the reliability of the original matching result, and a regularization term is used to ensure that the disparity of adjacent pixels on the same side of the second edge feature is smooth, so as to avoid sharp disparity jumps, thereby improving the continuity and consistency of the matching result. By minimizing the global energy function, the optimal disparity value distribution in the first preset range, i.e. the second disparity value, can be obtained, so that an accurate intermediate disparity image is obtained, the first edge feature of the original image is aligned with the second edge feature of the intermediate disparity image, and the matching error caused by unclear texture is reduced.

[0084] Optionally, the global energy function is solved by taking the minimization of the global energy function as an optimization goal, and the second disparity value is obtained, including:

[0085] a graph model is constructed based on each pixel in the first preset range around the second edge feature;

[0086] a minimum cut of the graph model is determined, and the minimum value of the global energy function and the second disparity value are obtained according to the minimum cut.

[0087] Specifically, the graph model is constructed, the graph model includes a source point, a sink point, a node, a first edge and a second edge, the source point corresponds to a label for retaining the first disparity value, the sink point corresponds to a label for optimizing the first disparity value, one node corresponds to one pixel, the first edge connects adjacent pixels, the weight of the first edge is the regularization term, the second edge connects the pixel and the source point and the sink point, and the weight of the second edge is the cost term; the maximum flow from the source point to the sink point is obtained, and the minimum cut is determined based on the maximum flow; wherein the weight of the minimum cut is the minimum value of the global energy function, and the result of the minimum cut is the second disparity value.

[0088] ​Further, a graph model is constructed. A node is created for each pixel within a first preset range around the second edge feature of the initial disparity image. Adjacent nodes are connected by a first edge, and the weight of the first edge is set as a regularization term of the global energy function. The nodes and the source point and the sink point are connected by a second edge, and the weight of the second edge is set as a cost term of the global energy function. The source point represents retaining the first disparity value, and the sink point represents optimizing the first disparity value. A maximum flow algorithm (such as Ford-Fulkerson algorithm or Edmonds-Karp algorithm, etc.) is used to calculate the maximum flow from the source point to the sink point. The core idea of the maximum flow algorithm is to increase the flow by constantly finding an augmenting path until no augmenting path can be found. According to the maximum flow minimum cut theorem, the maximum flow is equal to the weight of the minimum cut, and the minimum value of the global energy function is directly obtained. At the same time, after the maximum flow is determined, the result of the minimum cut is obtained, that is, which nodes are connected to the source point and which nodes are connected to the sink point. According to the result of the minimum cut, the second disparity value of each pixel can be determined. If the node of the pixel is divided into the source point part, the first disparity value of the pixel is taken as the second disparity value; if the node of the pixel is divided into the sink point part, the optimized first disparity value of the pixel is taken as the second disparity value.

[0089] In this optional embodiment, by constructing a graph model and determining the minimum cut of the graph model, the global energy function can be efficiently minimized to obtain the optimized first disparity value (second disparity value), thereby obtaining the intermediate disparity image, and ensuring that the optimized first disparity value not only conforms to the distribution of the first disparity value, but also satisfies the constraint condition that the disparity values of adjacent pixels on the same side of the second edge feature are similar, thereby improving the matching degree of the corresponding points.

[0090] Optionally, as shown in Figure 2 the filtering processing on the second disparity values to obtain a target disparity image, includes:

[0091] determining a weight of a second target pixel based on the gray values and distances of the second target pixel and its neighborhood pixels; wherein the second target pixel represents the pixel located within the first preset range around the second edge feature; and the neighborhood pixels represent the pixels located within a second preset range around the second edge feature, and the second preset range includes the first preset range;

[0092] performing weighted average on the second disparity value corresponding to the second target pixel according to the weight of the second target pixel to obtain the target disparity image.

[0093] Specifically, the image-guided filtering method is to perform weighted average on the second disparity values of the pixels within a local window. The weight of each pixel within the window depends on the linear model between the local disparity and the gray value. The image-guided filtering operator satisfies the following formula:

[0094] ;

[0095] wherein, is any one pixel in the first preset range, is a neighborhood pixel of , is a third disparity value of , is a second disparity value of , is a set of all pixels in the first preset range, is a set of all pixels in the second preset range, is a filter window centered at , is a weight between and .

[0096] On the basis of determining the second edge feature, the second preset range is determined. The second preset range is centered at the second edge feature and is obtained by further expanding on the basis of the first preset range. The second preset range contains the first preset range. The second preset range is used to provide more context information in the process of filtering processing, which is conducive to realizing smooth transition of the disparity value outside the first preset range, thereby reducing the mismatched pixels and improving the matching degree of the homonym point. When the second disparity value of the pixel in the first preset range is filtered, the filter window moves in the second preset range, and the first disparity after optimization (i.e., the second disparity value) is constrained.

[0097] In some embodiments, the radius of the second preset range is set to be half of the cost window plus half of the first preset range. For example, the cost window adopts a 6x6 pixel window, the first preset range adopts a buffer zone with a radius of 10 pixels, and the radius of the second buffer zone is set to 13 pixels.

[0098] In this optional embodiment, since a slight disparity break may occur in the second preset range, the second disparity of the pixel in the first preset range is weighted and averaged by the weight determined based on the gray value and distance of the pixel and its neighborhood pixel, the disparity change is smoothed, the influence of the mismatched pixels is reduced, and important topographic details are preserved. Finally, a more accurate target disparity image is generated, which provides a solid foundation for subsequent actual water depth calculation.

[0099] Optionally, the determining the actual water depth corresponding to the preset target point based on the third disparity value comprises:

[0100] acquire a shooting parameter corresponding to each of the original images, and determine a preset water depth corresponding to the preset target point based on the third parallax value and the shooting parameter;

[0101] refractive correction is performed on the preset water depth of the target point to obtain the actual water depth corresponding to the preset target point.

[0102] Specifically, when the original images of the target water area are acquired by using a UAV carrying a high-resolution camera to capture left and right view angles respectively, the preset water depth of the target point satisfies the following formula:

[0103] ;

[0104] wherein, is the focal length of the camera, is the baseline distance between the left-view-angle camera and the right-view-angle camera, is the third parallax of the target point, is the preset water depth of the target point.

[0105] Since the propagation of light from air to water will cause refraction, the influence of the refraction law on water depth detection needs to be considered. The refraction law satisfies the following formula:

[0106] ;

[0107] wherein, is the refractive index of air, is the refractive index of water, is the angle of incidence of light when entering water from air, is the angle of refraction of light when entering water from air.

[0108] The angle of incidence of light is determined according to the shooting angle of the camera and the position of the target point, the angle of refraction of light is determined by using the refraction law, and the actual water depth of the target point satisfies the following formula:

[0109] ;

[0110] wherein, is the preset water depth of the target point, is the angle of incidence of light when entering water from air, is the angle of refraction of light when entering water from air, is the actual water depth of the target point.

[0111] In this optional embodiment, the preset water depth of the target point can be calculated more accurately by combining the third parallax value and the shooting parameters. The shooting parameters (such as focal length, baseline distance, etc.) provide necessary geometric information, so that the parallax information can be accurately converted into depth information. Further refraction correction considers the propagation characteristics of light in water, and corrects the detection error caused by light refraction, so as to obtain the actual water depth closer to the true value.

[0112] Optionally, as shown in Figure 2 According to the acquired original images of the target water area at different viewing angles, an initial parallax image is generated, including:

[0113] The original images are subjected to epipolar rectification, and the matching cost of each pixel in the original images is determined;

[0114] The matching cost is aggregated in multiple directions to generate the initial parallax image, wherein the first parallax value of the pixel in the initial parallax image corresponds to the minimum value of the aggregated matching cost of the pixel.

[0115] Specifically, a plurality of original images of the target water area are acquired at different viewing angles (such as left and right viewing angles). These original images can be acquired by a drone, a satellite or other remote sensing platforms. The acquired original images are subjected to epipolar rectification to ensure that corresponding pixels in the original images at different viewing angles are located on the same epipolar line. A cost function is defined to measure the matching similarity between corresponding pixels in the original images at different viewing angles. Common cost functions include sum of absolute differences (SAD), sum of squared differences (SSD) and cross-correlation coefficient, etc. By using the cost function, the matching cost of each pixel in the original images at different viewing angles is calculated to generate a cost volume. By aggregating the matching cost in multiple directions, the cost space is optimized. Common aggregation methods include scan line method, dynamic programming method, path aggregation method in SGM algorithm, etc. These methods consider the information of neighboring pixels to reduce the influence of noise and improve the accuracy of matching. In the aggregated cost space, the parallax value corresponding to the minimum matching cost of each pixel is found as the first parallax value of the pixel. The first parallax values of all pixels are integrated into a two-dimensional image to generate an initial parallax image.

[0116] In this optional embodiment, the epipolar rectification ensures the geometric accuracy of matching, the matching cost calculation provides a measure of the similarity between pixels, and the cost aggregation further improves the reliability and accuracy of matching, so that the generated initial parallax image can better reflect the actual parallax distribution, thereby providing reliable data support for subsequent processing.

[0117] As shown in Figure 3 The water depth detection device 300 provided by the embodiment of the present application comprises:

[0118] The generating module 310 is configured to generate an initial disparity image according to the acquired original images of different perspectives of the target water area.

[0119] The first determining module 320 is configured to determine a first edge feature of any original image and a second edge feature of the initial disparity image.

[0120] The optimization module 330 is configured to acquire first disparity values of pixels within a first preset range around the second edge feature in the initial disparity image, and perform optimization processing on the first disparity values to align the first edge feature and the second edge feature, so as to obtain an intermediate disparity image.

[0121] The filtering module 340 is configured to acquire second disparity values of pixels within the first preset range around the second edge feature in the intermediate disparity image, and perform filtering processing on the second disparity values, so as to obtain a target disparity image.

[0122] The second determining module 350 is configured to acquire a third disparity value of a preset target point according to the target disparity image, and determine an actual water depth corresponding to the preset target point based on the third disparity value.

[0123] Optionally, the first determining module 320 is specifically configured to detect a line feature of the initial disparity image, and when the first disparity values of pixels within the first preset range around the line feature are all greater than a preset threshold, the line feature is taken as the second edge feature.

[0124] Optionally, the optimization module 330 is specifically configured to construct a global energy function according to the first disparity values of pixels within the first preset range around the second edge feature in the initial disparity image, and solve the global energy function to obtain the second disparity value, with minimization of the global energy function as an optimization target; wherein the global energy function includes a cost term and a regularization term, the cost term is used to measure distances between the second disparity values of the pixels and the first disparity values, and the regularization term is used to smooth the second disparity values of a first target pixel and its neighborhood pixels, and the first target pixel and its neighborhood pixels represent the pixels located on the same side of the second edge feature.

[0125] Optionally, the optimization module 330 includes a processing module, and the processing module is specifically configured to construct a graph model based on the pixels within the first preset range around the second edge feature, determine a minimum cut of the graph model, and obtain a minimum value of the global energy function and the second disparity value according to the minimum cut.

[0126] Optionally, the filtering module 340 is specifically configured to determine the weight of the second target pixel based on the gray value and distance of the second target pixel and its neighborhood pixels; wherein the second target pixel represents the pixel located within the first preset range around the second edge feature; the neighborhood pixel represents the pixel located within a second preset range around the second edge feature, and the second preset range includes the first preset range; and the second parallax value corresponding to the second target pixel is weighted and averaged according to the weight of the second target pixel to obtain the target parallax image.

[0127] Optionally, the second determining module 350 is specifically configured to obtain a shooting parameter corresponding to each of the original images, and determine a preset water depth corresponding to the preset target point based on the third parallax value and the shooting parameter; and perform refraction correction on the preset water depth of the target point to obtain the actual water depth corresponding to the preset target point.

[0128] Optionally, the generating module 310 is specifically configured to perform epipolar correction on the original image, and determine a matching cost of each pixel in the original image; and aggregate the matching cost in multiple directions to generate the initial parallax image, wherein the first parallax value of the pixel in the initial parallax image corresponds to the minimum value of the aggregated matching cost of the pixel.

[0129] As shown in Figure 4 The electronic device 400 provided by the embodiment of the present application includes a memory 410 and a processor 420; the memory 410 is used to store a computer program; and the processor 420 is used to implement the water depth detection method as described above when executing the computer program.

[0130] Alternatively, an electronic device 400 includes a memory 410 and a processor 420 coupled to the memory 410; the memory 410 is configured to store a computer program; and the processor 420 is configured to perform the following operations when executing the computer program:

[0131] generating an initial parallax image according to the obtained original images of different perspectives of a target water area;

[0132] determining a first edge feature of the original image of any perspective, and determining a second edge feature of the initial parallax image;

[0133] obtaining a first parallax value of each pixel within a first preset range around the second edge feature in the initial parallax image, and performing optimization processing on each of the first parallax values to align the first edge feature and the second edge feature, thereby obtaining an intermediate parallax image;

[0134] Obtaining second disparity values of each pixel in the first preset range around the second edge feature in the intermediate disparity image, and performing filtering processing on each second disparity value to obtain a target disparity image;

[0135] Obtaining a third disparity value of a preset target point according to the target disparity image, and determining an actual water depth corresponding to the preset target point based on the third disparity value.

[0136] The embodiment of the present application provides a computer readable storage medium, and the storage medium stores a computer program.

[0137] Alternatively, a non-volatile computer readable storage medium stores a computer program.

[0138] Generating an initial disparity image according to the obtained original images of the target water area at different perspectives;

[0139] Determining a first edge feature of the original image at any perspective, and determining a second edge feature of the initial disparity image;

[0140] Obtaining first disparity values of each pixel in a first preset range around the second edge feature in the initial disparity image, and performing optimization processing on each first disparity value to align the first edge feature and the second edge feature, so as to obtain an intermediate disparity image;

[0141] Obtaining second disparity values of each pixel in the first preset range around the second edge feature in the intermediate disparity image, and performing filtering processing on each second disparity value to obtain a target disparity image;

[0142] Obtaining a third disparity value of a preset target point according to the target disparity image, and determining an actual water depth corresponding to the preset target point based on the third disparity value.

[0143] An electronic device 400, which can be a server or a client of the present application, will now be described, which is an example of a hardware device that can be applied to aspects of the present application. The electronic device 400 is intended to represent various forms of digital electronic computer devices such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device 400 can also represent various forms of mobile devices such as personal digital assistants, cellular telephones, smart phones, wearable devices, and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present application described and / or claimed in this document.

[0144] The electronic device 400 includes a computing unit that can perform various appropriate actions and processes in accordance with a computer program stored in a read-only memory (ROM) or a computer program loaded from a storage unit into a random access memory (RAM). In the RAM, various programs and data required for device operation can also be stored. The computing unit, the ROM, and the RAM are connected to each other through a bus. An input / output (I / O) interface is also connected to the bus.

[0145] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a computer readable storage medium. When the program is executed, it can include the processes of the above-mentioned embodiments. The storage medium can be a magnetic disc, an optical disc, a read-only memory (ROM), or a random access memory (RAM). In this application, the units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, i.e., they can be located in one place or distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiments of the present application. In addition, the functional units in each embodiment of the present application can be integrated into one processing unit, or each unit can exist physically, or two or more units can be integrated into one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0146] Although the present application is disclosed as above, the protection scope of the present application is not limited to this. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the present application, and these changes and modifications will fall within the protection scope of the present application.

Claims

1. A method for detecting water depth, characterized in that, include: An initial parallax image is generated based on the original images of the target water area from different perspectives. Determine a first edge feature of the original image from any viewpoint, and determine a second edge feature of the initial parallax image; The first disparity value of each pixel within a first preset range around the second edge feature in the initial disparity image is obtained, and each first disparity value is optimized to align the first edge feature and the second edge feature to obtain an intermediate disparity image. The second disparity value of each pixel within the first preset range surrounding the second edge feature in the intermediate disparity image is obtained, and each second disparity value is filtered to obtain the target disparity image. The third disparity value of the preset target point is obtained based on the target disparity image, and the actual water depth corresponding to the preset target point is determined based on the third disparity value.

2. The water depth detection method according to claim 1, characterized in that, Determining the second edge feature of the initial parallax image includes: The line features of the initial disparity image are detected. When the first disparity value of each pixel within the first preset range around the line feature is greater than a preset threshold, the line feature is used as the second edge feature.

3. The water depth detection method according to claim 1, characterized in that, The optimization process for each of the first disparity values ​​includes: A global energy function is constructed based on the first disparity values ​​of each pixel within the first preset range surrounding the second edge feature in the initial disparity image. The global energy function is then solved with the minimization of the global energy function as the optimization objective to obtain the second disparity value. The global energy function includes a cost term and a regularization term. The cost term is used to measure the distance between the second disparity value and the first disparity value of each pixel. The regularization term is used to smooth and constrain the second disparity values ​​of the first target pixel and its neighboring pixels. The first target pixel and its neighboring pixels represent pixels located on the same side of the second edge feature.

4. The water depth detection method according to claim 3, characterized in that, The step of minimizing the global energy function as the optimization objective to obtain the second disparity value includes: A graphical model is constructed based on each pixel within the first preset range surrounding the second edge feature; Determine the minimum cut of the graphical model, and obtain the minimum value of the global energy function and the second disparity value based on the minimum cut.

5. The water depth detection method according to claim 1, characterized in that, The step of filtering each of the second disparity values ​​to obtain the target disparity image includes: The weight of the second target pixel is determined based on the grayscale value and distance of the second target pixel and its neighboring pixels; wherein the second target pixel represents the pixel located within the first preset range around the second edge feature; the neighboring pixels represent the pixels located within the second preset range around the second edge feature, and the second preset range includes the first preset range; Based on the weight of the second target pixel, the second disparity value corresponding to the second target pixel is weighted and averaged to obtain the target disparity image.

6. The water depth detection method according to claim 1, characterized in that, The step of determining the actual water depth corresponding to the preset target point based on the third disparity value includes: Obtain the shooting parameters corresponding to each of the original images, and determine the preset water depth corresponding to the preset target point based on the third parallax value and the shooting parameters; The preset water depth at the target point is refracted and corrected to obtain the actual water depth corresponding to the preset target point.

7. The water depth detection method according to claim 1, characterized in that, The process of generating an initial parallax image based on original images of the target water area from different perspectives includes: Epipolar correction is performed on the original image, and the matching cost of each pixel in the original image is determined. The matching costs are aggregated in multiple directions to generate the initial disparity image, wherein the first disparity value of the pixel in the initial disparity image corresponds to the minimum value of the matching cost after aggregation of the pixel.

8. A water depth detection device, characterized in that, include: The generation module is used to generate initial parallax images based on the original images of the target water area from different perspectives that have been acquired; The first determining module is used to determine the first edge features of the original image from any viewpoint, and to determine the second edge features of the initial parallax image; An optimization module is used to obtain the first disparity value of each pixel within a first preset range around the second edge feature in the initial disparity image, and to optimize each first disparity value to align the first edge feature and the second edge feature to obtain an intermediate disparity image. The filtering module is used to obtain the second disparity value of each pixel within the first preset range around the second edge feature in the intermediate disparity image, and to filter each second disparity value to obtain the target disparity image. The second determining module is used to obtain a third disparity value of a preset target point based on the target disparity image, and to determine the actual water depth corresponding to the preset target point based on the third disparity value.

9. An electronic device, characterized in that, Including memory and processor; The memory is used to store computer programs; The processor is configured to implement the water depth detection method as described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the water depth detection method as described in any one of claims 1 to 7.

Images