A downstream river channel image deformation monitoring method based on point cloud three-dimensional data

Abstract

The application discloses a kind of downstream river course image deformation monitoring methods based on point cloud three-dimensional data, comprising the following steps: obtaining panoramic image, the implementation of the described obtaining panoramic image includes obtaining the plane observation diagram of all to be measured area;Depth image is constructed, the implementation of the described depth image is constructed includes obtaining the point cloud data of all to be measured area, point cloud data is fitted into three-dimensional image;Image matching is carried out, the implementation of the described image matching includes matching three-dimensional image and plane observation diagram;Deformation analysis is compared according to reference point measurement value and matching result to determine deformation;It can obtain depth image by point cloud data and panoramic image respectively, improve the reliability of depth image, utilize point cloud data to enhance the spatial position data of pixel point simultaneously, obtain reference standard point by judging un-deformed area to carry out deformation analysis, improve the relevance of deformation analysis, facilitate to carry out safety analysis.

Description

Technical Field

[0001] This invention relates to the field of image monitoring technology for downstream riverbank protection at hydropower stations, and in particular to a method for monitoring deformation of downstream riverbank images based on point cloud 3D data. Background Technology

[0002] Deformation monitoring methods in engineering surveying mainly include global navigation satellite systems, intelligent total stations, ground laser scanners, photogrammetry, and ground-based synthetic aperture radar. With advancements in surveying science and technology, automated, real-time, and integrated intelligent surveying hardware has facilitated deformation monitoring. Currently, ecological environment monitoring, especially river channel safety monitoring, typically focuses on riverbank protection. Riverbank protection improves overall river stability, reduces soil erosion, and fundamentally serves the purpose of flood control and drainage. The condition of the riverbank reflects its safety level. Existing technologies for riverbank safety monitoring often involve monitoring the protection itself or using various sensors to monitor different physical quantities at the protection site. Image processing-based deformation monitoring also exists, but these methods often require comprehensive analysis of multiple data points, making the monitoring process complex and susceptible to numerous factors. Accurate deformation monitoring of riverbanks cannot be achieved solely through image processing.

[0003] For example, a "method and cloud monitoring platform for online real-time monitoring of riverbank safety based on Internet of Things and image analysis technology" disclosed in Chinese patent literature, with announcement number CN112945298A, discloses a method that includes dividing the river channel into regions and obtaining the water erosion coefficient, water flow impact coefficient and water pressure impact coefficient for each region, statistically analyzing the river water erosion coefficient, and analyzing the internal density hazard coefficient to achieve monitoring of the slope safety. However, this scheme has many monitoring items and does not have an accurate monitoring method for slope deformation through image processing. Summary of the Invention

[0004] To address the problem that existing technologies rely on too many monitoring items without using image processing to monitor slope deformation, this invention provides a method for monitoring downstream river channel image deformation based on point cloud 3D data. This method can obtain accurate 3D data for comparison to determine slope deformation through image acquisition and processing analysis.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] A method for monitoring deformation of downstream river channel images based on point cloud 3D data includes the following steps:

[0007] The process involves acquiring panoramic images, including obtaining planar observation maps of the entire area to be measured; constructing depth images, including acquiring point cloud data of the entire area to be measured and fitting the point cloud data into a 3D image; performing image matching, including matching the 3D image with the planar observation map; and performing deformation analysis, including establishing reference point measurements and comparing the reference point measurements with the matching results to determine the deformation. By detecting the area to be measured using both 2D images and 3D data, and reconstructing the 2D images using the 3D data, two different 3D images are obtained. Simultaneously, the deformation correlation of the detected area can be incorporated into the deformation judgment, enabling the monitoring and accurate analysis of river channel image deformation, avoiding the inaccuracy problems caused by relying on a single image.

[0008] Preferably, acquiring panoramic images involves defining the monitoring area using boundary coordinates, acquiring multiple images within the monitoring area to cover the entire area, projecting these images onto the same plane, and weighted feathering of the overlapping areas. Multiple images are acquired through rotational shooting until the monitoring area is fully covered, with an acquisition interval of k. These images are then merged, overlapped, and projected at intervals of k to obtain a panoramic image. This method covers the entire monitoring area, ensuring that every deformation can be detected. The merged panoramic image facilitates analysis, and the weighted feathering makes the boundaries clearer and more accurate, avoiding ambiguity and misjudgment at the boundaries during deformation analysis.

[0009] Preferably, constructing a depth image includes converting a panoramic image into a first depth image, segmenting the panoramic image, obtaining the photographic angles of the segmented images, determining depth values ​​based on the photographic angles and the segmented images, adding the depth values ​​to the panoramic image, and performing point-to-point transformation on the panoramic image to obtain the first depth image. The panoramic image is segmented according to a circular distribution and a segmentation angle i, where the segmentation angle i and the acquisition interval k have a trigonometric function relationship; disparity calculation is performed on the overlapping parts of adjacent segmented images, and depth values ​​are obtained based on the disparity function; the depth values ​​are mapped to corresponding pixels, and the pixels are transformed into multiple planes based on their corresponding depth values ​​to obtain the first depth image. This method can determine the actual position of the image based on the panoramic image, facilitating comparison with the positions obtained from point cloud data.

[0010] Preferably, constructing a depth image includes converting point cloud data into a second depth image. The second depth image involves projecting the point cloud data onto a panoramic image in the same direction to obtain point cloud depth. The ends of the point cloud depth data furthest from the panoramic image are connected to form a network. The network is then mapped to the pixels of the panoramic image to obtain the 3D coordinates of the pixels. The process involves mapping the pixels in the point cloud data to the pixels in the panoramic image, converting the coordinates in the point cloud data into a point cloud depth with the panoramic image as the reference plane, and using line segments fitted to the 3D coordinates of the point cloud data as the length of the point cloud depth. The point cloud depth network is a surface network of the actual scene. After connecting the pixels to the point cloud depth network, a 3D network of point cloud data is obtained. The 3D coordinates of the pixels are then determined based on this 3D network. This method combines the point cloud network with the panoramic image to obtain the 3D coordinates of the pixels, thereby generating a depth image based on the point cloud data, which facilitates comparative analysis and deformation.

[0011] Preferably, image matching includes matching a first depth image and a second depth image. Matching involves mapping the first and second depth images according to their size and pixels, and matching the depth value and 3D coordinates corresponding to each pixel. The boundaries of the first and second depth images are mapped, pixels are divided into regions and cluster centers are obtained, and the cluster centers of the first and second depth images are matched. Pixels associated with successfully matched cluster centers are mapped, and the depth values ​​and 3D coordinates of the successfully mapped pixels are matched. This increases the accuracy of the mapping between pixel depth values ​​and 3D coordinates. By ensuring correct pixel mapping, the correct matching of depth values ​​and 3D coordinates is ensured, thereby ensuring the accurate location of the pixels.

[0012] Preferably, deformation analysis includes acquiring reference point measurements, performing deformation analysis based on the reference point measurements and image matching results, adding stability weights to the reference point measurements, calculating vector similarity between depth values ​​and 3D coordinates in the image matching results, and comparing the vector similarity with the reference point measurements weighted with stability weights to achieve deformation analysis. By establishing reference point measurements and determining deformation standards, the dynamic determination of measurement standards facilitates the safety analysis of slope protection, ensuring that the monitored slope deformation is determined based on the overall deformation degree of the slope. This allows for the detection of not only minute displacements but also the monitoring of local deformation after overall deformation, improving the accuracy of monitoring in various situations.

[0013] Preferably, obtaining reference point measurements includes determining deformable regions based on both the panoramic and depth images. Determining deformable regions involves region traversal, which includes filtering pixels within each region to obtain abrupt pixel data. By dividing the panoramic and depth images into multiple detection regions, and filtering pixel positions within each detection region to obtain abrupt pixel data, the region is classified as a deformable region. This preliminary division of deformable regions determines the extent of deformation and identifies undeformed regions, facilitating the acquisition of reference point measurements.

[0014] Preferably, obtaining reference point measurements involves dividing the deformation area into stable regions, setting reference points within the stable regions, and sampling them. By dividing the entire monitoring area into multiple deformation and stable regions, reference point measurements can be obtained from multiple locations. Each deformation region can obtain reference point measurements from its adjacent stable regions, ensuring that deformation analysis within each deformation region is based on changes in its neighboring data, thus improving the accuracy of deformation analysis.

[0015] The present invention has the following advantages:

[0016] (1) It can obtain depth images through point cloud data and panoramic images respectively, improve the reliability of depth images, and enhance the spatial position data of pixels by using point cloud data. By judging the undeformed area, reference standard points are obtained for deformation analysis, which improves the correlation of deformation analysis and the correlation between multiple deformations, making it easier to conduct safety analysis; (2) It can cover the entire monitoring area, so that every deformation can be monitored. The merged panoramic image is easy to analyze. The boundary after weighted feathering can make the junction clearer and more accurate, avoiding ambiguity and misjudgment at the junction during deformation analysis; (3) It ensures the correct matching of depth value and three-dimensional coordinates by ensuring the correct correspondence of pixels, thereby ensuring the accurate position of pixels. Attached Figure Description

[0017] The accompanying drawings described below are merely exemplary. Those skilled in the art can derive other embodiments based on the provided drawings without any inventive effort.

[0018] Figure 1 This is a flowchart of the method steps of the present invention.

[0019] Figure 2 This is the logic flowchart of the present invention. Detailed Implementation

[0020] The following specific embodiments illustrate the implementation of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0021] like Figure 1-2 As shown, in a preferred embodiment, the present invention discloses a method for monitoring deformation of downstream river channel images based on point cloud 3D data, comprising:

[0022] The process includes: acquiring a panoramic image, which involves acquiring a planar observation map of the entire area to be measured; constructing a depth image, which involves acquiring point cloud data of the entire area to be measured and fitting the point cloud data into a three-dimensional image; performing image matching, which involves matching the three-dimensional image with the planar observation map; and performing deformation analysis, which involves establishing reference point measurements and comparing the reference point measurements with the matching results to determine the deformation.

[0023] In operation, a panoramic image of the monitoring area is acquired using a telescope camera, and point cloud data of the monitoring area in the total station coordinate system is obtained using a total station. This point cloud data includes coordinate and color data. Compared to conventional laser scanning, its advantage lies in the elimination of the need for additional targets for point cloud stitching. To combine with the image data, the point cloud data is projected onto the previously generated panoramic image, resulting in an irregular two-dimensional grid of points and the distance from the camera's projection center to each point. Further point interpolation is used to form a regular grid and create a depth image. After acquiring multiple depth images, the three-dimensional displacement can be determined by finding corresponding points in each image. Finally, the displacement of the monitoring points is obtained using a reference point as a baseline.

[0024] Acquiring panoramic images involves defining the monitoring area using boundary coordinates, acquiring multiple images within the monitoring area to cover the entire area, projecting these images onto the same plane, and then weighted feathering the overlapping areas. Multiple images are acquired through rotational shooting until the monitoring area is fully covered, with an acquisition interval of k. These images are then merged, overlapped, and projected at intervals of k to obtain a panoramic image. This method ensures complete coverage of the monitoring area, allowing every deformation to be detected. The merged panoramic image facilitates analysis, and the weighted feathering makes the boundaries clearer and more accurate, avoiding ambiguity and misjudgment at these points during deformation analysis.

[0025] In practice, the monitoring area is first defined using boundary coordinates. Then, images of the entire monitoring area are collected at calculated acquisition intervals with a certain degree of overlap. For ease of subsequent analysis, the collected images need to be stitched together into a panoramic image. The telescope camera can be considered as a fixed projection camera that only rotates without translating, thus projecting all images onto the same plane. For exposure differences between images, especially differences in overlapping areas, weighted feathering can be applied based on the distance from the pixel to the image boundary.

[0026] Constructing a depth image involves converting a panoramic image into a first depth image, segmenting the panoramic image, obtaining the photographic angles of the segmented images, determining depth values ​​based on the photographic angles and the segmented images, adding the depth values ​​to the panoramic image, and performing point-to-point transformation on the panoramic image to obtain the first depth image. The panoramic image is segmented according to a circular distribution and a segmentation angle *i*, where the segmentation angle *i* and the acquisition interval *k* have a trigonometric function relationship. Disparity calculation is performed on the overlapping portions of adjacent segmented images, and depth values ​​are obtained based on the disparity function. The depth values ​​are mapped to corresponding pixels, and the pixels are transformed into multiple planes based on their corresponding depth values ​​to obtain the first depth image.

[0027] In practice, the depth value of a two-dimensional synthetic panoramic image is calculated based on its photographic angle. This depth value is then mapped to the pixels in the two-dimensional image and converted into a three-dimensional image, thus transforming the two-dimensional image into a first three-dimensional depth image. This is used as part of the analysis process to improve the accuracy of deformation analysis.

[0028] Constructing a depth image involves converting point cloud data into a second depth image. The second depth image involves projecting the point cloud data onto a panoramic image in the same direction to obtain point cloud depth. The ends of the point cloud depth data furthest from the panoramic image are connected to form a network. This network is then mapped to the pixels of the panoramic image to obtain the 3D coordinates of the pixels. Alternatively, the pixels in the point cloud data can be mapped to the pixels in the panoramic image. The coordinates in the point cloud data are converted into point cloud depth with the panoramic image as the reference plane. The length of the point cloud depth is a line segment data fitted to the 3D coordinates of the point cloud data. The point cloud depth network is a surface network of the actual scene. After connecting the pixels to the corresponding points in the point cloud depth network, a 3D network of the point cloud data is obtained. The 3D coordinates of the pixels are then determined based on this 3D network.

[0029] In use, point cloud data and two-dimensional total station images are combined to obtain a second depth image, which represents the three-dimensional data detected by the total station. This step converts the three-dimensional points in the point cloud data into three-dimensional surfaces, which is used as part of deformation analysis to improve the accuracy of deformation analysis.

[0030] Image matching involves matching a first depth image and a second depth image. This matching includes mapping the first and second depth images based on their size and pixel counts, and matching the depth value and 3D coordinates for each pixel. The boundaries of the first and second depth images are mapped, and pixels are divided into regions with cluster centers. These cluster centers are then matched, and pixels associated with the successfully matched cluster centers are mapped. The depth values ​​and 3D coordinates of these successfully matched pixels are then matched, thus increasing the accuracy of the mapping between pixel depth values ​​and 3D coordinates. Ensuring correct pixel mapping ensures accurate matching of depth values ​​and 3D coordinates, thereby guaranteeing the precise location of each pixel.

[0031] When in use, a clustering algorithm is used to match two 3D depth maps, allowing the volume and coordinates in 3D space to be matched and connected, thus avoiding the influence of single-image calculation errors such as panoramic image calculation errors and point cloud detection errors on the final pixel position.

[0032] Deformation analysis involves acquiring reference point measurements, performing deformation analysis based on these measurements and image matching results, and applying stability weights to the reference point measurements. Then, it calculates the vector similarity between depth values ​​and 3D coordinates from the image matching results, comparing this vector similarity with the reference point measurements weighted for stability. By establishing reference point measurements and determining deformation standards, the dynamic determination of these standards facilitates slope safety analysis. This ensures that monitored slope deformation is based on the overall degree of slope deformation, enabling the detection of not only minute displacements but also monitoring of localized deformation after overall deformation.

[0033] When in use, the reference point measurement value is selected from the monitoring area. It can detect the local deformation after the overall deformation of the monitoring area, and obtain the change of local deformation relative to the overall deformation.

[0034] Obtaining reference point measurements involves identifying deformable regions based on both panoramic and depth images. This identification includes region traversal, which involves filtering pixels within each region to identify abrupt pixel changes. By dividing the panoramic and depth images into multiple detection regions, and filtering pixel positions within each region to identify abrupt pixel changes, this region is classified as a deformable area. This initial delineation of deformable regions determines the extent of deformation and identifies undeformed areas, facilitating the acquisition of reference point measurements. Obtaining reference point measurements involves defining stable regions outside the deformable regions, setting reference points within these stable regions, and sampling. By dividing the entire monitoring area into multiple deformable and stable regions, reference point measurements can be obtained from multiple locations. Each deformable region can obtain reference point measurements from its adjacent stable regions, ensuring that deformation analysis within each region is based on changes in its neighboring data, thus improving the accuracy of deformation analysis.

[0035] In practice, after depth image matching is completed, the depth image is divided into regions, with the size of each region being a matrix [j]. The 3D data within each matrix is ​​traversed to determine stable and deformable intervals. Reference points are selected within the stable intervals, and the rate of change of the connecting lines between the reference points and their surrounding points is statistically analyzed to obtain the range of change rates. The rate of change of corresponding points within the deformable intervals is calculated; points exceeding the rate of change range are considered deformed, while those within the range are considered undeformed compared to the stable interval. The selection of stable regions for comparison with deformed regions includes choosing the closest stable region to the deformed region or the stable region with the highest similarity to the deformed region in the 2D image. The size matrix of the region division can be changed to alter the accuracy of deformation monitoring. Increasing the size matrix determines whether there is deformation over a larger area, while decreasing it determines whether an area is deformed relative to other areas within a smaller region. By deploying reference points in stable regions outside the deformed regions, and measuring prisms deployed in these stable regions using a total station, high-precision measurement results can be obtained. Furthermore, by combining the acquired multi-period depth images, deformation analysis of the monitored area can be conducted. The stability of the testing site itself needs to be considered during this process.

[0036] Although the present invention has been described in detail above with general descriptions and specific embodiments, modifications or improvements can be made to it, which will be obvious to those skilled in the art. Therefore, all such modifications or improvements made without departing from the spirit of the present invention fall within the scope of protection claimed by the present invention.

Claims

1. A method for monitoring deformation of downstream river channel images based on point cloud 3D data, characterized in that, The steps include the following: The process involves: acquiring a panoramic image, which includes acquiring a planar observation map of the entire area to be measured; constructing a depth image, which includes acquiring point cloud data of the entire area to be measured, fitting the point cloud data into a 3D image, converting the panoramic image into a first depth image, and converting the point cloud data image into a second depth image; performing image matching, which includes matching the 3D image with the planar observation map, and matching the first depth image with the second depth image; and performing deformation analysis, which includes establishing reference point measurements, comparing the reference point measurements with the matching results to determine the deformation, and adding stability weights to the reference point measurements, then calculating the vector similarity between the depth values ​​and 3D coordinates in the image matching results, and comparing the vector similarity with the reference point measurements with added stability weights to achieve deformation analysis.

2. The method for monitoring deformation of downstream river channels based on point cloud 3D data according to claim 1, characterized in that, The process of acquiring panoramic images includes defining the monitoring area using boundary coordinates, acquiring multiple images within the monitoring area, and projecting the multiple images onto the same plane, and then performing weighted feathering on the overlapping areas of the multiple images.

3. A method for monitoring deformation of downstream river channel images based on point cloud 3D data according to claim 1 or 2, characterized in that, The construction of the depth image includes converting the panoramic image into a first depth image, segmenting the panoramic image, obtaining the shooting angle of the segmented image, determining the depth value based on the shooting angle and the segmented image, adding the depth value to the panoramic image, and performing point conversion on the panoramic image to obtain the first depth image.

4. The method for monitoring deformation of downstream river channel images based on point cloud 3D data according to claim 3, characterized in that, The construction of the depth image includes converting the point cloud data image into a second depth image. The second depth image includes projecting the point cloud data onto the panoramic image in the same direction to obtain the point cloud depth, connecting the ends of the point cloud depth away from the panoramic image to form a network, and mapping the network to the pixels of the panoramic image to obtain the three-dimensional coordinates of the pixels.

5. The method for monitoring deformation of downstream river channels based on point cloud 3D data according to claim 1, characterized in that, The matching process includes mapping the first depth image to the second depth image based on size and pixels, and matching the depth value and three-dimensional coordinates corresponding to each pixel.

6. The method for monitoring deformation of downstream river channel images based on point cloud 3D data according to claim 5, characterized in that, Obtaining reference point measurements includes determining the deformable region based on the panoramic image and the depth image, respectively. Determining the deformable region includes performing region traversal, which involves filtering pixels within each region to obtain the jump pixel data within that region.

7. A method for monitoring deformation of downstream river channel images based on point cloud 3D data according to claim 5 or 6, characterized in that, Obtaining reference point measurements involves dividing the area outside the deformation zone into a stable region, setting reference points within the stable region, and sampling them.

Images