Pump gate deformation monitoring method based on computer vision

Abstract

The invention provides an embankment-penetrating pump gate deformation monitoring method based on computer vision, and relates to the field of hydraulic engineering safety monitoring, and the method comprises the steps: installing a high-definition camera at a preset part of a target pump gate, so as to collect a structure surface image in real time; performing graying processing and image enhancement on the structure surface image to obtain an enhanced image; denoising the enhanced image based on a mean filtering method to obtain a denoised image; extracting structural features in the de-noised image by adopting a method of combining edge detection and corner detection; and inputting the structural features into a trained pump gate deformation monitoring model to obtain a monitoring result. According to the pump gate deformation monitoring method based on computer vision, efficient, accurate and real-time monitoring is achieved through advanced technical means, the technical progress in the pump gate monitoring field is promoted, and meanwhile important support is provided for related structure safety management.

Description

Technical Field

[0001] The invention relates to the field of water conservancy project safety monitoring, and in particular to a pump gate deformation monitoring method based on computer vision. Background Art

[0002] There are many pump gates built on the riverbanks of the main stream and tributaries in the middle reaches of the Yangtze River. Because most riverbanks are built on a binary stratum structure, when the clay cover is thin and the river water level is high, seepage deformation is prone to occur near the foot of the riverbank's backwater slope. When building pump gates, the excavation of inlets and outlets not only weakens the embankment but also disturbs the weakly permeable covering layer, making pump gates more susceptible to seepage damage than riverbanks during flood season.

[0003] The invention patent with application number 202410043759.1 discloses a real-time risk rate quantification and early warning method for deformation of a sluice building, including S1: building a monitoring system; S2: all monitoring equipment monitors the deformation of the sluice in real time, and regularly transmits the collected data to the data processing module; S3: the data processing module establishes a sluice building deformation monitoring model based on the received original data of sluice deformation, and quantifies the risk rate of a single measuring point through the model; S4: the intelligent decision-making and early warning module analyzes the risk rate of each measuring point in real time, and issues an early warning when it exceeds the preset value; wherein, step S2 specifically includes all monitoring equipment in the data acquisition module collecting the deformation information of each measuring point of the sluice building in real time, the temperature sensor collecting the structural temperature information of the sluice building in real time, and the water level sensor collecting the water level information of the upstream and downstream waters of the sluice in real time, and regularly transmitting the collected original data to the data processing module, and storing it in the data storage module. It can be seen from the above records that the patent cannot capture visual features, and information such as cracks, displacements, and deformations of buildings can be more intuitively presented through images, and real-time image collection can record dynamic changes. These visual features cannot be obtained by traditional sensors, resulting in reduced sensitivity and accuracy of their detection. Summary of the invention

[0004] In order to overcome the deficiencies of the prior art, the object of the present invention is to provide a pump gate deformation monitoring method based on computer vision, which can realize efficient, accurate and real-time pump gate deformation monitoring.

[0005] To achieve the above object, the present invention provides the following solutions:

[0006] A method for monitoring pump gate deformation based on computer vision, comprising:

[0007] Install high-definition cameras at preset locations of target pump gates to collect real-time images of the structure surface;

[0008] grayscale processing and image enhancement are performed on the surface image of the structure to obtain an enhanced image;

[0009] De-noising the enhanced image based on a mean filtering method to obtain a de-noised image;

[0010] Extracting structural features from the denoised image using a method combining edge detection and corner detection;

[0011] The structural features are input into the trained pump-gate deformation monitoring model to obtain monitoring results.

[0012] Preferably, grayscale processing and image enhancement are performed on the structure surface image to obtain an enhanced image, including:

[0013] grayscale the structure surface image to obtain a grayscale image;

[0014] Based on fuzzy theory, fuzzy enhancement functions of different pixels of the grayscale image are constructed; the formula of the fuzzy enhancement function is: Among them, μ dark represents the first fuzzy enhancement function, μ birg represents the second fuzzy enhancement function, μ gray represents the third fuzzy enhancement function, x ij Represents the gray value at the position (i, j);

[0015] Construct a saliency enhancement model based on the fuzzy enhancement functions of different pixels;

[0016] The grayscale image is processed according to the saliency enhancement model to obtain the enhanced image.

[0017] Preferably, the formula of the significance enhancement model is:

[0018]

[0019] Among them, x i ' j Indicates that when the input pixel is x ij When the pixel value after saliency enhancement is performed, ν d represents the first weight, ν b represents the second weight, ν g Represents the third weight.

[0020] Preferably, denoising the enhanced image based on a mean filtering method to obtain a denoised image comprises:

[0021] Taking each pixel in the enhanced image as the center, taking a neighborhood;

[0022] Calculate the pixel feature value of each pixel in the corresponding neighborhood;

[0023] Constructing a smoothing denoising model according to the pixel feature values;

[0024] The enhanced image is denoised using the smoothing denoising model to obtain the denoised image.

[0025] Preferably, the formula of the smoothing denoising model is:

[0026]

[0027] in, Represents the gradient value of pixel point x in the corresponding neighborhood, is the pixel mean of the neighborhood centered on pixel point x, u(x) represents the pixel value of pixel point x, and u median (x) represents the pixel median of the neighborhood centered on pixel x.

[0028] Preferably, denoising the enhanced image using the smoothing denoising model to obtain the denoised image comprises:

[0029] Calculate the smoothing value of each pixel using the smoothing denoising model;

[0030] All pixel points whose smoothing values ​​are greater than the denoising threshold are denoised using a mean filtering method to obtain the denoised image.

[0031] Preferably, a method combining edge detection and corner point detection is used to extract structural features in the denoised image, including:

[0032] The denoised image is subjected to edge detection using the Canny edge detection algorithm to obtain an edge image and the edge strength of the edge image; the edge strength is calculated using the following formula: Among them, G is x is the horizontal gradient of the image, G y is the vertical gradient of the image;

[0033] The Harris corner detection algorithm is used to perform corner detection on the edge image to obtain a corner image and a corner response value; the corner response value is: R = det(M) - k(trace(M)) 2 ; Wherein M is the autocorrelation matrix of the image, k is an empirical constant, and R is the corner point response value;

[0034] Based on the weighted formula of edge amplitude, the structural feature is determined according to the edge strength and the corner point response value; the weighted formula is: F(x, y) = w1·G+w2·R; wherein w1 is the weight of edge detection, w2 is the weight of corner point detection, and F(x, y) is the structural feature.

[0035] Preferably, the pump gate deformation monitoring model includes:

[0036] Obtain a preset structural feature data set;

[0037] Construct an initial convolutional neural network;

[0038] Training the initial convolutional neural network according to the structural feature data set to obtain a trained classifier network;

[0039] The LSTM neural network and the fully connected layer are sequentially connected after the classification network to obtain the pump gate deformation monitoring model.

[0040] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0041] The present invention provides a method for monitoring the deformation of a pump gate based on computer vision, comprising: installing a high-definition camera at a preset position of a target pump gate to collect a structural surface image in real time; graying and enhancing the structural surface image to obtain an enhanced image; denoising the enhanced image based on a mean filtering method to obtain a denoised image; extracting structural features from the denoised image using a method combining edge detection with corner point detection; inputting the structural features into a trained pump gate deformation monitoring model to obtain monitoring results. The method for monitoring the deformation of a pump gate based on computer vision of the present invention achieves efficient, accurate and real-time monitoring through advanced technical means, which not only promotes technological progress in the field of pump gate monitoring, but also provides important support for related structural safety management. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0043] Figure 1 A flow chart of a method provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0046] Figure 1 A flow chart of a method provided by an embodiment of the present invention, such as Figure 1 As shown, the present invention provides a pump gate deformation monitoring method based on computer vision, comprising:

[0047] Step 100: Install a high-definition camera at a preset position of the target pump gate to collect a surface image of the structure in real time;

[0048] Step 200: grayscale processing and image enhancement are performed on the structure surface image to obtain an enhanced image;

[0049] Step 300: De-noising the enhanced image based on a mean filtering method to obtain a de-noised image;

[0050] Step 400: extracting structural features in the denoised image by combining edge detection with corner point detection;

[0051] Step 500: input the structural features into the trained pump gate deformation monitoring model to obtain monitoring results.

[0052] Specifically, the preset parts of this embodiment include:

[0053] Critical load-bearing parts: The camera is installed on the main load-bearing structure of the pump gate, such as the arch or side wall that crosses the water flow. These locations are critical areas prone to deformation or cracks.

[0054] Areas prone to deformation, especially those locations where deformation has been found to occur frequently or where potential risks are high during historical monitoring, such as welded joints, connection parts, supporting structures, etc.

[0055] Areas affected by water level changes: In this embodiment, the impact of water level changes on the pump gate structure is taken into consideration, and cameras are also installed in places where the upstream and downstream water levels change greatly, so as to obtain image data related to the water level.

[0056] Furthermore, the camera of this embodiment is also set in a well-lit position, and a place with good lighting conditions is selected to ensure that clear images can be effectively captured under various weather and lighting conditions. And avoid obstacles to ensure that the camera's field of view is not blocked by other equipment or structures, and an unobstructed panoramic image can be obtained. The camera of this embodiment is set at an appropriate height and angle. The camera should be installed at a suitable height and angle to obtain the best shooting field of view and ensure that the key parts of the pump gate are within the camera range. By installing high-definition cameras in these preset positions, it can be ensured that the monitoring system can effectively capture the real-time deformation information of the pump gate, thereby providing a reliable image basis for subsequent data processing and risk assessment.

[0057] Preferably, grayscale processing and image enhancement are performed on the structure surface image to obtain an enhanced image, including:

[0058] grayscale the structure surface image to obtain a grayscale image;

[0059] Based on fuzzy theory, fuzzy enhancement functions of different pixels of the grayscale image are constructed; the formula of the fuzzy enhancement function is: Among them, μ dark represents the first fuzzy enhancement function, μ birg represents the second fuzzy enhancement function, μ gray represents the third fuzzy enhancement function, x ij Represents the gray value at the position (i, j);

[0060] Construct a saliency enhancement model based on the fuzzy enhancement functions of different pixels;

[0061] The grayscale image is processed according to the saliency enhancement model to obtain the enhanced image.

[0062] This embodiment takes into account that the image may be blurred due to the background, weather and light of the pump gate. Based on this, the present invention uses fuzzy theory to enhance the original image. Based on fuzzy theory, the present invention increases the contrast between the monitoring surface area and the background area, making the monitoring surface area brighter and the background area darker. Therefore, the present invention stipulates the following fuzzy rules:

[0063] (1) If the current pixel is dark, make this pixel darker.

[0064] (2) If the current pixel is gray, leave it unchanged.

[0065] (3) If the current pixel is bright, make this pixel brighter.

[0066] Preferably, the formula of the significance enhancement model is:

[0067]

[0068] Among them, x i ' j Indicates that when the input pixel is x ij When the pixel value after saliency enhancement is performed, ν d represents the first weight, ν b represents the second weight, ν g Represents the third weight.

[0069] Preferably, denoising the enhanced image based on a mean filtering method to obtain a denoised image comprises:

[0070] Taking each pixel in the enhanced image as the center, taking a neighborhood;

[0071] Calculate the pixel feature value of each pixel in the corresponding neighborhood;

[0072] Constructing a smoothing denoising model according to the pixel feature values;

[0073] The enhanced image is denoised using the smoothing denoising model to obtain the denoised image.

[0074] Preferably, the formula of the smoothing denoising model is:

[0075]

[0076] in, Represents the gradient value of pixel point x in the corresponding neighborhood, is the pixel mean of the neighborhood centered on pixel point x, u(x) represents the pixel value of pixel point x, and u median (x) represents the pixel median of the neighborhood centered on pixel x.

[0077] Preferably, denoising the enhanced image using the smoothing denoising model to obtain the denoised image comprises:

[0078] Calculate the smoothing value of each pixel using the smoothing denoising model;

[0079] All pixel points whose smoothing values ​​are greater than the denoising threshold are denoised using a mean filtering method to obtain the denoised image.

[0080] In practical applications, the present invention can set a denoising threshold according to the smoothing value of the pixel point, and at the same time, perform mean filtering on all the pixels in the noise interval to obtain the median pixel point, and then use the median pixel point to replace the values ​​of all the pixels in the neighborhood, so as to obtain a denoised image. The original filtering algorithm, such as the mean filtering algorithm, performs mean processing on the pixels in each neighborhood on the image, so the processed image will become blurred. The present invention can find the noise points on the image by using the smoothing denoising model, and then perform mean filtering on the corresponding noise points, which can smooth out the noise points in the image while maximally maintaining the original information of the target image.

[0081] Preferably, a method combining edge detection and corner point detection is used to extract structural features in the denoised image, including:

[0082] The denoised image is subjected to edge detection using the Canny edge detection algorithm to obtain an edge image and the edge strength of the edge image; the edge strength is calculated using the following formula: Among them, G is x is the horizontal gradient of the image, G y is the vertical gradient of the image;

[0083] The Harris corner detection algorithm is used to perform corner detection on the edge image to obtain a corner image and a corner response value; the corner response value is: R = det(M) - k(trace(M)) 2 ; Wherein M is the autocorrelation matrix of the image, k is an empirical constant, and R is the corner point response value;

[0084] Based on the weighted formula of edge amplitude, the structural feature is determined according to the edge strength and the corner point response value; the weighted formula is: F(x, y) = w1·G+w2·R; wherein w1 is the weight of edge detection, w2 is the weight of corner point detection, and F(x, y) is the structural feature.

[0085] Optionally, the present embodiment constructs a labeled data set containing different types of edge and corner information and corresponding target monitoring outputs (e.g., degree of deformation). Then, a learning model (such as a linear model, a neural network, etc.) is constructed based on the labeled data set to predict the monitoring results. During the model training process, the optimal values ​​of w1 and w2 are learned at the same time. The present embodiment uses an optimization algorithm (such as gradient descent, genetic algorithm, etc.) to find the best combination of weights so that the fusion feature has the best performance in a specific task (such as deformation detection). And the present embodiment further defines an objective function, such as minimizing the prediction error, and adjusts w1 and w2 by optimizing the objective function. During the model training process, cross-validation is used to evaluate the impact of different weight combinations on model performance, thereby selecting the optimal weight value.

[0086] Furthermore, the edge detection algorithm of this embodiment may also include Sobel, Canny and Prewitt. This embodiment ensures the clarity of the edge by adjusting the parameters in the algorithm (such as the low threshold and high threshold in Canny detection). Then this embodiment performs edge detection on the denoised image to highlight the obvious boundaries of the structure.

[0087] Optionally, in addition to the Harris corner detection algorithm in this embodiment, the Shi-Tomasi corner detection algorithm can also be used. In the image after edge detection, corners are further identified. Corner points usually appear at the intersection of significant structures in the image.

[0088] Furthermore, this embodiment also eliminates mismatches and combines edge and corner features through the RANSAC (Random Sampling Consistency) algorithm to ensure that only reliable feature points that are consistent in space are selected. Finally, the features extracted from edge and corner detection are integrated to form a comprehensive set of structural features for further analysis.

[0089] Furthermore, this embodiment verifies the accuracy of the detection by visualizing the superimposed structural feature points on the original image. The extracted features are further optimized to remove redundant or insignificant feature points and improve the expressiveness of the feature set. Finally, this embodiment inputs the combined structural features into the trained pump gate deformation monitoring model to perform deformation analysis and risk assessment. The current structural state is determined through the model output results, and necessary early warning operations are performed.

[0090] Preferably, the pump gate deformation monitoring model includes:

[0091] Obtain a preset structural feature data set;

[0092] Construct an initial convolutional neural network;

[0093] Training the initial convolutional neural network according to the structural feature data set to obtain a trained classifier network;

[0094] The LSTM neural network and the fully connected layer are sequentially connected after the classification network to obtain the pump gate deformation monitoring model.

[0095] Furthermore, the monitoring results in this embodiment include:

[0096] (1) Displacement: The displacement of the pump gate structure, usually in millimeters or centimeters, indicating the degree of deformation at different measurement time points.

[0097] (2) Tilt: The tilt angle of a structure relative to the vertical or horizontal state, which represents the stability of the structure.

[0098] (3) Local deformation: If deformation is detected in a specific area, the location and extent of the local deformation will be marked in the monitoring results.

[0099] (4) Overall deformation: used to describe the deformation state of the overall structure of the pump gate, including overall sinking, warping or twisting.

[0100] (5) Edge feature changes: Monitor the changes in edge features extracted on the surface of the structure, such as edge length, edge strength and other parameters.

[0101] (6) Corner point feature changes: Monitor the extracted corner point features and describe the changes in the number and distribution of corner points.

[0102] (7) Structural safety level: Based on the deformation amount and deformation type, the structural safety level is given, such as normal, minor warning, severe warning, etc.

[0103] (8) Stability analysis: Evaluate the current stability of the structure and determine whether there is a risk of collapse or instability.

[0104] (9) Historical deformation trend: Analyze the deformation results at different times and generate trend graphs to help identify long-term deformation trends.

[0105] The beneficial effects of the present invention are as follows:

[0106] (1) The present invention realizes real-time acquisition of the structural surface by installing a high-definition camera at a preset position of the target pump gate, which can quickly obtain the latest monitoring data and identify potential deformation problems in a timely manner.

[0107] (2) The present invention uses grayscale processing and image enhancement technology to improve image quality, and effectively removes noise through mean filtering, so that subsequent edge detection and corner point detection can more accurately extract structural features, thereby improving monitoring accuracy.

[0108] (3) The present invention uses mean filtering to smooth the image and reduce noise interference, thereby reducing false detection and missed detection and ensuring the reliability of the monitoring results.

[0109] (4) The present invention combines edge detection and corner point detection, utilizing the advantages of both to comprehensively extract more comprehensive and accurate structural features, which can effectively characterize the shape and deformation of the pump gate.

[0110] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0111] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A method for monitoring pump and lock deformation based on computer vision, characterized in that: include: Install high-definition cameras at preset locations of target pump gates to collect real-time images of the structure surface; grayscale processing and image enhancement are performed on the surface image of the structure to obtain an enhanced image; De-noising the enhanced image based on a mean filtering method to obtain a de-noised image; Extracting structural features from the denoised image using a method combining edge detection and corner detection; The structural features are input into the trained pump-gate deformation monitoring model to obtain monitoring results.

2. The method for monitoring pump and lock deformation based on computer vision according to claim 1, characterized in that: Grayscale processing and image enhancement are performed on the surface image of the structure to obtain an enhanced image, including: grayscale the structure surface image to obtain a grayscale image; Based on fuzzy theory, fuzzy enhancement functions of different pixels of the grayscale image are constructed; the formula of the fuzzy enhancement function is: Among them, μ dark represents the first fuzzy enhancement function, μ birg represents the second fuzzy enhancement function, μ gray represents the third fuzzy enhancement function, x ij Represents the gray value at the position (i, j); Construct a saliency enhancement model based on the fuzzy enhancement functions of different pixels; The grayscale image is processed according to the saliency enhancement model to obtain the enhanced image.

3. The method for monitoring pump and lock deformation based on computer vision according to claim 2, characterized in that: The formula of the significance enhancement model is: Among them, x i ' j It means that when the input pixel is x ij When the pixel value after saliency enhancement is performed, ν d represents the first weight, ν b represents the second weight, ν g Represents the third weight.

4. The method for monitoring pump and lock deformation based on computer vision according to claim 1, characterized in that: Denoising the enhanced image based on a mean filtering method to obtain a denoised image, including: Taking each pixel in the enhanced image as the center, taking a neighborhood; Calculate the pixel feature value of each pixel in the corresponding neighborhood; Constructing a smoothing denoising model according to the pixel feature values; The enhanced image is denoised using the smoothing denoising model to obtain the denoised image.

5. The method for monitoring pump and lock deformation based on computer vision according to claim 4 is characterized in that: The formula of the smoothing denoising model is: in, Represents the gradient value of pixel point x in the corresponding neighborhood, is the pixel mean of the neighborhood centered on pixel point x, u(x) represents the pixel value of pixel point x, and u median (x) represents the pixel median of the neighborhood centered on pixel x.

6. The method for monitoring pump and lock deformation based on computer vision according to claim 4, characterized in that: Denoising the enhanced image using the smoothing denoising model to obtain the denoised image, comprising: Calculate the smoothing value of each pixel using the smoothing denoising model; All pixel points whose smoothing values ​​are greater than the denoising threshold are denoised using a mean filtering method to obtain the denoised image.

7. The method for monitoring pump and lock deformation based on computer vision according to claim 1, characterized in that: The structural features in the denoised image are extracted by combining edge detection with corner detection, including: The denoised image is subjected to edge detection using the Canny edge detection algorithm to obtain an edge image and the edge strength of the edge image; the edge strength is calculated using the following formula: Among them, G is x is the horizontal gradient of the image, G y is the vertical gradient of the image; The Harris corner detection algorithm is used to perform corner detection on the edge image to obtain a corner image and a corner response value; the corner response value is: R = det(M) - k(trace(M)) 2 ; Wherein M is the autocorrelation matrix of the image, k is an empirical constant, and R is the corner point response value; Based on the weighted formula of edge amplitude, the structural feature is determined according to the edge strength and the corner point response value; the weighted formula is: F(x, y) = w1·G+w2·R; wherein w1 is the weight of edge detection, w2 is the weight of corner point detection, and F(x, y) is the structural feature.

8. The method for monitoring pump and lock deformation based on computer vision according to claim 1, characterized in that: The pump gate deformation monitoring model includes: Obtain a preset structural feature data set; Construct an initial convolutional neural network; Training the initial convolutional neural network according to the structural feature data set to obtain a trained classifier network; The LSTM neural network and the fully connected layer are sequentially connected after the classification network to obtain the pump gate deformation monitoring model.

Images