Dam surface crack detection and management system and method and electronic equipment

Abstract

The invention provides a dam surface crack detection and management system and method and electronic equipment. The system comprises a data management module, an image display module, an image processing module and a crack analysis and auxiliary decision-making module. The data management module processes original image data, shot by the unmanned aerial vehicle camera, of the surface of the dam, and the obtained independent image data are spliced. And the image display module displays the spliced surface image of the arch dam body. The image processing module carries out image processing on the image data obtained by the camera, so that the image quality is improved, and subsequent automatic image crack extraction is facilitated. And the crack analysis and auxiliary decision-making module automatically extracts and analyzes crack information on the surface of the dam from the dam image according to a crack detection algorithm, and carries out risk assessment based on the extracted crack information to assist decision-making. The dam surface crack detection level can be further improved, dam surface crack information is analyzed in time, and safe operation of a hydropower station is maintained.

Description

Technical Field

[0001] The present application relates to the technical field of dam surface crack detection, and in particular to a dam surface crack detection and management system, method and electronic equipment. Background Art

[0002] Dams are important water conservancy projects, and their safety is paramount. Surface cracks on dams are a major safety hazard, and timely and accurate detection is crucial for ensuring their safe operation. However, existing technologies for detecting surface cracks in dams suffer from several shortcomings, including inefficient image data processing, significant human influence on detection accuracy, a lack of effective decision support, and significant subjectivity in analysis due to the varying personnel involved. Summary of the Invention

[0003] The embodiments of the present application provide a dam surface crack detection and management system, method, and electronic equipment.

[0004] According to a first aspect of an embodiment of the present application, a dam surface crack detection and management system is provided, comprising:

[0005] a data management module for acquiring a plurality of original image data of the dam surface taken by a drone camera, performing coordinate transformation on the plurality of original image data based on a transformation relationship between an image pixel coordinate system and an object coordinate system of a target, and performing image stitching on the plurality of original image data after the coordinate transformation to obtain a stitched image of each section of the dam surface;

[0006] an image display module, configured to display the stitched image based on a selected image browsing mode, wherein the image browsing mode includes a two-dimensional image browsing mode and a three-dimensional model browsing mode, wherein when the selected image browsing mode is the three-dimensional model browsing mode, a three-dimensional dam model corresponding to the stitched image is displayed, wherein the three-dimensional dam model is established based on the stitched image using an aerial triangulation method, and the three-dimensional dam model is scalable;

[0007] An image processing module, configured to perform image processing on the stitched image based on at least one processing method; wherein the at least one processing method includes an image dodging processing method, an image enhancement processing method, and a noise reduction processing method;

[0008] The crack analysis and decision-making support module is used to input the spliced ​​image processed by the image processing module into a pre-trained neural network model to identify cracks, obtain crack geometry information output by the neural network model, analyze defect information on the surface of each section of the dam based on the crack geometry information and the three-dimensional dam model, and conduct risk assessment based on the defect information to assist decision-making.

[0009] According to a second aspect of an embodiment of the present application, a method for detecting cracks on a dam surface is provided. The method is implemented based on the dam surface crack detection and management system described in the first aspect, and the method includes the following steps:

[0010] Acquire multiple original image data of the dam surface taken by the drone camera;

[0011] performing coordinate transformation on the plurality of original image data based on a transformation relationship between an image pixel coordinate system and an object coordinate system of a target, and performing image stitching on the plurality of original image data after the coordinate transformation to obtain a stitched image of the surface of each section of the dam;

[0012] displaying the stitched image based on a selected image browsing mode, the image browsing mode including a two-dimensional image browsing mode and a three-dimensional model browsing mode, wherein when the selected image browsing mode is the three-dimensional model browsing mode, a three-dimensional dam model corresponding to the stitched image is displayed, the three-dimensional dam model being established based on the stitched image through an aerial triangulation method, and the three-dimensional dam model being scalable;

[0013] Performing image processing on the stitched image based on at least one processing method; wherein the at least one processing method includes an image dodging processing method, an image enhancement processing method, and a noise reduction processing method;

[0014] Inputting the spliced ​​image processed by the image processing module into a pre-trained neural network model to identify cracks, and obtaining crack geometry information output by the neural network model;

[0015] The defect information on the surface of each section of the dam is analyzed based on the crack geometry information and the three-dimensional dam model, and risk assessment is performed based on the defect information to assist decision-making.

[0016] According to a third aspect of the embodiments of the present application, there is provided an electronic device, including:

[0017] at least one processor; and

[0018] a memory communicatively connected to the at least one processor; wherein,

[0019] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method described in the second aspect.

[0020] According to a fourth aspect of an embodiment of the present application, a storage medium is provided, wherein the storage medium stores instructions. When the instructions are executed on an electronic device, the electronic device executes the method described in the first aspect above.

[0021] According to a fifth aspect of an embodiment of the present application, a program product is provided, comprising at least one of a program and an instruction, wherein when the at least one of the program and the instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0022] According to the technical solution of this application, raw image data of the dam surface is automatically collected by drones. The raw image data captured by the drones is spliced ​​together by a data management module to obtain spliced ​​images of the surface of each section of the dam. The spliced ​​images are processed by an image processing module to improve the image quality. The spliced ​​images are input into a pre-trained neural network model through a crack analysis and decision support module to identify cracks and obtain crack geometry information. Based on the crack geometry information and a three-dimensional dam model, defect information of each section of the dam surface is analyzed. Risk assessment is performed based on the defect information to assist in decision-making, thereby achieving automatic processing and analysis of crack information and providing decision support. In other words, this application integrates data fusion, display, processing, and analysis functions into a unified system to achieve automatic detection of dam surface crack information. This embodies the functional implementation and collaborative work of each module of the system, effectively solving the technical problems of dam surface crack detection and management, further improving the level of dam surface crack detection, timely analyzing dam surface crack information, and maintaining the safe operation of hydropower stations.

[0023] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0025] Figure 1 A block diagram of the dam surface crack detection and management system provided in an embodiment of the present application;

[0026] Figure 2 A flow chart of a dam surface crack detection method provided in an embodiment of the present application;

[0027] Figure 3 is a block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0028] The following describes in detail embodiments of the present application, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0029] The following describes the dam surface crack detection and management system, method, and electronic device according to embodiments of the present application with reference to the accompanying drawings.

[0030] Figure 1 This is a block diagram of the dam surface crack detection and management system provided by the embodiment of this application. Figure 1 As shown, the dam surface crack detection and management system may include: a data management module 100 , an image display module 200 , an image processing module 300 , and a crack analysis and decision support module 400 .

[0031] Among them, the data management module 100 is used to obtain multiple original image data of the dam surface taken by the drone camera, and perform coordinate transformation on the multiple original image data based on the transformation relationship between the image pixel coordinate system and the object coordinate system of the target, and perform image stitching on the multiple original image data after coordinate transformation to obtain stitched images of the surface of each section of the dam.

[0032] In some embodiments, as Figure 1 As shown, the data management module 100 may include an image data import submodule 101, an image data stitching submodule 102, and an image data storage submodule 103. Among them, the image data import submodule 101 is used to monitor the specified drone image storage path in real time, and when it is detected that there is new original image data in the drone image storage path, it will start the import program to import the new original image data and update the metadata information of the image file in the database. Exemplarily, after the drone automatically completes the task of shooting the dam surface image, the image file can be automatically transferred to the drone image storage path specified by the system through the 4G (or 5G) network. The image data import submodule 101 monitors the storage path in real time, and once a new image file is found, it immediately starts the import program and updates the metadata information of the image file in the database. The metadata information may include, for example, the file name, shooting time, shooting coordinates, etc.

[0033] The image data stitching submodule 102 is used to read the imported raw image data, convert the different pixel coordinate systems of each image data into a unified real-world coordinate system through the conversion relationship between the image pixel coordinate system and the target object coordinate system, and complete the image stitching to obtain the stitched image of each section of the dam surface. For example, the conversion relationship can be expressed as follows:

[0034]

[0035] Among them, (x j ,y j ) is the pixel coordinate system of the jth image ij, (X j ,Y j ,Z j ) is the object coordinate system, αij is the conversion parameter. Through this conversion relationship, the different pixel coordinate systems of each image taken by the drone camera can be converted into a unified real-world coordinate system and the images can be stitched together to obtain complete image data of the entire dam surface for each section.

[0036] The image data storage submodule 103 is used to store the stitched image in a database, create an index in the database, and record image-related parameters. The index and image-related parameters are used for rapid access and retrieval. These image-related parameters may include, but are not limited to, image stitching parameters and shooting range. In other words, the image data storage submodule can store the complete stitched image data in an image database, eliminating the need to re-stitch the image data each time the system is started, improving data processing efficiency and facilitating subsequent operations.

[0037] The image display module 200 is used to display the spliced ​​image based on the selected image browsing mode. The image browsing mode can include a 2D image browsing mode and a 3D model browsing mode. When the selected image browsing mode is the 3D model browsing mode, the 3D dam model corresponding to the spliced ​​image is displayed. This 3D dam model is constructed based on the spliced ​​image using aerial triangulation methods and is scalable. In other words, the main function of the image display module 200 is to display the spliced ​​surface image of the arch dam (i.e., the spliced ​​image mentioned above) according to different browsing modes. Image browsing modes can be divided into 2D image browsing mode and 3D model browsing mode. For example, if the user selects the 2D image browsing mode through the system interface, the image display module 200 reads the spliced ​​image from the database and displays the image in the interface window. The image can be zoomed in and out using the mouse wheel and panned by dragging the mouse, making it easy to view cracks in different areas of the dam surface. The user selects the 3D model browsing mode through the system interface. The image display module 200 first performs aerotriangulation on the stitched images. Software then automatically calculates the image's exterior orientation elements and encrypted point coordinates to construct a sparse point cloud model of the dam. Through point cloud encryption and texture mapping, a 3D dam model with realistic textures is generated. Within the system's 3D browsing interface, the model can be rotated, scaled, translated, and navigated freely using the keyboard and mouse. Built-in measurement tools can also be used to measure geometric information such as the length and width of cracks within the 3D dam model. Thus, the image display module provides two different browsing modes, allowing for convenient observation of cracks on the dam surface from different angles.

[0038] The image processing module 300 is used to perform image processing on the stitched image based on at least one processing method; wherein the at least one processing method includes an image dodging processing method, an image enhancement processing method, and a noise reduction processing method.

[0039] In some embodiments, the image processing module 300 may include at least one of the following: an image uniformity processing submodule 301, an image enhancement processing submodule 302, and a noise reduction processing submodule 303. Optionally, in this embodiment, the image processing module 300 may include an image uniformity processing submodule 301, an image enhancement processing submodule 302, and a noise reduction processing submodule 303. The image uniformity processing submodule 301 can be used to filter the stitched image using Gaussian filtering, separate the background illumination component, and adjust the brightness of the stitched image using a uniformity correction formula based on the background illumination component. Exemplarily, the formula for determining the background illumination component can be expressed as follows: B(x, y) = G σ (x,y)*I(x,y), I(x,y) is the brightness value of the original image at the pixel point (x,y). It is the input image without any processing and is directly taken by the camera carried by the drone. It serves as the input data of the algorithm and reflects the actual light distribution on the dam surface. G σ (x, y) is the Gaussian kernel function, σ is the standard deviation of the Gaussian kernel, which controls the smoothing scale of the filter, (x, y) is the coordinate inside the kernel, and the size of the kernel is usually (6σ+1)×(6σ+1) to ensure that the main energy is covered. Through the convolution operation B(x, y) = G σ (x,y)*I(x,y), the low-pass characteristic of the Gaussian kernel can filter out high-frequency details (such as cracks and noise) in the image and retain low-frequency background illumination information. The uniform light correction formula is ε is a minimum constant, and μ is a global brightness adjustment factor, which is used to adjust the global brightness level of the image after uniform lighting.

[0040] Image enhancement processing submodule 302 can be used to process the stitched image using a low-light enhancement method based on Retinex theory. This low-light enhancement method based on Retinex theory is used to highlight the characteristic information of the cracks in the stitched image and weaken the background information. In other words, the main purpose of image enhancement processing submodule 302 is to highlight the characteristic information of the cracks and weaken the information of the background. This further highlights the crack details and facilitates the subsequent extraction and identification of concrete crack features. Crack information enhancement mainly uses a low-light enhancement method based on Retinex theory, and its model is as follows: Among them, R(x,y) is the image after low-light enhancement, I(x,y) is the original low-light spliced ​​image, and I low (x,y) is the low-frequency component, and ε is the smoothing factor.

[0041] The noise reduction processing submodule 303 can be used to perform noise reduction processing on low-stitched images using a filtering algorithm. In other words, the purpose of the noise reduction processing submodule 303 is to remove noise that damages image quality due to problems such as light and photosensitive elements. Common methods include mean filtering, median filtering, Wiener filtering, and wavelet noise reduction. The present invention uses median filtering to reduce noise to improve image quality and image availability. Therefore, the image processing module solves the problems of uneven image brightness, highlighting crack features, and removing noise through the image uniformity processing submodule, image data image enhancement submodule, and image data noise reduction processing submodule, respectively, which can improve image quality and facilitate subsequent automatic crack extraction.

[0042] The crack analysis and decision support module 400 is used to input the spliced ​​image processed by the image processing module into a pre-trained neural network model to identify cracks, obtain the crack geometry information output by the neural network model, and analyze the defect information of each section of the dam surface based on the crack geometry information and the three-dimensional dam model, and conduct risk assessment based on the defect information to assist decision-making.

[0043] In other words, the crack analysis and decision support module 400 can automatically extract and analyze dam surface defect information from dam images using a crack detection algorithm, and perform risk assessment based on the extracted defect information to assist in decision-making. In some embodiments, the crack analysis and decision support module 400 may include a crack geometry information extraction submodule 401, a 3D modeling submodule 402, and a report generation submodule 403.

[0044] The crack geometry information extraction submodule 401 is used to input the spliced ​​image processed by the image processing module into a neural network model for crack identification, obtaining crack geometry information and overlaying it on the spliced ​​image in the form of outlines and labeled boxes. This geometric information may include, but is not limited to, the crack morphology and size. As an example, the neural network model can be based on a multi-structure convolutional neural network model. This model can be trained with a large amount of annotated dam crack image data to accurately identify crack morphology, size, and location information.

[0045] 3D modeling submodule 402 is used to create a 3D dam model using aerial triangulation methods based on the stitched images. This allows the user to view a realistic 3D dam model, which is scalable and helps inspectors quickly identify potential problems.

[0046] Report generation submodule 403 is used to perform three-dimensional measurement of dam surface cracks based on the crack geometry information and the three-dimensional dam model, calculate the crack geometry data, and perform risk assessment based on the crack geometry data and location information to obtain risk assessment information. The crack geometry data (such as volume, area, etc., but not limited to), location information, and risk assessment information are compiled into a report to assist users in making decisions. In other words, report generation submodule 403 can perform three-dimensional measurement and information extraction on dam surface cracks based on the crack geometry information and the three-dimensional dam model output by crack geometry information extraction submodule 401. The report is then compiled into a report format and output in Word format, providing managers with intuitive data to assist them in risk assessment and scientific decision-making, achieving a transition from passive response to active prevention and control in dam safety management, and significantly enhancing the intelligence and efficiency of dam safety management.

[0047] In the above embodiment, the original image data of the dam surface is automatically collected by drones. The original image data taken by the drones is spliced ​​by the data management module to obtain spliced ​​images of the surface of each section of the dam. The spliced ​​images are processed by the image processing module to improve the image quality. The spliced ​​images are input into a pre-trained neural network model through the crack analysis and decision support module to identify cracks and obtain crack geometry information. Based on the crack geometry information and the three-dimensional dam model, the defect information of each section of the dam surface is analyzed. Based on the defect information, risk assessment is performed to assist decision-making, thereby achieving automatic processing and analysis of crack information and providing decision support. In other words, this application integrates the functions of data fusion, display, processing, and analysis into the system to achieve automatic detection of dam surface crack information, embodying the functional implementation and collaborative work of each module of the system, effectively solving the technical problems of dam surface crack detection and management, and can further improve the level of dam surface crack detection, timely analyze dam surface crack information, and maintain the safe operation of the hydropower station.

[0048] Figure 2 This is a flow chart of the dam surface crack detection method provided in the embodiment of the present application. It should be noted that, in some embodiments, the dam surface crack detection method can be applied to the dam surface crack detection and management system described in any of the aforementioned embodiments. Figure 2 As shown, the dam surface crack detection method may include but is not limited to the following steps.

[0049] In step S201 , a plurality of original image data of the dam surface taken by a drone camera are obtained.

[0050] For example, a drone camera can be used to capture the dam surface. Once the drone has automatically completed capturing the dam surface image, the image file is automatically transferred via a 4G (or 5G) network to a system-specified drone image storage path. This storage path can be monitored in real time. Once a new image file is discovered, an import process is immediately initiated to obtain multiple raw images of the dam surface captured by the drone camera. The image file's metadata in the database is then updated. This metadata may include, for example, the file name, capture time, and capture coordinates.

[0051] In step S202, coordinate transformation is performed on a plurality of original image data based on the transformation relationship between the image pixel coordinate system and the object coordinate system of the target, and the plurality of original image data after coordinate transformation are stitched to obtain stitched images of the surfaces of each section of the dam.

[0052] As an example, the conversion relationship can be expressed as follows:

[0053]

[0054] Among them, (x j ,y j ) is the jth image i j The pixel coordinate system, (X j ,Y j ,Z j ) is the object coordinate system, α ij is the conversion parameter. Through this conversion relationship, the different pixel coordinate systems of each image taken by the drone camera can be converted into a unified real-world coordinate system and the images can be stitched together to obtain complete image data of the entire dam surface for each section.

[0055] In some embodiments, the stitched image is stored in a database, an index is created in the database, and image-related parameters are recorded, wherein the index and image-related parameters are used for fast calling and retrieval.

[0056] In step S203, the stitched image is displayed based on the selected image browsing mode, which includes a two-dimensional image browsing mode and a three-dimensional model browsing mode. When the selected image browsing mode is the three-dimensional model browsing mode, the three-dimensional dam model corresponding to the stitched image is displayed. The three-dimensional dam model is established based on the stitched image through the aerial triangulation calculation method, and the three-dimensional dam model is scalable.

[0057] In step S204, image processing is performed on the stitched image based on at least one processing method; wherein the at least one processing method includes an image dodging processing method, an image enhancement processing method, and a noise reduction processing method.

[0058] In some embodiments, image processing is performed on the stitched image based on the image dodging processing method, including: filtering the stitched image using a Gaussian filter to separate the background illumination component, and adjusting the brightness of the stitched image using a dodging correction formula based on the background illumination component. Exemplarily, the background illumination component can be determined by the following formula: B(x, y) = G σ (x,y)*I(x,y), I(x,y) is the brightness value of the original image at the pixel point (x,y). It is the input image without any processing and is directly taken by the camera carried by the drone. It serves as the input data of the algorithm and reflects the actual light distribution on the dam surface. G σ (x, y) is the Gaussian kernel function, σ is the standard deviation of the Gaussian kernel, which controls the smoothing scale of the filter, (x, y) is the coordinate inside the kernel, and the size of the kernel is usually (6σ+1)×(6σ+1) to ensure that the main energy is covered. Through the convolution operation B(x, y) = G σ (x,y)*I(x,y), the low-pass characteristic of the Gaussian kernel can filter out high-frequency details (such as cracks and noise) in the image and retain low-frequency background illumination information. The uniform light correction formula is ε is a minimum constant, and μ is a global brightness adjustment factor, which is used to adjust the global brightness level of the image after uniform lighting.

[0059] In some embodiments, the stitched images are processed based on an image enhancement method, including: processing the stitched images using a low-light enhancement method based on Retinex theory; the low-light enhancement method based on Retinex theory is used to highlight the characteristic information of the cracks in the stitched images and weaken the background information. In other words, the main purpose of image enhancement processing is to highlight the characteristic information of the cracks and weaken the information of the background, thereby further highlighting the crack details and facilitating the subsequent extraction and identification of concrete crack features. Crack information enhancement mainly uses a low-light enhancement method based on Retinex theory, and its model is as follows: Among them, R(x,y) is the image after low-light enhancement, I(x,y) is the original low-light spliced ​​image, and I low (x,y) is the low-frequency component, and ε is the smoothing factor.

[0060] In step S205 , the spliced ​​image processed by the image processing module is input into a pre-trained neural network model for crack recognition, and the crack geometry information output by the neural network model is obtained.

[0061] As an example, the neural network model can be a multi-structure convolutional neural network model, which can be trained with a large amount of labeled dam crack image data and can accurately identify the shape, size and location information of the cracks.

[0062] In step S206, the defect information of each section of the dam surface is analyzed based on the crack geometry information and the three-dimensional dam model, and a risk assessment is performed based on the defect information to assist decision-making.

[0063] In some embodiments, based on the crack geometry information and the three-dimensional dam model, dam surface cracks can be three-dimensionally measured, the crack geometry data can be calculated, and a risk assessment can be performed based on the crack geometry data and location information to obtain risk assessment information. The crack geometry data (such as, but not limited to, length, width, area, etc.), location information, and risk assessment information can be compiled into a report to assist users in making decisions. In other words, based on the crack geometry information and the three-dimensional dam model, dam surface cracks can be three-dimensionally measured and information extracted. The relevant statistical information (such as the crack geometry data, location information, and risk assessment information) can be compiled into a report and output in Word format, providing managers with intuitive data to assist them in risk assessment and scientific decision-making, realizing the transformation of dam safety management from passive response to active prevention and control, and greatly enhancing the intelligence and efficiency of dam safety management.

[0064] In the above embodiment, the original image data of the dam surface is automatically collected by drones. The original image data taken by the drones is spliced ​​to obtain spliced ​​images of the surface of each section of the dam. The spliced ​​images are processed to improve the image quality. The spliced ​​images are input into a pre-trained neural network model to identify cracks and obtain crack geometry information. Based on the crack geometry information and the three-dimensional dam model, the defect information of each section of the dam surface is analyzed. Based on the defect information, risk assessment is performed to assist decision-making, thereby achieving automatic processing and analysis of crack information and providing decision support. In other words, this application integrates data fusion, display, processing, and analysis functions into the system to achieve automatic detection of dam surface crack information, embodying the functional implementation and collaborative work of each module of the system, effectively solving the technical problems of dam surface crack detection and management, further improving the level of dam surface crack detection, timely analyzing dam surface crack information, and maintaining the safe operation of the hydropower station.

[0065] In order to facilitate those skilled in the art to more clearly understand the present application, the following will be combined with the embodiments to clearly and completely describe the technical solution of the present application. Taking the crack detection of the dam surface of a hydropower station as an example, the main contents are introduced as follows:

[0066] 1. Operating environment: A high-performance server is selected as the system operating platform, and a drone equipped with a camera is selected to support multi-angle high-definition image shooting to ensure the acquisition of high-quality dam surface image data.

[0067] 2. Specific implementation steps:

[0068] (1) Data management module operation

[0069] 1. Image Data Import: After the drone automatically captures the dam surface image, it automatically transfers the image file to a system-specified storage path via the 4G (or 5G) network. The image data import submodule monitors this path in real time. Once a new image file is detected, it immediately initiates the import process and updates the image file's metadata in the database, including file name, capture time, and capture coordinates.

[0070] 2. Image data stitching: The image data stitching submodule reads the imported image data and uses the image pixel coordinate system (x j ,y j ) and the object coordinate system (X j ,Y j ,Z j ) The different pixel coordinate systems of each image are converted into a unified real coordinate system and the images are stitched together to obtain complete image data of the surface of each section of the entire dam.

[0071] 3. Image data storage: After the stitching is completed, the image data storage submodule will store the generated complete image data in the database, and at the same time create an index in the database to record the image stitching parameters, shooting range and other information for subsequent quick call and retrieval.

[0072] (2) Image display module operation

[0073] 1. 2D Image Plane Display: Users select 2D image viewing mode through the system interface. The image display module reads the spliced ​​dam image data from the database and displays the image in the interface window. Use the mouse wheel to zoom in and out, and drag the mouse to pan the image, making it easy to view cracks in different areas of the dam surface.

[0074] 2. 3D Dam Model Display: The image display module first performs aerotriangulation on the stitched images. The software automatically calculates the image's exterior orientation elements and encrypted point coordinates to construct a sparse point cloud model of the dam. Next, through point cloud encryption and texture mapping, a 3D dam model with realistic textures is generated. Within the system's 3D browsing interface, the model can be rotated, scaled, translated, and navigated freely using the keyboard and mouse. The software's built-in measurement tools can also be used to measure geometric parameters such as crack length and width.

[0075] (3) Image processing module execution

[0076] 1. Image uniformity processing: The image uniformity processing submodule reads the spliced ​​image data and filters the image using the Gaussian filter formula to separate the background illumination component. The image brightness is adjusted using the uniformity correction formula to achieve uniform image brightness, where σ = 50 and ε = 10. -6 , μ=128.

[0077] 2. Image data image enhancement: The image data image enhancement submodule is based on Retinex theory and uses low-light enhancement method to process images. Processing, where I(x,y) is the original image, I low (x, y) is the low-frequency component, ε is the smoothing factor, and the smoothing factor can be set to 10 to highlight the characteristic information of the crack and weaken the background information.

[0078] 3. Image data denoising: The image data denoising submodule uses the median filtering method, sets the filter window size to 3×3, traverses each pixel of the image, sorts the pixel values ​​in the window by grayscale value, and takes the median as the new grayscale value of the current pixel, effectively removing salt and pepper noise and Gaussian noise in the image and improving image quality.

[0079] (IV) Operation of crack analysis and decision support modules

[0080] 1. Crack Geometric Information Extraction: The crack geometric information extraction submodule inputs the processed image data from the image processing module into a multi-structure convolutional neural network model. Trained with extensive annotated dam crack image data, the model accurately identifies crack morphology, size, and location. The extracted crack information is displayed as outlines and labeled boxes overlaid on the image, and the crack's geometric parameters (length, width, direction, etc.) are stored in a database.

[0081] 2. 3D Modeling: The 3D modeling submodule uses aerial triangulation to construct a model of the dam using the stitched drone images. This model features highly accurate geometry and realistic textures. Inspectors can freely navigate the 3D scene, observing the dam surface from various angles to identify potential cracks.

[0082] 3. Report Generation: The report generation submodule uses the image extraction results and 3D model construction results to perform 3D measurements of dam surface cracks and calculate parameters such as crack volume and area. Crack geometry, location, and risk assessment data are compiled into reports and output in Word format, providing managers with intuitive and comprehensive dam crack analysis reports to assist in decision-making.

[0083] Through the above detailed embodiments, the whole process of dam surface crack detection and management system from data collection and processing to analysis and decision-making is fully demonstrated, reflecting the functional realization and collaborative work of each module of the system, effectively solving the technical problems of dam surface crack detection and management, and has good practicality and promotion value.

[0084] According to an embodiment of the present application, the present application also provides an electronic device and a readable storage medium.

[0085] like Figure 3 , is a block diagram of an electronic device according to an embodiment of the present application. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.

[0086] like Figure 3 As shown, the electronic device includes: one or more processors 301, a memory 302, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are connected to each other using different buses and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the electronic device, including instructions stored in or on the memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to the interface). In other embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple electronic devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 3 A processor 301 is taken as an example.

[0087] Memory 302 is the non-transitory computer-readable storage medium provided in this application. The memory stores instructions executable by at least one processor, causing the at least one processor to perform the dam surface crack detection method provided in this application. The non-transitory computer-readable storage medium of this application stores computer instructions for causing a computer to perform the dam surface crack detection method provided in this application.

[0088] The memory 302 is a non-transient computer-readable storage medium that can be used to store non-transient software programs, non-transient computer executable programs and modules, such as the program instructions / modules corresponding to the dam surface crack detection method in the embodiment of the present application (for example, the attached Figure 1 The processor 301 executes the non-transient software programs, instructions, and modules stored in the memory 302 to execute various functional applications and data processing of the server, thereby implementing the dam surface crack detection method in the above-mentioned method embodiment.

[0089] The memory 302 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and applications required for at least one function; the data storage area may store data created based on the use of the electronic device, etc. In addition, the memory 302 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory 302 may optionally include a memory remotely located relative to the processor 301, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0090] The electronic device may further include: an input device 303 and an output device 304. The processor 301, the memory 302, the input device 303 and the output device 304 may be connected via a bus or other means. Figure 3 The bus connection is taken as an example.

[0091] The input device 303 can receive input digital or character information and generate key signal input related to user settings and function control of the electronic device, such as input devices such as a touch screen, a keypad, a mouse, a trackpad, a touch pad, a pointer, one or more mouse buttons, a trackball, and a joystick. The output device 304 may include a display device, an auxiliary lighting device (e.g., an LED), and a tactile feedback device (e.g., a vibration motor). The display device may include, but is not limited to, a liquid crystal display (LCD), a light emitting diode (LED) display, and a plasma display. In some embodiments, the display device may be a touch screen.

[0092] Various implementations of the systems and techniques described herein can be realized in digital electronic circuit systems, integrated circuit systems, dedicated ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0093] These computer programs (also referred to as programs, software, software applications, or code) include machine instructions for a programmable processor and can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and / or device (e.g., a magnetic disk, an optical disk, a memory, a programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0094] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0095] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.

[0096] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact via a communication network. This client-server relationship is established by computer programs running on the respective computers, establishing a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host, a host product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosts and VPS services ("Virtual Private Servers" or simply "VPS"). The server may also be a server in a distributed system or a server integrated with blockchain.

[0097] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this application can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this application can be achieved. This is not a limitation herein.

[0098] The above specific embodiments do not constitute a limitation on the scope of protection of this application. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application shall be included within the scope of protection of this application.

Claims

1. A dam surface crack detection and management system, characterized in that: include: a data management module for acquiring a plurality of original image data of the dam surface taken by a drone camera, performing coordinate transformation on the plurality of original image data based on a transformation relationship between an image pixel coordinate system and an object coordinate system of a target, and performing image stitching on the plurality of original image data after the coordinate transformation to obtain a stitched image of each section of the dam surface; an image display module, configured to display the stitched image based on a selected image browsing mode, wherein the image browsing mode includes a two-dimensional image browsing mode and a three-dimensional model browsing mode, wherein when the selected image browsing mode is the three-dimensional model browsing mode, a three-dimensional dam model corresponding to the stitched image is displayed, wherein the three-dimensional dam model is established based on the stitched image using an aerial triangulation method, and the three-dimensional dam model is scalable; An image processing module, configured to perform image processing on the stitched image based on at least one processing method; wherein the at least one processing method includes an image dodging processing method, an image enhancement processing method, and a noise reduction processing method; The crack analysis and decision-making support module is used to input the spliced ​​image processed by the image processing module into a pre-trained neural network model to identify cracks, obtain crack geometry information output by the neural network model, analyze defect information on the surface of each section of the dam based on the crack geometry information and the three-dimensional dam model, and conduct risk assessment based on the defect information to assist decision-making.

2. The system according to claim 1, wherein: The data management module includes: The image data import submodule is used to monitor the specified drone image storage path in real time, and when it detects the presence of new raw image data in the drone image storage path, it will start the import program to import the new raw image data and update the metadata information of the image file in the database; An image data stitching submodule is used to read the imported original image data, convert the different pixel coordinate systems of each image data into a unified real coordinate system through the conversion relationship between the image pixel coordinate system and the target object coordinate system, and complete the image stitching to obtain the stitched image of each section of the dam surface; The image data storage submodule is used to store the stitched image obtained after stitching into the database, create an index in the database, and record image-related parameters, wherein the index and the image-related parameters are used for fast calling and retrieval.

3. The system according to claim 1, wherein: The image processing module includes at least one of the following: An image dodging processing submodule is used to filter the stitched image using a Gaussian filter to separate the background illumination component, and adjust the brightness of the stitched image using a dodging correction formula based on the background illumination component; An image enhancement processing submodule, configured to process the stitched image using a low-light enhancement method based on Retinex theory; the low-light enhancement method based on Retinex theory is configured to highlight feature information at cracks in the stitched image and weaken background information; The noise reduction processing submodule is used to perform noise reduction processing on the stitched image using a filtering algorithm.

4. The system according to claim 3, characterized in that The model of the low-light enhancement method based on Retinex theory is expressed as follows: Among them, R(x, y) is the image after low-light enhancement, I(x, y) is the original low-light stitched image, and I low (x,y) is the low-frequency component, and ε is the smoothing factor.

5. The system according to claim 1, wherein: The crack analysis and decision support module includes: a crack geometry information extraction submodule, configured to input the spliced ​​image processed by the image processing module into the neural network model to identify cracks, obtain the crack geometry information, and superimpose the crack geometry information on the spliced ​​image in the form of contour lines and annotation boxes; A three-dimensional modeling submodule is used to build a model of the spliced ​​images using an aerial triangulation method to obtain a three-dimensional dam model; The report generation submodule is used to perform three-dimensional measurement of the dam surface cracks based on the crack geometry information and the three-dimensional dam model, calculate the geometric data of the cracks, and perform risk assessment based on the geometric data and position information of the cracks to obtain risk assessment information. The geometric data, position information and risk assessment information of the cracks are compiled into a report, and the report is used to assist users in making decisions.

6. A method for detecting cracks on a dam surface, characterized in that: The method is implemented based on the dam surface crack detection and management system according to any one of claims 1 to 5, and the method comprises the following steps: Acquire multiple original image data of the dam surface taken by the drone camera; performing coordinate transformation on the plurality of original image data based on a transformation relationship between an image pixel coordinate system and an object coordinate system of a target, and performing image stitching on the plurality of original image data after the coordinate transformation to obtain a stitched image of the surface of each section of the dam; displaying the stitched image based on a selected image browsing mode, the image browsing mode including a two-dimensional image browsing mode and a three-dimensional model browsing mode, wherein when the selected image browsing mode is the three-dimensional model browsing mode, a three-dimensional dam model corresponding to the stitched image is displayed, the three-dimensional dam model being established based on the stitched image through an aerial triangulation method, and the three-dimensional dam model being scalable; Performing image processing on the stitched image based on at least one processing method; wherein the at least one processing method includes an image dodging processing method, an image enhancement processing method, and a noise reduction processing method; Inputting the spliced ​​image processed by the image processing module into a pre-trained neural network model to identify cracks, and obtaining crack geometry information output by the neural network model; The defect information on the surface of each section of the dam is analyzed based on the crack geometry information and the three-dimensional dam model, and risk assessment is performed based on the defect information to assist decision-making.

7. The method according to claim 6, characterized in that The acquisition of multiple original image data of the dam surface taken by the drone camera includes: The designated drone image storage path is monitored in real time, and when new original image data is detected in the drone image storage path, an import program is started to obtain multiple original image data of the dam surface taken by the drone camera.

8. The method according to claim 6, characterized in that The method further comprises: The stitched image obtained after stitching is stored in the database, an index is created in the database, and image-related parameters are recorded, wherein the index and the image-related parameters are used for fast calling and retrieval.

9. The method according to any one of claims 6 to 7, characterized in that Performing image processing on the stitched image based on the image dodging processing method, including: filtering the stitched image using a Gaussian filter to separate a background illumination component, and adjusting the brightness of the stitched image using a dodging correction formula based on the background illumination component; The stitched image is processed based on the image enhancement processing method, including: processing the stitched image using a low-light enhancement method based on Retinex theory; the low-light enhancement method based on Retinex theory is used to highlight the feature information of the cracks in the stitched image and weaken the background information.

10. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 6 to 9.

Images