A multi-source data integration system for intelligent identification and detection of underwater defects in hydropower station dams

Abstract

This invention discloses a multi-source data integration system for intelligent identification and detection of underwater defects in hydropower station dams. The system comprises a data acquisition module, a data integration module, a raw data input module, a data detection module, a detection data output module, and a data display module. The data acquisition module integrates multi-source sensor information, including underwater high-definition video, image sonar, three-dimensional laser scanners, and robotic arm contact detection. The data integration module develops multi-source information fusion processing technology and establishes an underwater station dam defect database, providing data support for underwater station dam defect detection. The data detection module enables the fusion and real-time extraction of multi-source data feature information. The data display module filters, queries, and analyzes stored data, addressing issues such as opaque data management and visualization. The entire system framework features a clear structure and convenient operation, providing a well-organized human-computer interaction platform.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent identification and detection of underwater defects in hydropower station dams, and relates to a multi-source data integration system applied to intelligent identification and detection of underwater defects in hydropower station dams. Background Art

[0002] Under long-term loads and environmental influences, concrete dams are bound to develop various defects, including cracks, spalling, and exposed rebar. These defects are a major contributing factor to dam failures. Deepwater areas of dams are affected by the water environment, making inspection difficult, inefficient, and requiring limited means. Existing intelligent underwater inspections primarily utilize umbilical cable underwater robots equipped with acoustic and optical detection equipment. To address this issue, integrating multi-source information perception and data interaction modules, such as high-definition cameras, image sonar, and 3D laser scanners, can more fully utilize existing inspection resources and provide strong support for data collection, management, and sharing.

[0003] However, there is currently a problem that different detection equipment and different forms of detection data cannot be effectively integrated (Chen D, A Review of Detection Technologies for Underwater Cracks on Concrete Dam Surfaces, Applied Sciences, 2023.13.3564). Secondly, in terms of intelligent identification, there is a lack of corresponding underwater defect databases that integrate disease information (Ben H, Underwater dam crack image generation based on unsupervised image-to-image translation, Automation in Construction, 2024.105430). At the same time, in actual applications, defect inspection, defect location, and defect size measurement cannot be recorded synchronously online.

[0004] To this end, we propose a multi-source data integration system for intelligent identification and detection of underwater defects in hydropower station dams, which synchronously acquires acoustic and optical images, three-dimensional point cloud feature data, and records defect element information. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this paper proposes a multi-source data integration system for intelligent identification and detection of underwater defects in hydropower station dams. This system integrates multi-source sensing information, including underwater high-definition videography, image sonar, and laser scanners, to centrally manage equipment and detection information, facilitating the entire detection process. The system also establishes an underwater dam defect database, stores and analyzes detection data, and enables data visualization, providing an effective and feasible solution for multi-source data integration in the current field of intelligent detection of underwater dam defects.

[0006] The technical solution of the present invention:

[0007] A multi-source data integration system for intelligent identification and detection of underwater defects in hydropower station dams includes a data acquisition module S1, a data integration module S2, a raw data input module S3, a data detection module S4, a detection data output module S5 and a data display module S6. The data acquisition module S1 is connected to the data integration module S2, and a user obtains raw data during the dam detection process through the data acquisition module S1 and uploads it to the data integration module S2 for integrated storage; the data integration module S2 is connected to the raw data input module S3, and the raw data input module S3 is connected to the data detection module S4, and performs intelligent identification on the raw data collected by the user; the data detection module S4 is connected to the detection data output module S5, and the detection data output module S5 is connected to the data integration module S2, and uploads the detected data to the data integration module S2 for classified storage; the data integration module S2 is connected to the data display module S6, and a user views the collected raw data and detection result data by accessing the data display module S6.

[0008] The data acquisition module S1 includes a robot observation system S101, a sonar data management system S102 and a three-dimensional laser scanning system S103.

[0009] The data integration module S2 includes a data receiving unit S201, a data processing unit S202, a data classification unit S203 and a data storage unit S204. The data receiving unit S201 is connected to the data processing unit S202, the data processing unit S202 is connected to the original data input module S3, the detection data output module S5 is connected to the data classification unit S203, and the data classification unit S203 is connected to the data storage unit S204.

[0010] The data detection module S4 is mainly composed of a dam underwater defect intelligent detection system S401.

[0011] The data display module S6 includes a data screening unit S601, a data query unit S602 and a data analysis unit S603. The data screening unit S601 is connected to the data query unit S602 and the data analysis unit S603 respectively.

[0012] The present invention proposes a multi-source data integration system for intelligent identification and detection of underwater defects in hydropower station dams. This system exhibits the following beneficial effects: The data acquisition module integrates multi-source sensing information, including underwater high-definition videography, image sonar, and three-dimensional laser scanners, enriching the management of underwater station and dam detection information. The data integration module develops multi-source information fusion processing technology and establishes an underwater station and dam defect database, providing data support for underwater station and dam defect detection. The data detection module enables the fusion and real-time extraction of multi-source data feature information. The data display module filters, queries, and analyzes stored data, resolving issues such as opaque data management visualization. The entire system framework boasts a clear structure and convenient operation, providing a well-organized human-computer interaction platform. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 The present invention is a schematic diagram of the overall structure of a multi-source data integration system for intelligent identification and detection of underwater defects in hydropower station dams.

[0014] Figure 2 The present invention is a structural diagram of a data acquisition module in a multi-source data integration system for intelligent identification and detection of underwater defects in a hydropower station dam.

[0015] Figure 3 The present invention is a structural diagram of a data integration module in a multi-source data integration system for intelligent identification and detection of underwater defects in a hydropower station dam.

[0016] Figure 4 The present invention is a structural schematic diagram of a data display module in a multi-source data integration system for intelligent identification and detection of underwater defects in a hydropower station dam.

[0017] Figure 5 This is a structural diagram of an underwater dam surface defect detection algorithm in a multi-source data integration system used for intelligent identification and detection of underwater defects in hydropower station dams according to the present invention.

[0018] In the figure: S1 data acquisition module; S101 robot observation system; S102 sonar data management system; S103 three-dimensional laser scanning system; S2 data integration module; S201 data receiving unit; S202 data integration unit; S203 data classification unit; S204 data processing unit; S3 raw data input module; S4 data detection module; S401 underwater dam surface defect intelligent detection system; S5 detection data output module; S6 data display module; S601 data screening unit; S602 data query unit; S603 data analysis unit. DETAILED DESCRIPTION

[0019] The specific implementation of the present invention is further described below in conjunction with the accompanying drawings and technical solutions.

[0020] Example:

[0021] A multi-source data integration system for intelligent identification and detection of underwater defects in hydropower station dams, such as Figure 1 As shown, it includes a data acquisition module S1, a data integration module S2, a raw data input module S3, a data detection module S4, a detection data output module S5 and a data display module S6. The data acquisition module S1 is connected to the data integration module S2. The user obtains the raw data during the dam detection process through the data acquisition module S1 and uploads it to the data integration module S2 for integrated storage. The data integration module S2 is connected to the raw data input module S3. The raw data input module S3 is connected to the data detection module S4 to perform intelligent recognition on the raw data collected by the user. The data detection module S4 is connected to the detection data output module S5. The detection data output module S5 is connected to the data integration module S2 to upload the completed detection data to the data integration module S2 for classified storage. The data integration module S2 is connected to the data display module S6. The user views the collected raw data and detection result data by accessing the data display module S6.

[0022] Further, such as Figure 2 As shown, the data acquisition module S1 includes a robot observation system S101, a sonar data management system S102 and a three-dimensional laser scanning system S103.

[0023] Further, such as Figure 3 As shown, the data integration module S2 includes a data receiving unit S201, a data processing unit S202, a data classification unit S203 and a data storage unit S204, the data receiving unit S201 is connected to the data processing unit S202, the data processing unit S202 is connected to the original data input module S3, the detection data output module S5 is connected to the data classification unit S203, and the data classification unit S203 is connected to the data storage unit S204.

[0024] Further, such as Figure 4 As shown, the data display module S6 includes a data screening unit S601, a data query unit S602 and a data analysis unit S603, and the data screening unit S601 is connected to the data query unit S602 and the data analysis unit S603 respectively.

[0025] Working Principle: This invention integrates multi-source sensing information, including underwater high-definition videography, image sonar, and 3D laser scanners, improving equipment utilization efficiency and enriching methods for managing underwater station and dam inspection information. Furthermore, it establishes an underwater station and dam defect database to store and analyze inspection data, providing data support for underwater station and dam defect detection. The stored data is screened, queried, and analyzed, addressing issues such as opaque data management and visualization, providing an effective and feasible approach to multi-source data integration in the current field of underwater station and dam defect detection.

[0026] The present invention discloses a multi-source data integration system for intelligent identification and detection of underwater defects in hydropower station dams. The system comprises a main system and subsystems. The main system includes a data integration module S2, a raw data input module S3, a detection data output module S5, and a data display module S6. The main system is primarily used for the integration, classification, storage, screening, and analysis of multi-source detection data for underwater structural defects in hydropower station dams. The subsystems comprise a data acquisition module S1 and a data detection module S4. The subsystems are primarily used for the collection and detection of underwater structural defects in hydropower station dams. The underwater acoustic, optical, and point cloud data acquired by the data acquisition module S1 constitute the underwater multi-source data. The detection information acquired by the data detection module S4 encompasses underwater defect preprocessing, intelligent identification, positioning, and measurement.

[0027] Data acquisition module S1 comprises a robotic observation system S101, a sonar data management system S102, and a 3D laser scanning system S103. The robotic observation system S101 primarily acquires image data captured by underwater high-definition cameras. This image data is transmitted and managed via an SDK interface provided by the C# language. The system's user interface is developed using the WPF (Windows Presentation Foundation) framework. The sonar data management system S102 collects underwater acoustic data from multi-beam, single-beam, and side-scan sonars. Due to the closed-source design of the sonar software, a WebView webpage nesting tool is used, combined with image capture and recording methods, to achieve multi-sonar data acquisition. The 3D laser scanning system S103 collects point cloud data generated by the 3D laser scanner. This point cloud data acquisition is guided by executing CMD commands within a C# process. Together, these three systems comprise data acquisition module S1, which is used for multi-source detection data acquisition of underwater stations and dams. The collaborative operation of these systems enables comprehensive acquisition of optical, acoustic, and 3D geometric information of the underwater environment.

[0028] The data integration module S2 includes a data receiving unit S201, a data processing unit S202, a data classification unit S203, and a data storage unit S204. The data receiving unit S201 directly acquires the multi-source data output by the data acquisition module S1. The data processing unit S202 performs compliance processing on the raw data obtained by the data receiving unit S201 from the data acquisition module S1, and then inputs the data to be tested into the raw data input module S3. The data classification unit S203 classifies the test information data obtained from the test data output module S5 for easy storage. The data storage unit S204 stores the test data processed by the data classification unit S203 in a standardized manner, facilitating the integration and management of multi-source data.

[0029] The data integration module S2 is the core of the entire system, responsible for data interaction and management within the system. Its backend database is built using MySQL. The data receiving unit S201 is responsible for receiving multi-source data from various sensors and data sources (such as underwater high-definition cameras, image sonar, and 3D laser scanners). It uses multiple communication protocols, including HTTP and MQTT, to ensure compatibility with different devices. The multi-source data is accompanied by a JSON file that stores descriptive information (such as timestamps, device IDs, and sensor readings) for easy parsing and processing. The data processing unit S202 uses the Pandas (Python) library to process the received data, performing cleansing and format conversion, and converting the data into a dictionary format that conforms to the MySQL table structure. To improve processing efficiency, this unit implements batch processing to reduce database interactions. Data Classification Unit S203 categorizes the detection results output by Data Detection Module S4 based on device, time, location, disease details, and confidence level. It also performs key field matching on the multi-source data received by Data Receiving Unit S201 to achieve intelligent classification. Data Storage Unit S204 saves and synchronizes data to a MySQL database based on the classification results. Specifically, screenshots are uploaded directly, but due to the large size and low upload efficiency of recorded video files, a local storage strategy is adopted, uploading only the video folder path to the database.

[0030] The data detection module S4 is mainly composed of the dam underwater defect intelligent detection system S401. The data detection module S4 will obtain the data to be detected from the original data input module S3 for detection. The dam underwater defect intelligent detection system S401 mainly includes (1) detection model data set production, (2) detection interface, (3) detection model (optimized yolov8s network), and (4) detection model training.

[0031] (1) Detection model dataset preparation: including training data collection and labeling and training data enhancement. Data collection and labeling: First, the images of the dam underwater video collected in the field are extracted by frame extraction, and then typical underwater defect images are manually selected. Finally, they are manually labeled with the help of LabelImg. Data enhancement uses the OpenCV-Python image processing library to expand the labeled data. The purpose is to expand the number of manually labeled images. The expansion operations include random horizontal flipping, random scaling, random collection, and random brightness change. Before expansion, 80% of the dataset is used as the training set and 20% of the dataset is used as the test set. Then, the training set and test set are expanded respectively. The final number of expansions can be defined by yourself. The dam underwater defect dataset is generally expanded to between 5 and 10 times the original data.

[0032] (2) Detection interface: Detection interface: The detection interface is an interactive interface provided for underwater defect detection of dams. It is developed using PySide6 and has a front-end and back-end separation design. The front-end uses an operation interface built with QTDesigner. Through the interface, the detection model weight in the local folder is selected, and the input video and detection results are displayed in real time. The detection results can also be saved and exported. The back-end uses Python to implement specific model call operations and real-time display.

[0033] (3) Detection model: specially designed for defect target detection tasks in underwater environments. Figure 5 As shown in the figure, based on the actual situation of underwater defects, the YOLOv8s network structure is optimized to fit the characteristics of underwater defect data and improve the network detection performance. There are three specific improvements:

[0034] The original network Neck structure is replaced with Slim-Neck. This structure introduces the GSConv module, which can effectively reduce the negative impact of the depthwise separable convolution (DSC) on the network and fully utilize the advantages of depthwise separable convolution to effectively reduce computational costs.

[0035] Replace BottleNeck in the C2f module in the network backbone with ConvNeXt pure convolution lightweight high-performance structure to improve detection efficiency;

[0036] The positioning loss function is replaced by the Focal-EIoU function to balance the contribution of high-quality and low-quality samples to the loss function, and improve the problem of limited accuracy improvement speed caused by low-quality samples.

[0037] Based on the above improvements, a network for intelligent underwater defect recognition is proposed. The improved YOLOv8s network has higher detection efficiency and improves the recognition effect of underwater defects.

[0038] (4) Detection model training and testing: The training set of the prepared detection model training data set is sent to the detection model for training until the accuracy of the model on the test set is relatively stable. The training is stopped. At this time, the model will be used for the underwater defect detection task of the dam.

[0039] The data detection module S4 can perform real-time analysis on the input underwater image or video data, automatically identifying and marking the defect category (such as cracks, erosion, cavities, etc.), specific location (such as depth, azimuth coordinates), and detection confidence.

[0040] The data display module S6 includes a data screening unit S601, a data query unit S602, and a data analysis unit S603. The data screening unit S601 can filter the data required for display according to a series of candidate conditions; the data query unit S602 can display, preview, and sort the filtered data that meets the conditions, allowing users to freely view test segments and results; at the same time, the data analysis unit S603 can perform statistics and analysis on the test data, highlighting the key points of the test results and realizing data management visualization. Among them, the data screening unit S601 is designed with an interactive screening form, covering options such as detection equipment (high-definition camera, image sonar, 3D laser scanner), detection time and defect type (underwater defect types such as cracks, falling blocks, peeling and exposed rebar). After the user fills out the form as needed, the system will initiate a request to the MySQL database and immediately feedback the screening results for display; the data query unit S602, through the integration of DataQuery method, combines with the FFMPEG library to efficiently retrieve qualified image and video results, and present them intuitively in the form of an information window; the data analysis unit S603, through the chart control in the C# language, conducts in-depth statistics and analysis on the screened defect detection data, automatically generates detailed result reports and charts, and clearly shows the distribution characteristics and development trends of the defect data.

[0041] Specifically, after the user logs in with a username and password, he or she can start the robot observation system S101, sonar data management system S102, and three-dimensional laser scanning system S103 in the data acquisition module S1 to obtain the camera, image sonar, and three-dimensional laser scanner images respectively, and bind the specified window to preview the image, select screenshots and recordings of the specified images. For screenshots, after the screenshot is successfully taken, it is uploaded to the IIS server for storage. For recordings, due to the slow upload time, after the recording is successful, it is saved locally, but the folder directory where the video is saved will be uploaded to the IIS server, and then synchronized to the MySQL database for storage. For the collected data, the data detection module S4 can directly obtain and detect it from the server and output the results to the specified path. The data integration module S2 can obtain the detection results through this path and process and store them. Afterwards, the user can operate the data display module S6 to obtain the original data and result information they want to view, and can also calculate the statistical results in the form of charts to intuitively display the distribution characteristics and change trends of the defect data.

[0042] The above shows and describes the basic ideas, important features and outstanding effects of the present invention. For ordinary technicians in this field, the general principles defined in this article can be implemented in other embodiments without departing from the spirit or concept of the present invention. Therefore, the present invention will also have various changes and improvements, and these changes and improvements should fall within the scope of the invention claimed for protection.

Claims

1. A multi-source data integration system for intelligent identification and detection of underwater defects in hydropower station dams, characterized by: The multi-source data integration system includes a data acquisition module S1, a data integration module S2, a raw data input module S3, a data detection module S4, a detection data output module S5 and a data display module S6. The data acquisition module S1 is connected to the data integration module S2. The user obtains the raw data during the dam detection process through the data acquisition module S1 and uploads it to the data integration module S2 for integrated storage; the data integration module S2 is connected to the raw data input module S3, and the raw data input module S3 is connected to the data detection module S4 to perform intelligent recognition on the raw data collected by the user; the data detection module S4 is connected to the detection data output module S5, and the detection data output module S5 is connected to the data integration module S2 to upload the completed detection data to the data integration module S2 for classified storage; the data integration module S2 is connected to the data display module S6, and the user views the collected raw data and detection result data by accessing the data display module S6; The data acquisition module S1 includes a robot observation system S101, a sonar data management system S102 and a three-dimensional laser scanning system S103; The data integration module S2 includes a data receiving unit S201, a data processing unit S202, a data classification unit S203 and a data storage unit S204, wherein the data receiving unit S201 is connected to the data processing unit S202, the data processing unit S202 is connected to the original data input module S3, the detection data output module S5 is connected to the data classification unit S203, and the data classification unit S203 is connected to the data storage unit S204; The data detection module S4 is mainly composed of the dam underwater defect intelligent detection system S401; The data display module S6 includes a data screening unit S601, a data query unit S602 and a data analysis unit S603, wherein the data screening unit S601 is connected to the data query unit S602 and the data analysis unit S603 respectively; The data detection module S4 is mainly composed of the dam underwater defect intelligent detection system S401. The data detection module S4 obtains the data to be detected from the original data input module S3 for detection, including the production of detection model data set, detection interface, detection model, and detection model training; (1) Detection model dataset preparation: including training data collection and annotation and training data enhancement; Training data collection and labeling: First, we extract frames from the underwater videos of the dam collected on site to form the image dataset. Then, we select typical underwater defect images and finally perform manual labeling using LabelImg. Training data augmentation uses the OpenCV-Python image processing library to augment the labeled data. Augmentation operations include random horizontal flipping, random scaling, random sampling, and random brightness changes. Before augmentation, 80% of the dataset is used as the training set and 20% of the dataset is used as the test set. The training set and test set are then augmented separately. (2) Detection interface: Detection interface: The detection interface is an interactive interface provided for underwater defect detection of dams. It is developed using PySide6 and has a front-end and back-end separation design. The front-end uses an operation interface built with QTDesigner. Through the interface, the detection model weight in the local folder is selected, and the input video and detection results are displayed in real time. The detection results can also be saved and exported. The back-end uses Python to implement specific model call operations and real-time display. (3) Detection model Optimize the YOLOv8s network structure: The Neck structure of the YOLOv8s network is replaced with the Slim-Neck structure, and the Slim-Neck structure is introduced into the GSConv module; Replace BottleNeck in the C2f module in the YOLOv8s network backbone with ConvNeXt; Replace the positioning loss function with the Focal-EIoU function; (4) Detection model training and testing: The training set of the prepared detection model training data set is sent to the detection model for training until the accuracy of the model on the test set is relatively stable; In the data display module S6, the data screening unit S601 screens the data required for display; the data query unit S602 displays, previews and sorts the data that meets the conditions after screening, and the user can freely view the detection fragments and results; the data analysis unit S603 performs statistics and analysis on the detection data, highlights the key points of the detection results, and realizes data management visualization.

2. The multi-source data integration system according to claim 1, characterized in that: In the data acquisition module S1, the robot observation system S101 is mainly used to obtain image data captured by the underwater high-definition camera. The image data is transmitted and managed through the SDK interface provided by the C# language, and the operation interface of the robot observation system S101 is developed using the WPF framework; the sonar data management system S102 is used to collect underwater acoustic data of multi-beam, single-beam and side-scan sonars, and adopts the WEBVIEW webpage nesting tool, combined with the picture capture and recording method, to realize the acquisition of multi-sonar data; the three-dimensional laser scanning system S103 is used to collect point cloud data generated by the three-dimensional laser scanner, and guides the three-dimensional laser system to collect and acquire point cloud data by calling Process in C# to execute CMD commands.

3. The multi-source data integration system according to claim 1, characterized in that: In the data integration module S2, the data receiving unit S201 directly obtains the multi-source data output by the data acquisition module S1; the data processing unit S202 performs compliance processing on the raw data obtained by the data receiving unit S201 from the data acquisition module S1, and then inputs the data to be tested into the raw data input module S3; the data classification unit S203 classifies the detection information data obtained from the detection data output module S5; The data storage unit S204 stores the detection data processed by the data classification unit S203 in a standardized manner to facilitate the integration and management of multi-source data.

4. The multi-source data integration system according to claim 1, characterized in that: The data integration module S2 is responsible for data interaction and management within the multi-source data integration system, and the back-end database of the data integration module S2 is built using MySQL; among them, the data receiving unit S201 is responsible for receiving multi-source data and adopts multiple communication protocols; when receiving multi-source data, a JSON file is attached, as well as storage description information including timestamp, device ID, and sensor reading to facilitate parsing and processing; the data processing unit S202 uses the Pandas library to process the received data, clean and convert the format, and convert the data into a dictionary format that conforms to the MySQL data table structure; the data classification unit S203 classifies the detection results output by the data detection module S4 according to the device, time, location, disease details and confidence level, and matches the key fields of the multi-source data received by the data receiving unit S201 to achieve intelligent classification; the data storage unit S204 saves and synchronizes the data to the MySQL database based on the classification results.

5. The multi-source data integration system according to claim 1, characterized in that: The data screening unit S601 is designed with an interactive screening form that covers the detection equipment, detection time and defect type. After the user fills in the form as needed, the system initiates a request to the MySQL database and immediately feeds back the screening results for display; the data query unit S602 integrates the DataQuery method and combines the FFMPEG library to efficiently retrieve qualified image and video results, and presents them intuitively in the form of an information window; the data analysis unit S603 uses the chart control in the C# language to conduct in-depth statistics and analysis on the screened defect detection data, automatically generate detailed result reports and charts, and clearly show the distribution characteristics and development trends of the defect data.

Images