Multi-source data integration system applied to intelligent recognition and detection of underwater defects of hydropower station dam

Abstract

The invention discloses a multi-source data integration system applied to intelligent recognition and detection of underwater defects of a hydropower station dam, and the system comprises a data collection module, a data integration module, an original data input module, a data detection module, a detection data output module, and a data display module. Integration of multi-source sensing information such as underwater high-definition camera shooting, image sonar, a three-dimensional laser scanner and mechanical arm contact detection is realized; through the data integration module, a multi-source information fusion processing technology is developed, an underwater station dam defect database is established, and data support is provided for underwater station dam defect detection; fusion and real-time extraction of multi-source data feature information are realized through a data detection module; and the stored data are screened, queried and analyzed through the data display module, so that the problems of non-transparent data management visualization processing and the like are solved. The whole system framework has the advantages of being clear in structure and convenient to operate, and a good and unified man-machine interaction platform is provided.

Description

Technical Field

[0001] The invention belongs to the field of intelligent recognition and detection of underwater defects in a hydropower station dam, and relates to a multi-source data integration system applied to intelligent recognition and detection of underwater defects in a hydropower station dam. Background Art

[0002] Under long-term load and environmental influences, concrete dams will inevitably have various defects including cracks, spalling, and exposed steel bars. Such defects are one of the major causes of dam danger. The deepwater part of the dam is affected by the water environment, and has the characteristics of high detection difficulty, low detection efficiency, and single detection means. The existing underwater intelligent detection work is mainly carried out by using umbilical cable underwater robots equipped with sound and light detection equipment. To address this problem, the integration of multi-source information perception and multi-source data interaction modules such as high-definition cameras, image sonar, and three-dimensional laser scanners can make more full use of existing detection resources and provide strong support for data collection, data management, and data sharing.

[0003] However, there is currently a problem that different detection equipment and different forms of detection data cannot be effectively integrated (Chen D, A Review of Detection Technologies for Underwater Cracks on Concrete Dam Surfaces, Applied Sciences, 2023.13.3564). Secondly, in terms of intelligent identification, there is a lack of corresponding underwater defect database integrating disease information (Ben H, Underwater dam crack image generation based on unsupervised image-to-image translation, Automation in Construction, 2024.105430). At the same time, in actual applications, defect inspection, defect location, and defect size measurement cannot be recorded synchronously online.

[0004] To this end, we propose a multi-source data integration system for intelligent identification and detection of underwater defects in hydropower station dams, which synchronously acquires acoustic and optical images, three-dimensional point cloud feature data, and records defect element information. Summary of the invention

[0005] In view of the lack of existing technologies, the present invention proposes a multi-source data integration system for intelligent identification and detection of underwater defects in hydropower station dams. The system integrates multi-source perception information such as underwater high-definition video, image sonar, and laser scanners, and manages equipment and detection information in a unified manner, facilitating the entire detection process. At the same time, the system establishes an underwater station dam defect database, stores and analyzes detection data, and realizes data visualization, providing an effective and feasible solution for multi-source data integration in the current field of intelligent detection of underwater station dam defects.

[0006] The technical solution of the present invention:

[0007] A multi-source data integration system for intelligent identification and detection of underwater defects in a hydropower station dam comprises a data acquisition module S1, a data integration module S2, a raw data input module S3, a data detection module S4, a detection data output module S5 and a data display module S6, wherein the data acquisition module S1 is connected to the data integration module S2, and a user obtains raw data in a dam detection process through the data acquisition module S1 and uploads the raw data to the data integration module S2 for integrated storage; the data integration module S2 is connected to the raw data input module S3, and the raw data input module S3 is connected to the data detection module S4, and performs intelligent identification on the raw data collected by the user; the data detection module S4 is connected to the detection data output module S5, and the detection data output module S5 is connected to the data integration module S2, and uploads the detected data to the data integration module S2 for classified storage; the data integration module S2 is connected to the data display module S6, and a user views the collected raw data and detection result data by accessing the data display module S6.

[0008] The data acquisition module S1 includes a robot observation system S101, a sonar data management system S102 and a three-dimensional laser scanning system S103.

[0009] The data integration module S2 includes a data receiving unit S201, a data processing unit S202, a data classification unit S205 and a data storage unit S204, the data receiving unit S201 is connected to the data processing unit S202, the data processing unit S202 is connected to the original data input module S3, the detection data output module S5 is connected to the data classification unit S203, and the data classification unit S203 is connected to the data storage unit S204.

[0010] The data detection module S4 is mainly composed of a dam underwater defect intelligent detection system S401.

[0011] The data display module S6 includes a data screening unit S601, a data query unit S602 and a data analysis unit S603. The data screening unit S601 is connected to the data query unit S602 and the data analysis unit S603 respectively.

[0012] Beneficial effects of the invention: The invention proposes a multi-source data integration system for intelligent identification and detection of underwater defects in hydropower station dams, which has the following beneficial effects: through the data acquisition module, the integration of multi-source perception information such as underwater high-definition video, image sonar, and three-dimensional laser scanner is realized, enriching the underwater station dam detection information management method; through the data integration module, a multi-source information fusion processing technology is developed, and an underwater station dam defect database is established to provide data support for underwater station dam defect detection; through the data detection module, the fusion and real-time extraction of multi-source data feature information are realized; through the data display module, the stored data is screened, queried, and analyzed to solve the problem of opaque visualization of data management. The entire system framework has the characteristics of clear structure and convenient operation, and provides a well-unified human-computer interaction platform. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 It is a schematic diagram of the overall structure of a multi-source data integration system for intelligent identification and detection of underwater defects in a hydropower station dam according to the present invention.

[0014] Figure 2 It is a structural schematic diagram of a data acquisition module in a multi-source data integration system used for intelligent identification and detection of underwater defects in a hydropower station dam according to the present invention.

[0015] Figure 3 It is a structural schematic diagram of a data integration module in a multi-source data integration system for intelligent identification and detection of underwater defects in a hydropower station dam according to the present invention.

[0016] Figure 4 It is a structural schematic diagram of a data display module in a multi-source data integration system used for intelligent identification and detection of underwater defects in a hydropower station dam according to the present invention.

[0017] Figure 5 It is a structural diagram of an underwater dam surface defect detection algorithm in a multi-source data integration system used for intelligent identification and detection of underwater defects in a hydropower station dam according to the present invention.

[0018] In the figure: S1 data acquisition module; S101 robot observation system; S102 sonar data management system; S103 three-dimensional laser scanning system; S2 data integration module; S201 data receiving unit; S202 data integration unit; S203 data storage unit; S204 data processing unit; S205 data classification unit; S206 data calling unit; S3 raw data input module; S4 data detection module; S401 underwater dam surface defect intelligent detection system; S5 detection data output module; S6 data display module; S601 data screening unit; S602 data query unit; S603 data analysis unit. DETAILED DESCRIPTION

[0019] The specific implementation of the present invention is further described below in conjunction with the accompanying drawings and technical solutions.

[0020] Example:

[0021] A multi-source data integration system for intelligent identification and detection of underwater defects in hydropower station dams, such as Figure 1 As shown, it includes a data acquisition module S1, a data integration module S2, a raw data input module S3, a data detection module S4, a detection data output module S5 and a data display module S6. The data acquisition module S1 is connected to the data integration module S2. The user obtains the raw data in the dam detection process through the data acquisition module S1 and uploads it to the data integration module S2 for integrated storage. The data integration module S2 is connected to the raw data input module S3. The raw data input module S3 is connected to the data detection module S4 to perform intelligent recognition on the raw data collected by the user. The data detection module S4 is connected to the detection data output module S5. The detection data output module S5 is connected to the data integration module S2 to upload the completed detection data to the data integration module S2 for classified storage. The data integration module S2 is connected to the data display module S6. The user views the collected raw data and detection result data by accessing the data display module S6.

[0022] Further, such as Figure 2 As shown, the data acquisition module S1 includes a robot observation system S101, a sonar data management system S102 and a three-dimensional laser scanning system S103.

[0023] Further, such as Figure 3As shown, the data integration module S2 includes a data receiving unit S201, a data processing unit S202, a data classification unit S205 and a data storage unit S204, the data receiving unit S201 is connected to the data processing unit S202, the data processing unit S202 is connected to the original data input module S3, the detection data output module S5 is connected to the data classification unit S203, and the data classification unit S203 is connected to the data storage unit S204.

[0024] Further, such as Figure 4 As shown, the data display module S6 includes a data screening unit S601, a data query unit S602 and a data analysis unit S603, and the data screening unit S601 is connected to the data query unit S602 and the data analysis unit S603 respectively.

[0025] Working principle: The present invention realizes the integration of multi-source sensing information such as underwater high-definition video, image sonar, and three-dimensional laser scanner, which can improve the efficiency of equipment use and enrich the underwater station and dam detection information management method. At the same time, an underwater station and dam defect database is established to store and analyze the detection data, provide data support for underwater station and dam defect detection, and screen, query, and analyze the stored data to solve the problems of opaque visualization of data management, and provide an effective and feasible idea for multi-source data integration in the current field of underwater station and dam defect detection.

[0026] The present invention provides a multi-source data integration system for intelligent identification and detection of underwater defects in hydropower station dams, comprising a main system and a subsystem, wherein the data integration module S2, the original data input module S3, the detection data output module S5, and the data display module S6 together constitute the main system, and the main purpose is to integrate, classify, store, filter and view the multi-source detection data of underwater structural defects in hydropower station dams, and analyze the data. The data acquisition module S1 and the data detection module S4 each constitute a subsystem, and the main purpose is to collect and detect underwater structural defects in hydropower station dams, wherein the underwater sound, light, and point cloud data acquired by the data acquisition module S1 together constitute underwater multi-source data, and the detection information obtained by the data detection module S4 covers underwater defect preprocessing, intelligent identification, positioning, and measurement functions.

[0027] The data acquisition module S1 includes a robot observation system S101, a sonar data management system S102, and a three-dimensional laser scanning system S103. Among them, the robot observation system S101 is mainly used to obtain image data captured by an underwater high-definition camera. The image data is transmitted and managed through the SDK interface provided by the C# language. The system operation interface is developed through the WPF (Windows Presentation Foundation) framework; the sonar data management system S102 is used to collect underwater acoustic data of multi-beam, single-beam and side-scan sonars. Due to the closed-source design of the sonar software, the WEBVIEW webpage nesting tool is used here, combined with the screen capture and recording method to achieve the acquisition of multi-sonar data; the three-dimensional laser scanning system S103 is used to collect the point cloud data generated by the three-dimensional laser scanner. The three-dimensional laser system is guided to collect and acquire point cloud data by calling Process in C# to execute the CMD command. The three systems together constitute the data acquisition module S1, which is used for multi-source detection data acquisition of underwater stations and dams. Through the collaborative work of these systems, the optical, acoustic and three-dimensional geometric information of the underwater environment can be fully acquired.

[0028] The data integration module S2 includes a data receiving unit S201, a data processing unit S202, a data classification unit S203 and a data storage unit S204. The data receiving unit S201 will directly obtain the multi-source data output by the data acquisition module S1; the data processing unit S202 will perform compliance processing on the original data obtained by the data receiving unit S201 from the data acquisition module S1, and then input the data to be detected into the original data input module S3; the data classification unit S203 can classify the detection information data obtained from the detection data output module S5 for easy storage; the data storage unit S204 can standardize the detection data processed by the data classification unit S203 for easy integration and management of multi-source data.

[0029] The data integration module S2 is the core of the entire system, responsible for data interaction and management within the system. The backend database of the data integration module S2 is built using MySQL. Among them, the data receiving unit S201 is responsible for receiving multi-source data (underwater high-definition video, image sonar, 3D laser scanner, etc.) from various sensors and data sources, using a variety of communication protocols, including HTTP, MQTT, etc., to ensure compatibility with different devices. When receiving multi-source data, a JSON file is attached to store description information (such as timestamp, device ID, sensor reading, etc.) for easy parsing and processing; the data processing unit S202 uses the Pandas (Python) library to process the received data, clean and convert the format, and convert the data into a dictionary format that conforms to the MySQL data table structure. In order to improve processing efficiency, this unit implements batch processing to reduce the number of interactions with the database; the data classification unit S203 classifies the detection results output by the data detection module S4 according to the equipment, time, location, disease details and confidence, and matches the key fields of the multi-source data received by the data receiving unit S201 to achieve intelligent classification; the data storage unit S204 saves and synchronizes the data to the MySQL database based on the classification results. In particular, the screen screenshots are uploaded directly, and in view of the large size of the screen recording files and the low upload efficiency, a local storage strategy is adopted to upload only the video folder path to the database.

[0030] The data detection module S4 is mainly composed of the dam underwater defect intelligent detection system S401. The data detection module S4 will obtain the data to be detected from the original data input module S3 for detection. The dam underwater defect intelligent detection system S401 mainly includes (1) detection model data set production, (2) detection interface, (3) detection model (optimized yolov8s network), and (4) detection model training.

[0031] (1) Production of detection model data set: including training data collection and annotation and training data enhancement. Data collection and annotation: First, extract the image by frame extraction from the underwater video of the dam collected on site, then manually select the typical underwater defect images, and finally manually annotate them with the help of LabelImg. Data enhancement uses the OpenCV-Python image processing library to expand the annotated data, with the aim of increasing the number of manually annotated images. The operations used for expansion include random horizontal flipping, random scaling, random acquisition, and random brightness change. Before expansion, 80% of the data set is used as the training set and 20% of the data set is used as the test set. Then, the training set and the test set are expanded respectively. The final number of expansions can be defined by oneself. The dam underwater defect data set is generally expanded to between 5 and 10 times the original data.

[0032] (2) Detection interface: The detection interface is an interactive interface provided for underwater defect detection of dams. It is developed using PySide6 and has a front-end and back-end separation design. The front-end uses an operation interface built with QTDesigner. Through the interface, the detection model weight in the local folder is selected, and the input video and detection results are displayed in real time. The detection results can also be saved and exported. The back-end uses Python to implement specific model call operations and real-time display.

[0033] (3) Detection model: specially designed for defect target detection tasks in underwater environments. Figure 5 As shown in the figure, according to the actual situation of underwater defects, the YOLOv8s network structure is optimized to fit the characteristics of underwater defect data and improve the network detection performance. There are three specific improvements:

[0034] ①Replace the original network Neck structure with Slim-Neck. This structure introduces the GSConv module, which can effectively reduce the negative impact of the module's depthwise separable convolution (DSC) on the network and make full use of the advantages of depthwise separable convolution to effectively reduce the computational cost;

[0035] ②Replace BottleNeck in the C2f module in the network backbone with ConvNeXt pure convolution lightweight high-performance structure to improve detection efficiency;

[0036] ③Replace the positioning loss function with the Focal-EIoU function, balance the contribution of high-quality and low-quality samples to the loss function, and improve the problem of limited accuracy improvement speed caused by low-quality samples.

[0037] Based on the above improvements, a network for intelligent recognition of underwater defects is proposed. The improved YOLOv8s network has higher detection efficiency and improves the recognition effect of underwater defects.

[0038] (4) Detection model training and testing: The training set of the prepared detection model training data set is sent to the detection model for training until the accuracy of the model on the test set is relatively stable. The training is stopped. At this time, the model will be used for the underwater defect detection task of the dam.

[0039] The data detection module S4 can perform real-time analysis on the input underwater image or video data, automatically identify and mark the defect category (such as cracks, erosion, voids, etc.), specific location (such as depth, azimuth coordinates) and detection confidence.

[0040] The data display module S6 includes a data screening unit S601, a data query unit S602 and a data analysis unit S603. The data screening unit S601 can screen the data to be displayed according to a series of alternative conditions; the data query unit S602 can display, preview and sort the data that meets the conditions after screening, and the user can freely view the detection fragments and results; at the same time, the data analysis unit S603 can count and analyze the detection data, highlight the key points of the detection results, and realize data management visualization. Among them, the data screening unit S601 is designed with an interactive screening form, covering options such as detection equipment (high-definition camera, image sonar, 3D laser scanner), detection time and defect type (underwater defect types such as cracks, falling blocks, peeling and exposed reinforcement). After the user fills in the form as needed, the system initiates a request to the MySQL database and immediately feedbacks the screening results for display; the data query unit S602, through the integrated DataQuery method, combines the FFMPEG library to efficiently retrieve qualified image and video results, and presents them intuitively in the form of an information window; the data analysis unit S603, through the chart control in the C# language, conducts in-depth statistics and analysis on the screened defect detection data, automatically generates detailed result reports and charts, and clearly shows the distribution characteristics and development trends of the defect data.

[0041] Specifically, after the user logs in with the username and password, the robot observation system S101, the sonar data management system S102 and the three-dimensional laser scanning system S103 in the data acquisition module S1 can be started to obtain the camera, image sonar, and three-dimensional laser scanner images respectively, and bind the specified window for image preview, and select the screenshots and recordings of the specified images. For screenshots, after the screenshots are successfully uploaded to the IIS server for storage. For recordings, due to the slow upload time, when the recording is successful, it is saved locally, but the folder directory where the video is saved will be uploaded to the IIS server, and then synchronized to the MySQL database for storage. For the collected data, the data detection module S4 can directly obtain and detect from the server, and output the results to the specified path. The data integration module S2 can obtain the detection results through this path and process and store them. Afterwards, the user can operate the data display module S6 to obtain the original data and result information that he wants to view, and can statistically analyze the results in the form of charts to intuitively display the distribution characteristics and change trends of the defect data.

[0042] The basic ideas, important features and outstanding effects of the present invention are demonstrated and described above. For ordinary technicians in this field, the general principles defined in this article can be implemented in other embodiments without departing from the spirit or concept of the present invention. Therefore, the present invention may also have various changes and improvements, which should all fall within the scope of the present invention claimed.

Claims

1. A multi-source data integration system for intelligent identification and detection of underwater defects in hydropower station dams, characterized in that: The multi-source data integration system includes a data acquisition module S1, a data integration module S2, a raw data input module S3, a data detection module S4, a detection data output module S5 and a data display module S6. The data acquisition module S1 is connected to the data integration module S2, and the user obtains the raw data in the dam detection process through the data acquisition module S1 and uploads it to the data integration module S2 for integrated storage; the data integration module S2 is connected to the raw data input module S3, and the raw data input module S3 is connected to the data detection module S4, and performs intelligent recognition on the raw data collected by the user; the data detection module S4 is connected to the detection data output module S5, and the detection data output module S5 is connected to the data integration module S2, and uploads the completed detection data to the data integration module S2 for classified storage; the data integration module S2 is connected to the data display module S6, and the user views the collected raw data and detection result data by accessing the data display module S6; The data acquisition module S1 includes a robot observation system S101, a sonar data management system S102 and a three-dimensional laser scanning system S103; The data integration module S2 includes a data receiving unit S201, a data processing unit S202, a data classification unit S205 and a data storage unit S204, wherein the data receiving unit S201 is connected to the data processing unit S202, the data processing unit S202 is connected to the original data input module S3, the detection data output module S5 is connected to the data classification unit S203, and the data classification unit S203 is connected to the data storage unit S204; The data detection module S4 is mainly composed of the dam underwater defect intelligent detection system S401; The data display module S6 includes a data screening unit S601, a data query unit S602 and a data analysis unit S603. The data screening unit S601 is connected to the data query unit S602 and the data analysis unit S603 respectively.

2. The multi-source data integration system according to claim 1, characterized in that: In the data acquisition module S1, the robot observation system S101 is mainly used to obtain image data taken by an underwater high-definition camera. The image data is transmitted and managed through the SDK interface provided by the C# language, and the operation interface of the robot observation system S101 is developed through the WPF framework; the sonar data management system S102 is used to collect underwater acoustic data of multi-beam, single-beam and side-scan sonars, and adopts the WEBVIEW webpage nesting tool, combined with the screen capture and recording method, to achieve the acquisition of multi-sonar data; the three-dimensional laser scanning system S103 is used to collect point cloud data generated by the three-dimensional laser scanner, and guides the three-dimensional laser system to collect and acquire point cloud data by calling Process in C# to execute the CMD command.

3. The multi-source data integration system according to claim 1, characterized in that: In the data integration module S2, the data receiving unit S201 directly obtains the multi-source data output by the data acquisition module S1; the data processing unit S202 performs compliance processing on the original data obtained by the data receiving unit S201 from the data acquisition module S1, and then inputs the data to be detected into the original data input module S3; the data classification unit S203 classifies the detection information data obtained from the detection data output module S5; The data storage unit S204 stores the detection data processed by the data classification unit S203 in a standardized manner, so as to facilitate the integration and management of multi-source data.

4. The multi-source data integration system according to claim 1, characterized in that: The data integration module S2 is responsible for data interaction and management within the multi-source data integration system, and the back-end database of the data integration module S2 is built using MySQL; wherein, the data receiving unit S201 is responsible for receiving multi-source data and adopts a variety of communication protocols; when receiving multi-source data, a JSON file is attached, as well as storage description information including timestamp, device ID, and sensor readings, so as to facilitate parsing and processing; the data processing unit S202 uses the Pandas library to process the received data, clean and convert the format, and convert the data into a dictionary format that conforms to the MySQL data table structure; the data classification unit S203 classifies the detection results output by the data detection module S4 according to the equipment, time, location, disease details and confidence, and matches the key fields of the multi-source data received by the data receiving unit S201 to achieve intelligent classification; the data storage unit S204 saves and synchronizes the data to the MySQL database according to the classification results.

5. The multi-source data integration system according to claim 1, characterized in that: The data detection module S4 is mainly composed of the dam underwater defect intelligent detection system S401. The data detection module S4 obtains the data to be detected from the original data input module S3 for detection, including detection model data set production, detection interface, detection model, and detection model training; (1) Production of detection model dataset: including training data collection and annotation and training data enhancement; Training data collection and labeling: First, extract frames from the dam underwater video collected on site to form a data set, then select typical underwater defect images, and finally manually label them with the help of LabelImg; The training data enhancement uses the OpenCV-Python image processing library to expand the labeled data. The expansion operations include random horizontal flipping, random scaling, random acquisition, and random brightness change. Before the expansion, 80% of the data set is used as the training set and 20% of the data set is used as the test set. Then the training set and the test set are expanded respectively. (2) Detection interface: The detection interface is an interactive interface provided for underwater defect detection of dams. It is developed using PySide6 and has a front-end and back-end separation design. The front-end uses an operation interface built with QTDesigner. Through the interface, the detection model weight in the local folder is selected, and the input video and detection results are displayed in real time. The detection results can also be saved and exported. The back-end uses Python to implement specific model call operations and real-time display. (3) Detection model Optimize the YOLOv8s network structure: ①Replace the Neck structure of the YOLOv8s network with the Slim-Neck structure, and introduce the Slim-Neck structure into the GSConv module; ②Replace BottleNeck in the C2f module in the YOLOv8s network backbone with ConvNeXt; ③Replace the positioning loss function with the Focal-EIoU function; (4) Detection model training and testing: The training set of the prepared detection model training data set is sent to the detection model for training until the accuracy of the model on the test set is relatively stable.

6. The multi-source data integration system according to claim 1, characterized in that: In the data display module S6, the data screening unit S601 screens the data required for display; the data query unit S602 displays, previews and sorts the screened data that meets the conditions, and the user can freely view the detection fragments and results; the data analysis unit S603 counts and analyzes the detection data, highlights the key points of the detection results, and realizes data management visualization.

7. The multi-source data integration system according to claim 6, characterized in that: The data screening unit S601 is designed with an interactive screening form, covering the detection equipment, detection time and defect type. After the user fills in the form as needed, the system initiates a request to the MySQL database and immediately feeds back the screening results for display; the data query unit S602, through the integrated DataQuery method, combines the FFMPEG library to efficiently retrieve qualified image and video results, and presents them intuitively in the form of an information window; the data analysis unit S603, through the chart control in the C# language, conducts in-depth statistics and analysis on the screened defect detection data, automatically generates detailed result reports and charts, and clearly shows the distribution characteristics and development trends of the defect data.

Images