Automatic inspection method and system for hydropower station dam

Abstract

The invention relates to an automatic inspection method and system for a hydropower station dam, and the method comprises the steps: simulating the wall and surface characteristics of the hydropower station dam through a preset simulation tool, so as to build a three-dimensional dam inspection model of the hydropower station dam; simulating and generating an inspection hidden danger target and inspection interference noise in the three-dimensional dam inspection model, completing automatic labeling of the inspection hidden danger target, further constructing training data of a multi-target detection tracking model, and completing training of the model; and deploying the trained multi-target detection tracking model to an embedded platform, and executing automatic inspection of the hydropower station dam through an inspection robot integrated with the embedded platform. According to the method and the device, the generation of high-quality dam inspection scene data is realized, the accurate recognition of the model on multiple types of targets is effectively improved, and the method and the device are integrated with the inspection robot, so that the reliability and the automation degree of dam inspection are effectively improved, and the problem of how to improve the inspection reliability of the hydropower station dam is solved.

Description

Technical Field

[0001] This application relates to the field of hydropower generation, and in particular to an automated inspection method and system for hydropower station dams. Background Technology

[0002] As a critical national infrastructure, the safe operation of dams directly impacts their comprehensive benefits, including flood control, water supply, and power generation. However, during long-term service, dams are susceptible to structural defects such as cracks, leakage, and settlement due to the combined effects of material aging, environmental loads, and natural disasters. Without intervention, these defects can develop into serious threats to dam safety. Therefore, effectively identifying and dynamically monitoring typical defects, and understanding their evolution and development trends, is a crucial prerequisite for implementing dam safety management and risk prevention.

[0003] Regular inspections are crucial for detecting dam defects, but traditional manual visual inspection methods are inefficient. With the development of intelligent robots and computer vision, researchers have attempted to replace manual inspections with target detection algorithms and inspection platforms. These include using robots or drones to automatically collect dam images, introducing machine vision algorithms such as YOLO and Faster R-CNN for crack and seepage detection, and recording target trajectories using methods like KCF and SORT to track defect development. While these solutions have effectively improved the efficiency of dam defect identification, they still face many challenges, such as insufficient data (it's difficult to collect datasets covering all defects from the same dam, and combining datasets from different dams limits defect identification accuracy, plus the time-consuming and labor-intensive task of dataset annotation); poor real-time performance (traditional algorithms have high computational overhead and cannot meet real-time requirements); multi-target tracking problems (complex targets are prone to missed detection or mistracking); and poor field adaptability (algorithms lack robustness in complex environments).

[0004] Currently, no effective solution has been proposed for the problem of improving the reliability of hydropower station dam inspections in related technologies. Summary of the Invention

[0005] This application provides an automated inspection method and system for hydropower station dams, which at least addresses the problem of how to improve the reliability of hydropower station dam inspections in related technologies.

[0006] In a first aspect, embodiments of this application provide an automated inspection method for a hydropower station dam, the method comprising: The wall and surface characteristics of the hydropower station dam are simulated using a preset simulation tool to build a three-dimensional dam inspection model of the hydropower station dam. In the three-dimensional dam inspection model, inspection hazard targets are simulated and generated, including crack targets, seepage point targets, foreign object targets, and settlement targets. In the three-dimensional dam inspection model, inspection interference noise is simulated and generated, and the inspection hazard targets are automatically labeled to obtain a dam defect simulation dataset. Based on the dam defect simulation dataset, training data for a multi-target detection and tracking model is constructed, and the model is trained. The trained multi-target detection and tracking model is deployed to an embedded platform, and an inspection robot integrated with the embedded platform performs automated inspections of the hydropower station dam.

[0007] In some embodiments, the simulation and generation of inspection hazard targets in the three-dimensional dam inspection model includes: The inspection targets for potential hazards include cracks, leaks, foreign objects, and settlement. In the three-dimensional dam inspection model, the environmental load and time-varying damage mechanism of concrete are simulated to generate crack targets. The crack targets include regular cracks caused by structural stress concentration and random cracks driven by material degradation. The dynamic effect of water ripples is simulated in the three-dimensional dam inspection model to generate seepage point targets; The garbage objects in the three-dimensional dam inspection model are simulated to generate foreign object targets; The regional height changes in the three-dimensional dam inspection model are simulated to generate settlement targets.

[0008] In some embodiments, the automatic labeling of the inspection hazard targets includes: The target categories, target locations, and target trajectories of the inspected potential hazards are marked to obtain marked instance data. The target categories include crack targets, leakage point targets, foreign object targets, and settlement targets.

[0009] In some embodiments, before constructing training data for a multi-target detection and tracking model based on the dam defect simulation dataset and completing the training of the model, the method includes: A multi-target detection and tracking model is constructed to detect, classify, and track potential hazards in the inspection of hydropower station dams. The multi-target detection and tracking model includes a feature extraction layer, a feature fusion layer, a feature denoising layer, a target detection and classification layer, and a target tracking layer.

[0010] In some embodiments, constructing the feature extraction layer in the multi-object detection and tracking model includes: The lightweight ShuffleNetV2 network framework is used as the backbone network of the feature extraction layer in the multi-object detection and tracking model, and the depthwise separable convolution (DSC) is used as the convolution kernel of the feature extraction layer to construct the feature extraction layer.

[0011] In some embodiments, constructing the feature fusion layer, feature denoising layer, target detection classification layer, and target tracking layer in the multi-target detection and tracking model includes: The Feature Pyramid Network (FPN) is used as the network architecture of the feature fusion layer in the multi-target detection and tracking model to fuse feature information at different resolutions to construct the feature fusion layer. Based on a preset denoising formula, the feature denoising layer in the multi-target detection and tracking model is constructed; The target center point and bounding box size are obtained by using heatmap center point detection, and the target detection and classification tasks are prioritized by using dynamic task attention mechanism (DTA) to construct the target detection and classification layer. The target tracking layer is constructed by combining the similarity of the target's appearance features with the prediction of its motion trajectory.

[0012] In some embodiments, training the model includes: Construct a multi-task loss function for the multi-target detection and tracking model, wherein the multi-task loss function includes a target heatmap detection loss L. det Bounding box regression loss L reg And target classification loss L cls ; During the training of the multi-target detection and tracking model based on the training data, the network parameters of the model are optimized using the multi-task loss function to obtain the trained multi-target detection and tracking model.

[0013] In some embodiments, the trained multi-target detection and tracking model is deployed to an embedded platform, and an inspection robot integrated with the embedded platform performs automated inspections of the hydropower station dam, including: Deploy the trained multi-object detection and tracking model to an embedded platform; By integrating the embedded platform into the inspection robot, targets in the hydropower station dam scenario are detected, classified, and tracked during automated inspection. When the target is classified as a crack, the inspection robot performs a crack detection and tracking task to record the accurate boundary and dynamic changes of the crack, and generates a corresponding first analysis report. When the target is classified as a leak point, the inspection robot performs a leak point detection and tracking task to monitor and record the dynamic diffusion behavior of the leak point in real time and generate a corresponding second analysis report. When the target is classified as a foreign object, the inspection robot performs a foreign object detection and tracking task to describe the characteristics of the foreign object, record changes in the position of the foreign object, and generate a corresponding third analysis report.

[0014] In some embodiments, the wall and surface characteristics of the hydropower station dam are simulated using preset simulation tools to build a three-dimensional dam inspection model of the hydropower station dam, including: The wall and surface characteristics of the hydropower station dam were simulated using the Blender simulation tool to build a three-dimensional dam inspection model of the hydropower station dam.

[0015] Secondly, embodiments of this application provide an automated inspection system for a hydropower station dam. The system is used to perform the method described in the first aspect above. The system includes an inspection scenario modeling module, an inspection hazard modeling module, a diversity enhancement module, an inspection model construction module, and an automated inspection module. The inspection scene modeling module is used to simulate the wall and surface characteristics of the hydropower station dam using preset simulation tools, so as to build a three-dimensional dam inspection model of the hydropower station dam. The inspection hazard modeling module is used to simulate and generate inspection hazard targets in the three-dimensional dam inspection model. The inspection hazard targets include crack targets, seepage point targets, foreign object targets, and settlement targets. The diversity enhancement module is used to simulate and generate inspection interference noise in the three-dimensional dam inspection model, and to automatically label the inspection hazard targets to obtain a dam defect simulation dataset. The inspection model construction module is used to construct training data for the multi-target detection and tracking model from the dam defect simulation dataset, and to complete the training of the model. The automated inspection module is used to deploy the trained multi-target detection and tracking model to an embedded platform, and to perform automated inspection of the hydropower station dam through an inspection robot integrated with the embedded platform.

[0016] Compared to related technologies, this application provides an automated inspection method and system for hydropower station dams. The method uses a pre-set simulation tool to simulate the wall and surface characteristics of the dam, building a three-dimensional dam inspection model. Within the three-dimensional dam inspection model, it simulates and generates potential inspection hazards and interference noise, automatically labeling the hazards to construct training data for a multi-target detection and tracking model, and completing model training. The trained multi-target detection and tracking model is deployed to an embedded platform, and an inspection robot integrated with the embedded platform performs automated inspections of the hydropower station dam. This achieves the generation of high-quality dam inspection scene data, effectively improving the model's accurate identification of multiple targets. Integration with the inspection robot further enhances the reliability and automation of dam inspections, solving the problem of improving the reliability of hydropower station dam inspections. Attached Figure Description

[0017] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a flowchart of the steps of the automated inspection method for hydropower station dams according to an embodiment of this application; Figure 2 This is a flowchart illustrating the generation of a dam defect simulation dataset according to an embodiment of this application. Figure 3 This is a schematic diagram of the operation flow of a multi-target detection and tracking model according to an embodiment of this application; Figure 4 This is a schematic diagram of the internal structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the application is described and illustrated below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0019] Obviously, the accompanying drawings described below are merely some examples or embodiments of this application. Those skilled in the art can apply this application to other similar scenarios based on these drawings without any inventive effort. Furthermore, it is understood that although the efforts made in this development process may be complex and lengthy, for those skilled in the art related to the content disclosed in this application, any changes to design, manufacturing, or production based on the technical content disclosed in this application are merely conventional technical means and should not be construed as insufficient disclosure of the content of this application.

[0020] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment that is mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0021] Unless otherwise defined, the technical or scientific terms used in this application shall have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms “a,” “an,” “an,” “the,” and similar words used in this application do not indicate quantity limitation and may indicate singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units not listed, or may include other steps or units inherent to these processes, methods, products, or devices. The terms “connected,” “linked,” “coupled,” and similar words used in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. “Multiple” used in this application refers to two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following objects are in an "or" relationship. The terms "first," "second," and "third" used in this application are merely to distinguish similar objects and do not represent a specific ordering of the objects.

[0022] This application provides an automated inspection method for hydropower station dams. Figure 1 This is a flowchart illustrating the steps of an automated inspection method for hydropower station dams according to an embodiment of this application, as follows: Figure 1As shown, the method includes the following steps: Step S102: Simulate the wall and surface characteristics of the hydropower station dam using a preset simulation tool to build a three-dimensional dam inspection model of the hydropower station dam. Specifically, step S102 involves using the Blender simulation tool to simulate the wall and surface characteristics of the hydropower station dam in order to build a three-dimensional dam inspection model.

[0023] It should be noted that, Figure 2 This is a flowchart illustrating the generation of a dam defect simulation dataset according to an embodiment of this application, as shown below. Figure 2 As shown, step S102 involves building a 3D dam scene and textures. Using the Blender tool, a 3D model highly similar to the actual dam inspection scene is created, including the dam surface geometry (smooth, bumpy textures), materials (concrete textures), and environmental background. This simulates dam walls with different aspect ratios and smooth or rough surface characteristics.

[0024] Step S104: Simulate and generate inspection hazard targets in the three-dimensional dam inspection model. The inspection hazard targets include crack targets, seepage point targets, foreign object targets, and settlement targets. Step S104 specifically includes the following steps: Step S1041: Simulate the environmental load and time-varying damage mechanism of concrete in the three-dimensional dam inspection model to generate crack targets. The crack targets include regular cracks caused by structural stress concentration and random cracks driven by material degradation. Step S1042: Simulate the dynamic effect of water ripples in the three-dimensional dam inspection model to generate seepage point targets; Step S1043: Simulate the garbage objects in the three-dimensional dam inspection model to generate foreign object targets; Step S1044: Simulate the regional height changes in the three-dimensional dam inspection model to generate settlement targets.

[0025] Step S1045: Mark the target category, target location and target trajectory of the inspection target for potential hazards to obtain marked instance data. The target categories include crack targets, leakage point targets, foreign object targets and settlement targets.

[0026] It should be noted that, as Figure 2 As shown, step S104 involves modeling the inspection hazard targets. Specifically: (1) In the three-dimensional dam inspection model (i.e. Figure 2Crack targets are simulated on the surface of the 3D model of the dam. Based on the time-varying damage mechanism of concrete and the effects of environmental loads, crack targets are generated. Crack morphologies are divided into two categories: regular cracks – caused by structural stress concentration (such as straight cracks at dam joints), and random cracks – driven by material degradation (such as bifurcated cracks caused by freeze-thaw cycles). The parameterized generation formula for crack targets is as follows:

[0027] Where a represents the width of the crack, b represents the curvature of the crack, and c represents the location of the crack initiation point. To simulate random variations in cracks due to noise interference, for example, to simulate a 2 cm wide crack starting from the sidewall of a dam, you can set a=2, b=0.1, c=0. =Random number.

[0028] (2) Simulate the dynamic effect of water ripples by parametrically controlling the water droplet diffusion radius r(t). The parameterized formula for the leakage point target is as follows:

[0029] Where r0 is the initial radius, v is the diffusion velocity, t is time, and g is the viscous gravitational factor of the water droplet.

[0030] (3) Increase target diversity by generating foreign objects (such as garbage, rocks, branches) and changes in area height (such as subsidence depressions) in the scene.

[0031] Step S106: Simulate and generate inspection interference noise in the three-dimensional dam inspection model, and automatically label the inspection hidden danger targets to obtain the dam defect simulation dataset; wherein, the inspection interference noise includes lighting environment noise and dam surface noise.

[0032] It should be noted that, as Figure 2 As shown, step S106 involves target diversification enhancement and data annotation. Specifically: (4) Simulate crack visibility under different lighting conditions, including strong midday light, diffused light on cloudy days, and low-light environments at night. Introduce surface noise such as dust and wet spots into the scene to enhance the complexity of the virtual data. Simulate crack lengths ranging from 10 cm to 50 m and widths ranging from 1 mm to 5 cm to ensure that the training data covers the crack size range in complex environments.

[0033] (5) Data annotation. For example, generate the target category (defined as "crack"), target location (bounding box or pixel mask), and trajectory (if the crack expands over time) for each frame of the image. Example annotation is as follows. Category: Crack; Bounding box: ; Trajectory: Records the center point of the crack over time The changes.

[0034] It should be further noted that steps S102 to S106 above solve the problems of scarce training data and high annotation costs. The inspection data generation module based on virtual simulation can automatically create a large amount of diverse dam defect image data with precise annotations, effectively overcoming the bottlenecks of difficulty in obtaining samples and manual annotation in the real world.

[0035] Step S108: Based on the dam defect simulation dataset, construct the training data for the multi-target detection and tracking model, and complete the training of the model; Before step S108, the method includes step S107, which constructs a multi-target detection and tracking model for detecting, classifying and tracking potential hazards of hydropower station dams. The multi-target detection and tracking model includes a feature extraction layer, a feature fusion layer, a feature denoising layer, a target detection and classification layer and a target tracking layer.

[0036] Step S107 specifically includes the following steps: Step S1071: The lightweight ShuffleNetV2 network framework is used as the backbone network of the feature extraction layer in the multi-object detection and tracking model, and the depthwise separable convolution (DSC) is used as the convolution kernel of the feature extraction layer to construct the feature extraction layer.

[0037] It should be noted that the embodiments of this application design a lightweight deep learning model that integrates detection and tracking (i.e., a multi-target detection and tracking model), which adapts to the complex detection needs of targets such as cracks, seepage points and debris in dam inspection. It is optimized by efficient algorithms to run on an embedded platform, which fully ensures the real-time performance and robustness of the robot inspection system. Figure 3 This is a schematic diagram illustrating the operation flow of the multi-target detection and tracking model according to an embodiment of this application, as shown below. Figure 3 As shown, the feature extraction layer in step S1071 uses the lightweight ShuffleNetV2 as the backbone network and combines it with a deep feature fusion module (DLAFusion) to improve the detection capability of small and medium-sized targets such as cracks and leaks through multi-scale feature extraction, while reducing model complexity to adapt to the low computing resource environment of embedded devices. Specifically: To balance the performance constraints and detection accuracy of the embedded platform, the feature extraction layer employs a lightweight backbone network, ShuffleNetV2, and depthwise separable convolution (DSC). By reducing the computational cost of traditional fully connected convolutions, complexity is effectively reduced, improving real-time processing capabilities. Through DSC optimization, the computation is decomposed into depthwise convolutions and pointwise convolutions, resulting in a total complexity of:

[0038] Where K represents the kernel size, C in and C out These represent the number of input and output channels, respectively, and W and H represent the width and height of the input feature map.

[0039] The computational complexity of traditional convolution is: In typical scenarios, the computational cost of depthwise separable convolution is about 1 / 8 that of traditional convolution, for example, when the number of input feature channels is C. in =32, Number of output channels C out =64. When the kernel size K=3, traditional convolution: 18432 W H, depthwise separable convolution: 2336 W H, the optimization ratio is With this design, the single-frame processing latency is reduced to 20ms in dam inspection tasks (image resolution of 512×512), while maintaining high-precision extraction capability of target features (such as crack width less than 0.5cm).

[0040] Step S1072: The Feature Pyramid Network (FPN) is used as the network architecture of the feature fusion layer in the multi-target detection and tracking model to fuse feature information of different resolutions to construct the feature fusion layer. It should be noted that, as Figure 3 As shown, to address the characteristics of narrow and low-contrast cracks, a Feature Pyramid Network (FPN) is used to fuse information from different resolutions. The feature output is represented as follows:

[0041] Among them, F i Let α represent the feature map of the i-th layer. i The weights are dynamic (adjusted in real time based on the complexity of the target features). When detecting regions with high clarity at crack boundaries, the dynamic weights of the high-resolution feature layer are significantly enhanced, for example... This dynamic weighting method enhances the ability to extract crack edge features, especially at crack width W. crack ≈0.5cm, length L crack The area of ​​approximately 15cm further improves the detection accuracy.

[0042] Step S1073: Based on the preset denoising formula, construct the feature denoising layer in the multi-target detection and tracking model; It should be noted that, as Figure 3 As shown, considering the impact of image noise in actual inspection scenarios such as insufficient lighting, surface stains, and external obstructions, step S1073 adopts a feature denoising optimization strategy to enhance the robustness of detailed features. The denoising formula is as follows:

[0043] Among them, F original For the original feature map, G smooth For the smoothed feature map, λ1 and λ2 represent the adjustment parameters for illumination preservation and smoothing / denoising, respectively.

[0044] Step S1074: The center point and bounding box size of the target are obtained by using heatmap center point detection, and the priority of the target detection task and the classification task are allocated by using dynamic task attention mechanism (DTA) to construct the target detection classification layer. It should be noted that, as Figure 3 As shown, the target detection classification layer locates targets such as cracks, leaks, and debris using a center point detection method, and employs a multi-task joint optimization mechanism to improve detection performance. Specifically: (1) Target center point detection. The target center point (x) is obtained by using a heatmap-based center point detection method. c ,y c Given the bounding box dimensions (w, h), the prediction formula is as follows:

[0045] Where (Δx,Δy) represents the center point offset, and (w,h) represents the predicted target width and height. In the leak detection task, when the center drift of the leak point region is less than ±0.3cm, a stable bounding box can be effectively generated to track the diffusion trend.

[0046] (2) Dynamic Task Attention Mechanism (DTA). This mechanism uses task scoring s t Dynamic calculation, allocation of task priorities for object detection and classification, and task weights w t Represented as:

[0047] For example, the task score for a crack is calculated based on its expansion rate: When the crack detection module records a crack length increasing from 19cm to 23cm (at a rate of 4cm / h), the system dynamically increases the task weight of crack detection to w. crack =0.45, so respond with priority.

[0048] Step S1075: The similarity of the target's appearance features and the prediction of its motion trajectory are combined to associate and track dynamic targets in multiple frames, so as to construct a target tracking layer.

[0049] It should be noted that, as Figure 3 As shown, the target tracking layer combines target appearance features (Re-ID feature) with motion trajectory prediction to achieve multi-frame dynamic target association, solving the problems of occlusion and loss. Specifically: (1) Target association strategy. Target association is achieved by combining improved Mahalanobis distance with feature similarity:

[0050] Where λ1 and λ2 are weights dynamically assigned according to the scene (in clutter detection, when the target moves and generates a distance D). motion With a motion offset of 2.3, tracking and matching can be achieved through weight optimization. Characterizes similarity calculation based on appearance features. Characterizes Euclidean distance based on motion trajectory.

[0051] (2) Trajectory recovery and management. For targets that have temporarily lost their vision, position updates are predicted using a Kalman filter:

[0052] Among them, X t-1 and U t Let A be the optimal state and control input from the previous time step, and B be the state transition matrix and control input matrix, respectively. t This is the correction coefficient. When the leak point is temporarily lost due to insufficient light at night, the prediction error can be stably controlled within ±0.5cm.

[0053] Step S1076: Construct the multi-task loss function for the multi-target detection and tracking model, wherein the multi-task loss function includes the target heatmap detection loss L. det Bounding box regression loss L reg And target classification loss L cls ; It should be noted that, as Figure 3As shown, the multi-task loss function is used. The multi-object detection and tracking model employs a joint loss function to optimize network performance:

[0054] Among them, L det For target heatmap detection loss, L reg For bounding box regression loss, L cls For the target classification loss, when α and β are assigned values ​​of α=2.0 and β=1.0 respectively under the dynamic adjustment of task scoring, the accuracy of crack dynamic detection is greatly improved.

[0055] Specifically, step S108 is as follows: Figure 3 As shown, during the training of the multi-target detection and tracking model constructed in step S107 based on the training data, the network parameters of the model are optimized (i.e., the network performance is optimized) using the multi-task loss function, and the trained multi-target detection and tracking model is obtained.

[0056] Step S110: Deploy the trained multi-target detection and tracking model to the embedded platform, and perform automated inspection of the hydropower station dam through the inspection robot integrated with the embedded platform.

[0057] Specifically, step S110 involves deploying the trained multi-target detection and tracking model to an embedded platform; and using an inspection robot integrated with the embedded platform to detect, classify, and track targets in the hydropower station dam scenario during automated inspection. When the target is classified as a crack, the inspection robot performs crack detection and tracking to record the accurate boundaries and dynamic changes of the crack, and generates a corresponding first analysis report. When the target is classified as a leak point, the inspection robot performs leak point detection and tracking to monitor and record the dynamic diffusion behavior of the leak point in real time, and generates a corresponding second analysis report. When the target is classified as a foreign object, the inspection robot performs foreign object detection and tracking to describe the characteristics of the foreign object and record the changes in its location, and generates a corresponding third analysis report.

[0058] It should be noted that, to adapt to the embedded hardware environment of the dam inspection robot, this embodiment deploys the trained multi-target detection and tracking model to an embedded platform (preferably an NVIDIA Jetson device). The embedded device needs to be seamlessly integrated with other modules of the inspection robot system (including cameras, sensors, motion control modules, etc.) to achieve the following functions: (1) Real-time data acquisition: Acquire images or video sequences of the dam surface through cameras.

[0059] (2) Multi-target detection and tracking: Real-time processing of multiple targets within the dam inspection area, including cracks, seepage points, garbage and other foreign objects.

[0060] (3) Inspection path planning and adjustment: Generate robot movement path based on target detection results and avoid obstacles in real time.

[0061] Furthermore, during deployment, the focus is on optimizing the computing efficiency and power consumption of embedded devices to ensure that the inspection robot can operate stably for a long time, while maintaining high real-time performance and accuracy in large-scale data processing.

[0062] After deployment, considering the complexity and diversity of the targets, the following detection and tracking tasks were designed: (1) Crack Detection and Tracking Task. The crack detection module uses parametric descriptions of crack geometry (such as length, width, and edge curvature) and combines real-time image detection technology to accurately identify the boundaries of regular and random cracks. The system records the dynamic changes of the cracks:

[0063] Where L and W are the length and width of the crack, respectively, the crack expansion process is accurately recorded through trajectory tracking.

[0064] (2) Leakage point detection and tracking task. The system simulates the diffusion characteristics of leakage points through physical modeling, such as the dynamic diffusion radius R. t Growth model:

[0065] Where v is the diffusion velocity, t is time, and R0 is the initial diffusion radius. Based on this model, the system monitors the diffusion behavior of the leak point in real time and generates a path analysis report in real time.

[0066] (3) Foreign Object Detection and Tracking Task. A classification and description strategy is adopted for the characteristics of various foreign objects: light waste (paper scraps) - dynamically changing targets affected by wind speed; medium waste (twigs or light objects) - easily disturbed by vibration but with limited range of movement; heavy foreign objects (falling rocks, tangled cables) - relatively fixed but with obvious obstruction characteristics. The foreign objects are monitored in real time, and the changes in target characteristics (such as position and shape) are dynamically generated.

[0067] The steps described in this embodiment solve the problems of scarce training data and high annotation costs: The inspection data generation module based on virtual simulation can automatically create a large amount of diverse, precisely annotated dam defect image data, effectively overcoming the bottlenecks of difficult sample acquisition and manual annotation in the real world. It enhances the real-time processing capability of the inspection system: The use of lightweight neural networks with deep separable convolution significantly reduces computational complexity, enabling complex multi-target detection and tracking algorithms to run efficiently on embedded hardware, meeting the needs of real-time robot inspection. It improves the robustness of detection and tracking in complex scenarios: Through multi-scale feature fusion, dynamic task attention mechanisms, and a comprehensive association strategy combining appearance and motion information, it significantly improves the recognition accuracy and tracking stability of various targets such as cracks, leaks, and foreign objects in complex scenarios with changes in lighting, scale, and partial occlusion. It enables the monitoring of dynamic changes in defects: The system can not only detect the existence of targets but also continuously track and record dynamic processes such as crack expansion and leakage diffusion, providing richer and more accurate data support for dam health assessment, achieving a leap from static detection to dynamic monitoring.

[0068] In some embodiments, the automated inspection method for hydropower station dams provided in the above embodiments is used to simulate multi-scenario verification under different target types and complex conditions in the dam environment. Table 1 is an example table of key test results for each scenario. Table 1

[0069] After the test is completed, a multi-dimensional performance report will be generated (e.g., Figure 3 As shown), record the following: (1) Defect identification summary: The identified defects are listed in a table according to the classification, and the search and viewing are supported by different areas of the dam. The statistics of each defect are output. (2) Performance analysis: Based on the performance data of defect identification in classification detection, such as accuracy and recall, the system automatically generates charts such as recall curves and lost target trend heatmaps to visualize the distribution of high-risk targets; (3) Problem summary: In scenarios where the system's intelligent positioning and detection performance is low, analyze the reasons for the problem, such as whether the leak points identified under low light conditions are due to the decrease in the resolution of the device's camera.

[0070] It should be noted that the steps shown in the above process or in the flowchart of the accompanying figures can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0071] This application provides an automated inspection system for a hydropower station dam. The system includes an inspection scenario modeling module, an inspection hazard modeling module, a diversity enhancement module, an inspection model construction module, and an automated inspection module. The inspection scenario modeling module is used to simulate the wall and surface characteristics of the hydropower station dam using preset simulation tools, so as to build a three-dimensional dam inspection model of the hydropower station dam. The inspection hazard modeling module is used to simulate and generate inspection hazard targets in a three-dimensional dam inspection model. The inspection hazard targets include crack targets, seepage point targets, foreign object targets, and settlement targets. The diversity enhancement module is used to simulate and generate inspection interference noise in the three-dimensional dam inspection model, and to automatically label the inspection hidden danger targets to obtain the dam defect simulation dataset. The inspection model building module is used to construct training data for a multi-target detection and tracking model based on the dam defect simulation dataset, and to complete the training of the model. The automated inspection module is used to deploy the trained multi-target detection and tracking model to the embedded platform, and to perform automated inspection of the hydropower station dam through the inspection robot integrated with the embedded platform.

[0072] It should be noted that the above modules can be functional modules or program modules, and can be implemented through software or hardware. For modules implemented through hardware, the above modules can reside in the same processor; or the above modules can be located in different processors in any combination.

[0073] This embodiment provides an electronic device including a memory and a processor. The memory stores a computer program, and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0074] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0075] Optionally, the electronic device may further include a processor, memory, network interface, display screen, and input device connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements an automated inspection method for a hydropower station dam. The display screen may be a liquid crystal display (LCD) or an e-ink display. The input device may be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the device's casing, or an external keyboard, touchpad, or mouse.

[0076] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated here.

[0077] Furthermore, in conjunction with the automated inspection method for hydropower station dams in the above embodiments, this application embodiment can provide a storage medium for implementation. This storage medium stores a computer program; when executed by a processor, the computer program implements any of the automated inspection methods for hydropower station dams in the above embodiments.

[0078] In one embodiment, Figure 4 This is a schematic diagram of the internal structure of an electronic device according to an embodiment of this application, such as... Figure 4 As shown, an electronic device is provided, which can be a server, and its internal structure diagram can be as follows. Figure 4 As shown, the electronic device includes a processor, a network interface, internal memory, and non-volatile memory connected via an internal bus. The non-volatile memory stores the operating system, computer programs, and a database. The processor provides computing and control capabilities, the network interface communicates with external terminals via a network, the internal memory provides an environment for the operating system and computer programs to run, the computer programs are executed by the processor to implement an automated inspection method for a hydroelectric dam, and the database stores data.

[0079] Those skilled in the art will understand that Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the electronic device to which the present application is applied. A specific electronic device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0080] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0081] Those skilled in the art should understand that the technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments have been described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0082] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. An automated inspection method for a hydropower station dam, characterized in that, The method includes: The wall and surface characteristics of the hydropower station dam are simulated using a preset simulation tool to build a three-dimensional dam inspection model of the hydropower station dam. In the three-dimensional dam inspection model, inspection hazard targets are simulated and generated, including crack targets, seepage point targets, foreign object targets, and settlement targets. In the three-dimensional dam inspection model, inspection interference noise is simulated and generated, and the inspection hazard targets are automatically labeled to obtain a dam defect simulation dataset. Based on the dam defect simulation dataset, training data for a multi-target detection and tracking model is constructed, and the model is trained. The trained multi-target detection and tracking model is deployed to an embedded platform, and an inspection robot integrated with the embedded platform performs automated inspections of the hydropower station dam.

2. The method according to claim 1, characterized in that, The target for hazard inspection is simulated and generated in the three-dimensional dam inspection model, including: In the three-dimensional dam inspection model, the environmental load and time-varying damage mechanism of concrete are simulated to generate crack targets. The crack targets include regular cracks caused by structural stress concentration and random cracks driven by material degradation. The dynamic effect of water ripples is simulated in the three-dimensional dam inspection model to generate seepage point targets; The garbage objects in the three-dimensional dam inspection model are simulated to generate foreign object targets; The regional height changes in the three-dimensional dam inspection model are simulated to generate settlement targets.

3. The method according to claim 2, characterized in that, The automatic labeling of the aforementioned inspection hazard targets includes: The target categories, target locations, and target trajectories of the inspected potential hazards are marked to obtain marked instance data. The target categories include crack targets, leakage point targets, foreign object targets, and settlement targets.

4. The method according to claim 1, characterized in that, Before constructing training data for a multi-target detection and tracking model based on the dam defect simulation dataset and completing the training of the model, the method includes: A multi-target detection and tracking model is constructed to detect, classify, and track potential hazards in the inspection of hydropower station dams. The multi-target detection and tracking model includes a feature extraction layer, a feature fusion layer, a feature denoising layer, a target detection and classification layer, and a target tracking layer.

5. The method according to claim 4, characterized in that, The feature extraction layer in the multi-target detection and tracking model includes: The lightweight ShuffleNetV2 network framework is used as the backbone network of the feature extraction layer in the multi-object detection and tracking model, and the depthwise separable convolution (DSC) is used as the convolution kernel of the feature extraction layer to construct the feature extraction layer.

6. The method according to claim 4, characterized in that, The feature fusion layer, feature denoising layer, target detection classification layer, and target tracking layer in the multi-target detection and tracking model include: The Feature Pyramid Network (FPN) is used as the network architecture of the feature fusion layer in the multi-object detection and tracking model to fuse feature information at different resolutions to construct the feature fusion layer. Based on a preset denoising formula, the feature denoising layer in the multi-target detection and tracking model is constructed; The target center point and bounding box size are obtained by using heatmap center point detection, and the target detection and classification tasks are prioritized by using dynamic task attention mechanism (DTA) to construct the target detection and classification layer. The target tracking layer is constructed by combining the similarity of the target's appearance features with the prediction of its motion trajectory.

7. The method according to claim 4, characterized in that, Training the model includes: Construct a multi-task loss function for the multi-target detection and tracking model, wherein the multi-task loss function includes a target heatmap detection loss L. det Bounding box regression loss L reg And target classification loss L cls ; During the training of the multi-target detection and tracking model based on the training data, the network parameters of the model are optimized using the multi-task loss function to obtain the trained multi-target detection and tracking model.

8. The method according to claim 1, characterized in that, The trained multi-target detection and tracking model is deployed to an embedded platform, and an inspection robot integrated with the embedded platform performs automated inspections of the hydropower station dam, including: Deploy the trained multi-object detection and tracking model to an embedded platform; By integrating the embedded platform into the inspection robot, targets in the hydropower station dam scenario are detected, classified, and tracked during automated inspections. When the target is classified as a crack, the inspection robot performs a crack detection and tracking task to record the accurate boundary and dynamic changes of the crack, and generates a corresponding first analysis report. When the target is classified as a leak point, the inspection robot performs a leak point detection and tracking task to monitor and record the dynamic diffusion behavior of the leak point in real time and generate a corresponding second analysis report. When the target is classified as a foreign object, the inspection robot performs a foreign object detection and tracking task to describe the characteristics of the foreign object, record changes in the position of the foreign object, and generate a corresponding third analysis report.

9. The method according to claim 1, characterized in that, The wall and surface characteristics of the hydropower station dam are simulated using preset simulation tools to build a three-dimensional dam inspection model of the hydropower station dam, including: The wall and surface characteristics of the hydropower station dam were simulated using the Blender simulation tool to build a three-dimensional dam inspection model of the hydropower station dam.

10. An automated inspection system for a hydropower station dam, characterized in that, The system is used to perform the method according to any one of claims 1 to 9, and the system includes an inspection scenario modeling module, an inspection hazard modeling module, a diversity enhancement module, an inspection model construction module, and an automated inspection module; The inspection scene modeling module is used to simulate the wall and surface characteristics of the hydropower station dam using preset simulation tools, so as to build a three-dimensional dam inspection model of the hydropower station dam. The inspection hazard modeling module is used to simulate and generate inspection hazard targets in the three-dimensional dam inspection model. The inspection hazard targets include crack targets, seepage point targets, foreign object targets, and settlement targets. The diversity enhancement module is used to simulate and generate inspection interference noise in the three-dimensional dam inspection model, and to automatically label the inspection hazard targets to obtain a dam defect simulation dataset. The inspection model construction module is used to construct training data for the multi-target detection and tracking model from the dam defect simulation dataset, and to complete the training of the model. The automated inspection module is used to deploy the trained multi-target detection and tracking model to an embedded platform, and to perform automated inspection of the hydropower station dam through an inspection robot integrated with the embedded platform.

Images