Reservoir dam safety monitoring method based on multi-source information

Abstract

The invention discloses a reservoir dam safety monitoring method based on multi-source information, and the method comprises the steps: carrying out the high slope inspection through employing an unmanned aerial vehicle A and an unmanned aerial vehicle B, carrying out the high-definition camera and an infrared thermal imager, calculating the pitching angle between the unmanned aerial vehicles through GPS synchronization, image recognition and other technologies, and correcting the fusion image recognition potential safety hazards; path planning is carried out by adopting a cattle farming method, and the problems of unmanned aerial vehicle inspection path independence and task coverage are solved; the pixel value of the unmanned aerial vehicle B in the image A is compared with the normal horizontal state image, the pitching angle lambda is calculated and used for correcting the image A and the image B, and the potential hidden danger of the reservoir dam is fused and recognized; the method is low in cost and easy to maintain, a comprehensive inspection result is combined with a picture recognition deep learning algorithm by combining a manual inspection verification result, long-term tracking of a potential safety hazard part is established, and powerful support is provided for reservoir dam monitoring.

Description

Technical Field

[0001] The invention relates to the field of safety monitoring of hydropower and water conservancy projects, and in particular to a reservoir dam safety monitoring method based on multi-source information. Background Art

[0002] The stability of high slopes of reservoir dams has always been an important factor affecting the safety of dams. Especially as the service time of reservoir dams increases and when encountering accidental geological disasters, most of the slopes of reservoir dams are located in steep, uninhabited areas covered with dense vegetation. It is difficult for relevant operation and management personnel of hydropower stations to reach the site for inspection. In addition, due to the complex terrain, traditional manual inspection methods have the disadvantages of high operating difficulty, low measurement accuracy, long operation cycle, and high safety risks, and can no longer meet the operation needs of power stations.

[0003] Drone inspection technology, with its high quality, high efficiency, high precision, mobility and real-time characteristics, has been gradually applied to the inspection of high slopes of reservoir dams in hydropower and water conservancy projects. It is not restricted by geographical conditions and can set adaptive inspection routes according to the complexity and coverage of the slopes of reservoir dams. However, in actual application, single drone inspection technology still faces limitations such as limited endurance, lack of observation focus, and high requirements for image stitching technology. Summary of the invention

[0004] This application provides a reservoir dam safety monitoring method based on multi-source information, combines drone intelligent inspection and fixed-point camera monitoring to build a panoramic model map, aiming to achieve comprehensive, efficient and accurate monitoring of high slopes, and provide strong guarantees for the safe operation of reservoir dams.

[0005] The present application provides a reservoir dam safety monitoring method based on multi-source information, including: S101, setting a baseline reference line and multi-source information monitoring equipment in the reservoir area; S102, UAV A and UAV B are equipped with high-definition cameras and infrared thermal imagers to collect image and video data of the high slope according to the planned route; S103, UAV A and UAV B respectively perform flight inspection work to collect multi-angle image data of the reservoir slope; S104, synchronizing the three-dimensional relative coordinates of UAV A and UAV B in real time through GPS; S105, identifying drone B in image A, and calculating angle information when drone A photographs drone B; S106, comparing the pixel value of drone B in the image A with the pixel value of drone B in the normal horizontal state image, and calculating the pitch angle λ between drone B and the reference line through the pixel difference; S107, after correcting and fusing the A and B images using the pitch angle λ to identify the potential safety hazards of the reservoir dam, manual inspections are performed to further confirm the safety hazards of the dam.

[0006] Preferably, the S101 includes: the baseline reference line is a virtual line that completely surrounds the reservoir along the boundary of the reservoir; monitoring points are set on the reservoir dam and multi-source information monitoring equipment such as displacement sensors, stress sensors and water level sensors are deployed to collect real-time monitoring data on safety hazards such as crack expansion, abnormal seepage and increased displacement of the dam.

[0007] Preferably, S103 includes: UAV A flies above UAV B according to a preset flight path and baseline reference line; the picture taken by UAV A is called A picture, and the picture taken by UAV B is called B picture; A picture contains the image of UAV B and the slope information from the perspective of UAV A itself; B picture contains the image of UAV A and the slope information from the perspective of UAV B itself.

[0008] Preferably, the angle information in S105 includes a horizontal angle and a vertical angle, which are defined as α and β respectively.

[0009] Preferably, S106 includes: comparing the pixel values ​​of drone B in the A figure with the pixel values ​​of drone B in the normal horizontal state figure, and calculating the pitch angle λ between drone B and the baseline reference line through pixel difference; obtaining an image of drone B in the normal horizontal state as a reference image; comparing the pixel values ​​of the drone B area in the A figure with the corresponding area in the normal horizontal state figure, and analyzing the difference in pixel values; and inferring the pitch angle λ of drone B relative to the baseline reference line by combining the camera viewing angle, the physical characteristics of drone B, and the data of the multi-source information monitoring equipment.

[0010] Preferably, the S107 includes: S201, extracting historical inspection data from the UAV flight record, and using an image fusion algorithm to fuse historical A and B images to establish safety hazard sample information; S202, combining the historical pitch angle λ and the historical panoramic image to establish a panoramic model of the reservoir dam inspected by the drone in the historical time; S203, using the currently collected pitch angle λ and the panoramic image to establish a current panoramic model of the reservoir dam inspected by the drone; S204, using historical inspection data and annotated safety hazard samples to train a deep learning model to identify safety hazards; S205, applying the trained potential safety hazard identification model to the current panoramic model to identify potential safety hazards; S206: Analyze the prominent features of the identified safety hazards to determine the nature, extent and scope of impact of the hazards.

[0011] Preferably, the S202 includes: half a year of historical time as a cycle, and four cycles as a group of historical time; within the four historical cycles, collecting the flight data of the UAV, including the change record of the pitch angle λ; obtaining a panoramic image at the time point corresponding to the flight data; wherein the panoramic image is formed by fusing the A image and the B image taken by the UAV at different perspectives through the image fusion technology; using the three-dimensional modeling technology OpenGL tool, combining the pitch angle λ and the panoramic image to construct a panoramic model of the reservoir dam; using the three-dimensional modeling tool to fuse these data into the model to form a panoramic model that can reflect the dynamic changes of the state of the reservoir dam at different time points.

[0012] Preferably, the S203 includes: S301, classifying height information of the dam body from the panoramic model according to the spatial layout in the panoramic model; S302, adding the collected multi-source information features to the classified panoramic model to form a panoramic monitoring model of the dam status; S303, collecting panoramic monitoring models in historical time and training boundary point safety hazard prediction models; S304, predicting the safety hazards of the dam body at the boundary points within half a cycle, and displaying the safety hazard prediction situation in a dynamic three-dimensional image within half a cycle.

[0013] Preferably, the S301, grading the height information of the dam body, includes: dividing the dam body into three levels: bottom, middle and top according to the height information of the dam body; if the monitoring equipment is installed below the reservoir water level, the bottom includes the dam foundation and the dam toe part, the middle part is the main load-bearing part of the dam body, and the top includes the dam crest and the wave-breaking wall; if the monitoring equipment is installed above the reservoir water level, the bottom is the part above the reservoir water level line.

[0014] Preferably, the S301 further includes: S401, partitioning the graded panoramic model, and marking the boundary and position of each partition according to the determined partition standard; S402, determining whether there is a focus area according to the partitioned panoramic model; S403, determining whether it is necessary to perform separate zone inspections on drone A and drone B and collect separate zone panoramic models according to the status of the key focus area; S404, training a safety hazard prediction model for partition boundary points according to the partition panoramic model characteristics.

[0015] One or more technical solutions provided in this application have at least the following technical effects or advantages: By setting the baseline reference line, a clear flight path and shooting angle reference are provided for the drone inspection, ensuring the standardization and consistency of the inspection work. At the same time, combined with multi-source information monitoring equipment, the safety monitoring data of the dam is collected in real time, providing rich data support for subsequent data analysis and hidden danger identification. The collaborative work of drone A and drone B realizes the multi-angle and all-round image data collection of the reservoir slope. The three-dimensional relative coordinates of the drones are synchronized in real time by GPS, and the angle information between drones is calculated, which further improves the accuracy and availability of the image data.

[0016] By comparing the pixel values ​​of drone B in image A with the normal horizontal state image, the pitch angle between drone B and the reference line is calculated, effectively eliminating the image distortion caused by the pitch angle of the drone, and improving the quality and effect of image fusion. Fusion of the corrected images A and B provides more comprehensive field of view information, which helps to more accurately identify potential safety hazards. Combined with the data of multi-source information monitoring equipment, comprehensive evaluation and analysis of identified safety hazards can more accurately determine their severity and possible impact range, providing strong support for timely and effective emergency response measures.

[0017] By integrating drone flight records, image fusion algorithms, 3D modeling technology, and deep learning models, a comprehensive and dynamic inspection of reservoir dams has been achieved. This solution not only builds a dynamic panoramic model that can reflect the state of the dam at different time points, but also realizes the automatic identification of potential safety hazards by training deep learning models. In addition, the identified safety hazards are analyzed in detail to determine their nature, degree, and scope of impact, providing a scientific basis for the safety management of reservoir dams. This method improves the efficiency and accuracy of inspections, reduces the cost and risk of manual inspections, and has significant technical advantages and practical value.

[0018] By grading the panoramic model and combining the installation locations of multi-source information monitoring equipment, a panoramic monitoring model of the dam status was formed, which improved the accuracy and comprehensiveness of monitoring. The convolutional neural network deep learning model was trained using historical data, and a safety hazard prediction model was established, which can accurately predict the safety hazards that may arise in the dam in the future. At the same time, the safety hazard identification model was integrated with the prediction model to form a comprehensive dam safety management system, which improved management efficiency and accuracy and provided all-round protection for the safe operation of the dam.

[0019] By making detailed zoning on the graded panoramic model and clarifying the zoning standards such as geographical location, stress concentration area and historical hidden danger points, the boundaries and positions of each zone can be accurately marked. This not only improves the precision and practicality of the panoramic model, but also provides a solid foundation for the subsequent safety hazard prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 The present invention is a flowchart of a reservoir dam safety monitoring method based on multi-source information according to an embodiment of the present invention. DETAILED DESCRIPTION

[0021] To facilitate the understanding of the present invention, the present application will be described more comprehensively below with reference to the relevant drawings; the drawings show preferred embodiments of the present invention, but the present invention can be implemented in many different forms and is not limited to the embodiments described herein; on the contrary, the purpose of providing these embodiments is to enable a more thorough and comprehensive understanding of the disclosed content of the present invention.

[0022] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by technicians in the technical field to which the present invention belongs; the terms used in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention; the term "and / or" used herein includes any and all combinations of one or more related listed items. Embodiment 1

[0023] like Figure 1 As shown, a reservoir dam safety monitoring method based on multi-source information includes the following steps: S101, setting up a baseline reference line and multi-source information monitoring equipment in the reservoir area.

[0024] Among them, the baseline reference line is a virtual line that completely surrounds the reservoir along the boundary of the reservoir. It is used as a reference for drone inspections to help determine the flight path, shooting angle, and position of objects in the image. The specific location should be set in combination with the water level change range, measurement accuracy, and surrounding environment, and its use should be clarified in subsequent steps. Monitoring points are set on the reservoir dam and multi-source information monitoring equipment such as displacement sensors, stress sensors, and water level sensors are deployed to collect real-time monitoring data on safety hazards such as crack expansion, abnormal seepage, and increased displacement of the dam.

[0025] S102, UAV A and UAV B are equipped with high-definition cameras and infrared thermal imagers to collect image and video data of the high slope according to the planned route.

[0026] Among them, the ox-plowing method is used for path planning, and the grid technology is used to discretize the slope area of ​​the reservoir dam that needs to be inspected into regional grids; according to the number of drones, their respective endurance, execution mission and other information, as well as the layout of multi-source information monitoring equipment, the grid inspection paths for each area are planned to ensure that the paths of the drones are independent of each other and the tasks are fully covered during automatic inspections.

[0027] S103, UAV A and UAV B respectively perform flight inspections to collect multi-angle image data of the reservoir bank slope.

[0028] Among them, drone A flies above drone B according to the preset flight path and baseline reference line to ensure that all areas that need to be inspected are covered; the picture taken by drone A is called A picture, and the picture taken by drone B is called B picture; A picture contains the image of drone B and the slope information from the perspective of drone A itself; B picture contains the image of drone A and the slope information from the perspective of drone B itself. Inspections include daily inspections and special inspections, and the specific frequency is set according to demand; the method of mobile drone hangar is adopted, and the hangar is deployed on the patrol ship in the warehouse area to meet the continuous operation requirements of large-scale slope inspections; at the same time, multi-source information such as GPS data and sensor data during drone inspections is recorded.

[0029] S104, synchronizing the three-dimensional relative coordinates of UAV A and UAV B in real time through GPS.

[0030] For example, the GPS positioning data of drone A is (x 1 ,y 1 ,z 1 ), the GPS positioning data of drone B is (x 2 ,y 2 ,z 2 ), then the three-dimensional relative coordinates between them can be calculated by the following formula: Δx=x 2 -x 1 , Δy=y 2 -y 1 , Δz=z 2 -z 1 , so the three-dimensional relative coordinates of drone B relative to drone A are (Δx, Δy, Δz).

[0031] S105, identifying drone B in image A, and calculating the angle information when drone A photographs drone B.

[0032] The angle information includes the horizontal angle and the vertical angle, which are defined as α and β respectively.

[0033] Specifically, identify drone B in image A, and calculate the horizontal angle α and vertical angle β when drone A photographs drone B; use edge monitoring, shape matching, color recognition or deep learning algorithms to accurately identify drone B in image A; use the position information of drone B in the image and the flight direction of drone A to calculate the horizontal angle α through geometric relationships; use the vertical position of drone B in the image (relative to the center of the image) and the camera viewing angle parameters to calculate the vertical angle β; verify the algorithm, and perform calibration tests using images with known angles to ensure the accuracy of the calculation.

[0034] S106, comparing the pixel value of drone B in the image A with the pixel value of drone B in the normal horizontal state image, and calculating the pitch angle λ between drone B and the reference line through the pixel difference.

[0035] The pitch angle λ is the deflection angle relative to the reference line.

[0036] Specifically, the pixel values ​​of UAV B in Figure A are compared with the pixel values ​​of UAV B in the normal horizontal state image, and the pitch angle λ between UAV B and the baseline reference line is calculated through the pixel difference; the image of UAV B in the normal horizontal state is obtained as the reference image; the pixel values ​​of the UAV B area in Figure A are compared with the corresponding area in the normal horizontal state image, and the difference in pixel values ​​is analyzed; the pitch angle λ of UAV B relative to the baseline reference line is inferred by combining the camera perspective, the physical characteristics of UAV B and the data of multi-source information monitoring equipment.

[0037] Among them, obtaining the image of drone B in a normal horizontal state as a reference image specifically includes: selecting an environment where drone B can fly stably and the lighting conditions are good for shooting; adjusting drone B to a normal horizontal state to ensure that its flight posture is stable and the camera angle is correct; shooting at a position where the full view of drone B can be clearly captured, ensuring that there are no obstructions between the shooting position and drone B to obtain a clear image; recording the lighting conditions, camera settings (such as focal length, exposure time, etc.) and status information of drone B during shooting; and selecting the clearest image that best conforms to the normal horizontal state from the multiple images taken as the reference image.

[0038] S107, after correcting and fusing the A and B images using the pitch angle λ to identify the potential safety hazards of the reservoir dam, manual inspections are performed to further confirm the safety hazards of the dam.

[0039] Specifically, the calculated pitch angle λ is used to correct images A and B to eliminate image distortion caused by the pitch angle of the drone. Appropriate image fusion technology (such as pixel-level fusion, feature-level fusion or decision-level fusion) is selected to fuse the corrected images A and B to provide more comprehensive field of view information. In the fused image, edge monitoring, abnormal behavior recognition and other algorithms and technologies are used to identify potential safety hazards, including monitoring obstacles, identifying abnormal behaviors, analyzing environmental changes, etc. The data of multi-source information monitoring equipment (such as displacement, stress, water level, etc.) are combined to comprehensively evaluate and analyze the identified safety hazards to determine their severity and possible impact range.

[0040] The technical solutions in the above embodiments of the present application have at least the following technical effects or advantages: By setting the baseline reference line, a clear flight path and shooting angle reference are provided for the drone inspection, ensuring the standardization and consistency of the inspection work. At the same time, combined with multi-source information monitoring equipment, the safety monitoring data of the dam is collected in real time, providing rich data support for subsequent data analysis and hidden danger identification. The collaborative work of drone A and drone B realizes the multi-angle and all-round image data collection of the reservoir slope. The three-dimensional relative coordinates of the drones are synchronized in real time by GPS, and the angle information between drones is calculated, which further improves the accuracy and availability of the image data.

[0041] By comparing the pixel values ​​of drone B in image A with the normal horizontal state image, the pitch angle between drone B and the reference line is calculated, effectively eliminating the image distortion caused by the pitch angle of the drone, and improving the quality and effect of image fusion. Fusion of the corrected images A and B provides more comprehensive field of view information, which helps to more accurately identify potential safety hazards. Combined with the data of multi-source information monitoring equipment, comprehensive evaluation and analysis of identified safety hazards can more accurately determine their severity and possible impact range, providing strong support for timely and effective emergency response measures. Embodiment 2

[0042] The above-mentioned first embodiment realizes comprehensive and accurate monitoring of reservoir dams by integrating drone inspections and multi-source information monitoring, improves monitoring efficiency and accuracy, and provides a strong guarantee for the safe operation of reservoir dams. In order to more intuitively monitor the status of reservoir dams, improve the accuracy of pitch angle calculation and image fusion, and enhance the ability to identify safety hazards, step S107 of the first embodiment is further improved, specifically: S201, extract historical inspection data from the UAV flight record, and use the image fusion algorithm to fuse the historical A image and B image to establish safety hazard sample information.

[0043] The historical inspection data includes the pitch angle λ change data of the drone at different flight altitudes and speeds with timestamps. The safety hazard sample information includes the panoramic image fused from image A and image B, which contains the key areas of safety hazards marked with prominent marks and the inspection path.

[0044] Specifically, historical images A and B are obtained from the drone inspection data, and pre-processing operations such as image enhancement and denoising are performed on images A and B. Images A and B are fused using an image fusion algorithm (image stitching) to generate a panoramic image containing the information of images A and B; key areas (cracks, landslides) and inspection paths are marked in the panoramic image, and the marked information can be used for subsequent identification and analysis of safety hazards.

[0045] S202, combining the historical pitch angle λ and the historical panoramic image to establish a panoramic model of the reservoir dam inspected by the drone in the historical time.

[0046] Among them, half a year of historical time is a cycle, and four cycles constitute a group of historical time.

[0047] Specifically, in four historical cycles (half a year for each cycle, two years in total), the flight data of the drone was collected, including the change records of the pitch angle λ; at the same time, the panoramic images corresponding to the time points of these flight data were obtained; these panoramic images were formed by fusion of the A and B images taken by the drone at different perspectives through image fusion technology, which can fully show the surface conditions of the reservoir dam; the three-dimensional modeling technology OpenGL tool was used to combine the pitch angle λ and the panoramic image to build a panoramic model of the reservoir dam; in this process, the pitch angle λ, as an important input parameter for model construction, determines the perspective and flight posture of the drone during the inspection process, and thus affects the spatial form and detail performance of the model. The panoramic image provides the model with the required texture and detail information, making the model more realistic and accurate; the data of the four cycles were integrated in time series, the pitch angle λ and the panoramic image were matched according to the timestamp, and arranged in chronological order; finally, the three-dimensional modeling tool was used to fuse these data into the model to form a dynamic panoramic model that can reflect the state of the reservoir dam at different time points.

[0048] S203, using the currently collected pitch angle λ and panoramic image to establish a current panoramic model of the reservoir dam inspected by the drone.

[0049] S204: Use historical inspection data and annotated safety hazard samples to train a deep learning model to identify safety hazards.

[0050] Specifically, we first organize historical inspection data, including various sensor data (pitch angle λ, flight altitude, speed) recorded by drones during the inspection process and panoramic images at corresponding time points; at the same time, we collect and annotate a batch of safety hazard samples, which include images and descriptions of cracks and landslides that may appear on reservoir dams; we select a convolutional neural network (CNN) deep learning model for training; during the training process, we use historical inspection data as input and annotated safety hazard samples as output, and by continuously adjusting the parameters and structure of the model, the model can gradually learn to identify safety hazards from the input data.

[0051] S205, applying the trained potential safety hazard identification model to the current panoramic model to identify potential safety hazards.

[0052] Specifically, load the trained safety hazard identification model, divide the current panoramic model into several small areas or pixel blocks, preprocess (scaling, normalization) each small area or pixel block, input the preprocessed small area or pixel block into the safety hazard identification model, the model outputs the safety hazard probability of each small area or pixel block, and integrates the safety hazard probabilities of each small area or pixel block into a safety hazard distribution map of the panoramic model.

[0053] S206: Analyze the prominent features of the identified safety hazards to determine the nature, extent and scope of impact of the hazards.

[0054] Specifically, all safety hazard information output by the safety hazard identification model is first collected, including the location, type, probability, etc. of the hazard; then, this information is analyzed and processed to determine the nature, degree and scope of impact of the hazard; for the nature analysis of the hazard, it can be determined what type of safety hazard it belongs to based on the type and characteristics of the hazard; for the degree assessment of the hazard, the hazard can be graded (such as minor, moderate, severe) based on quantitative indicators such as the size, shape, and depth of the hazard; for the determination of the scope of impact of the hazard, the degree of impact of the hazard on the overall safety of the reservoir dam, as well as the area to which the hazard may spread or affect, can be analyzed.

[0055] It should be noted that in the analysis of identified safety hazards, some expert systems or rule bases are needed to assist in the analysis and assessment of hazards. Historical data has a certain reference value, but the dam status is dynamically changing, and it is necessary to set and update the thresholds of various quantitative indicators in a timely manner to ensure the accuracy and reliability of hazard analysis and assessment.

[0056] The technical solutions in the above embodiments of the present application have at least the following technical effects or advantages: By integrating drone flight records, image fusion algorithms, 3D modeling technology, and deep learning models, a comprehensive and dynamic inspection of reservoir dams has been achieved. This solution not only builds a dynamic panoramic model that can reflect the state of the dam at different time points, but also realizes the automatic identification of potential safety hazards by training deep learning models. In addition, the identified safety hazards are analyzed in detail to determine their nature, degree, and scope of impact, providing a scientific basis for the safety management of reservoir dams. This method improves the efficiency and accuracy of inspections, reduces the cost and risk of manual inspections, and has significant technical advantages and practical value. Embodiment 3

[0057] The above-mentioned second embodiment realizes the dynamic inspection of reservoir dams and the automatic identification and analysis of safety hazards by integrating drone technology, image fusion, three-dimensional modeling and deep learning, improves the inspection efficiency, reduces cost risks, and provides strong support for the safety management of reservoir dams. Because the state of the dam changes dynamically, if the dam is to be monitored in real time, the panoramic model is graded according to the panoramic image monitoring to predict the boundary safety hazards of the dam. Step S203 of the second embodiment is further improved as follows: S301, classifying height information of the dam body from the panoramic model according to the spatial layout in the panoramic model.

[0058] Among them, according to the height information of the dam body, the dam body can be divided into three levels: bottom, middle and top; if the monitoring equipment is installed below the reservoir water level, the bottom includes the dam foundation and the dam toe, the middle is the main load-bearing part of the dam body, and the top includes the dam crest and the wave-breaking wall; if the monitoring equipment is installed above the reservoir water level, the bottom is the part above the reservoir water level line, and confirmation is carried out step by step based on the structural characteristics of the dam body, the stress conditions and the installation position of the multi-source information monitoring equipment.

[0059] S302, adding the collected multi-source information features to the classified panoramic model to form a panoramic monitoring model of the dam status.

[0060] Specifically, the collected multi-source information is preprocessed and feature extracted to extract characteristic parameters that can reflect the dam status. The extracted characteristic parameters are fused with the graded panoramic model to form a panoramic monitoring model of the dam status.

[0061] S303, collecting panoramic monitoring models in historical time and training boundary point safety hazard prediction models.

[0062] Among them, the panoramic monitoring model of historical time is cleaned, denoised and normalized, and the convolutional neural network (CNN) deep learning model is selected for training. The prediction model is trained in chronological order to establish a safety hazard prediction model. The boundary safety hazard is the state of the dam when the dam is about to have a safety hazard. The safety hazard is predicted based on the dam state to predict the future safety hazard situation.

[0063] S304, predicting the safety hazards of the dam body at the boundary points within half a cycle, and displaying the safety hazard prediction situation in a dynamic three-dimensional image within half a cycle.

[0064] Specifically, the trained safety hazard prediction model is used to predict the characteristics of safety hazards in the dam body within half a cycle (within the next three months). The prediction results include the type, location, severity, etc. of the safety hazards. The prediction results will be displayed in the form of dynamic three-dimensional images through three-dimensional visualization software, so that managers can intuitively understand the safety status of the dam in the future.

[0065] It should be noted that the safety hazard identification model (such as the deep learning model in Example 2) is integrated with the safety hazard prediction model to form a comprehensive dam safety management system. The identification model is responsible for real-time monitoring of the current state of the dam and identifying potential safety hazards; the prediction model is responsible for predicting the safety state of the dam in the future; the two models can work together to improve the efficiency and accuracy of dam safety management. For example, when the identification model finds that the dam is in an abnormal state, it can trigger the prediction model for more in-depth analysis and prediction; at the same time, the results of the prediction model can also provide reference and verification for the identification model.

[0066] The technical solutions in the above embodiments of the present application have at least the following technical effects or advantages: By grading the panoramic model and combining the installation locations of multi-source information monitoring equipment, a panoramic monitoring model of the dam status was formed, which improved the accuracy and comprehensiveness of monitoring. The convolutional neural network deep learning model was trained using historical data, and a safety hazard prediction model was established, which can accurately predict the safety hazards that may arise in the dam in the future. At the same time, the safety hazard identification model was integrated with the prediction model to form a comprehensive dam safety management system, which improved management efficiency and accuracy and provided all-round protection for the safe operation of the dam. Embodiment 4

[0067] The above-mentioned third embodiment realizes real-time monitoring of dam status and prediction of potential safety hazards by integrating multiple technologies. By hierarchical panoramic models, integrating multi-source information, training deep learning prediction models, and integrating recognition and prediction models, a comprehensive dam safety management system is formed. In order to further improve the accuracy of dam monitoring, further improvements are made on the basis of step S301 of the third embodiment, specifically: S401, partitioning is performed on the graded panoramic model, and the boundary and position of each partition are marked according to the determined partition standard.

[0068] Among them, the boundaries and locations of each zone are marked according to the zoning standards of clear geographical locations, concentrated stress areas and historical hidden danger points.

[0069] S402: Determine whether there is a focus area according to the partitioned panoramic model.

[0070] Among them, the key focus areas can be areas with existing boundary hazards, concentrated stress area divisions, and historical hazard point divisions.

[0071] S403: Determine whether it is necessary to perform separate zone inspections on drone A and drone B and collect separate zone panoramic models based on the status of the key focus area.

[0072] Among them, based on the safety hazard level of the key focus area, evaluate whether a separate partition inspection is needed; if necessary, clarify the specific requirements of the accurate panoramic model under the current state, conduct a separate partition inspection, and collect a separate partition panoramic model, and add the newly collected model to and overwrite the original partition panoramic model.

[0073] S404, training a safety hazard prediction model for partition boundary points according to the partition panoramic model characteristics.

[0074] It should be noted that the model training involved in the solutions of the present invention are all existing technologies and will not be elaborated in this text.

[0075] The technical solutions in the above embodiments of the present application have at least the following technical effects or advantages: By making detailed zoning on the graded panoramic model and clarifying the zoning standards such as geographical location, stress concentration area and historical hidden danger points, the boundaries and positions of each zone can be accurately marked. This not only improves the precision and practicality of the panoramic model, but also provides a solid foundation for the subsequent safety hazard prediction.

[0076] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A reservoir dam safety monitoring method based on multi-source information, characterized in that: include: S101, setting a baseline reference line and multi-source information monitoring equipment in the reservoir area; S102, UAV A and UAV B are equipped with high-definition cameras and infrared thermal imagers to collect image and video data of the high slope according to the planned route; S103, UAV A and UAV B respectively perform flight inspection work to collect multi-angle image data of the reservoir slope; S104, synchronizing the three-dimensional relative coordinates of UAV A and UAV B in real time through GPS; S105, identifying drone B in image A, and calculating angle information when drone A photographs drone B; S106, comparing the pixel value of drone B in the image A with the pixel value of drone B in the normal horizontal state image, and calculating the pitch angle λ between drone B and the reference line through the pixel difference; S107, after correcting and fusing the A and B images using the pitch angle λ to identify the potential safety hazards of the reservoir dam, manual inspections are performed to further confirm the safety hazards of the dam.

2. The reservoir dam safety monitoring method based on multi-source information according to claim 1, characterized in that: The S101 includes: the baseline reference line is a virtual line that completely surrounds the reservoir along the boundary of the reservoir; monitoring points are set on the reservoir dam and multi-source information monitoring equipment such as displacement sensors, stress sensors and water level sensors are deployed to collect real-time monitoring data on safety hazards such as crack expansion, seepage anomalies and increased displacement of the dam.

3. A reservoir dam safety monitoring method based on multi-source information as claimed in claim 1, characterized in that: The S103 includes: UAV A flies above UAV B according to a preset flight path and baseline reference line; the picture taken by UAV A is called A picture, and the picture taken by UAV B is called B picture; A picture contains the image of UAV B and the slope information from the perspective of UAV A itself; B picture contains the image of UAV A and the slope information from the perspective of UAV B itself.

4. The reservoir dam safety monitoring method based on multi-source information according to claim 1, characterized in that: The angle information in S105 includes a horizontal angle and a vertical angle, which are defined as α and β respectively.

5. The reservoir dam safety monitoring method based on multi-source information according to claim 1, characterized in that: The S106 includes: comparing the pixel values ​​of drone B in the A image with the pixel values ​​of drone B in the normal horizontal state image, and calculating the pitch angle λ between drone B and the baseline reference line through pixel difference; obtaining an image of drone B in the normal horizontal state as a reference image; comparing the pixel values ​​of the drone B area in the A image with the corresponding area in the normal horizontal state image, and analyzing the difference in pixel values; and inferring the pitch angle λ of drone B relative to the baseline reference line by combining the camera viewing angle, the physical characteristics of drone B, and the data of the multi-source information monitoring equipment.

6. The reservoir dam safety monitoring method based on multi-source information according to claim 1, characterized in that: The S107 includes: S201, extracting historical inspection data from the UAV flight record, and using an image fusion algorithm to fuse historical A and B images to establish safety hazard sample information; S202, combining the historical pitch angle λ and the historical panoramic image to establish a panoramic model of the reservoir dam inspected by the drone in the historical time; S203, using the currently collected pitch angle λ and the panoramic image to establish a current panoramic model of the reservoir dam inspected by the drone; S204, using historical inspection data and annotated safety hazard samples to train a deep learning model to identify safety hazards; S205, applying the trained potential safety hazard identification model to the current panoramic model to identify potential safety hazards; S206: Analyze the prominent features of the identified safety hazards to determine the nature, extent and scope of impact of the hazards.

7. The reservoir dam safety monitoring method based on multi-source information according to claim 6 is characterized in that: The S202 includes: taking half a year of historical time as a cycle, and four cycles as a group of historical time; collecting the flight data of the UAV within the four historical cycles, including the change record of the pitch angle λ; obtaining a panoramic image at the time point corresponding to the flight data; wherein the panoramic image is formed by fusing the A image and the B image taken by the UAV at different perspectives through the image fusion technology; using the three-dimensional modeling technology OpenGL tool, combining the pitch angle λ and the panoramic image to construct a panoramic model of the reservoir dam; using the three-dimensional modeling tool to fuse these data into the model to form a panoramic model that can reflect the dynamic changes of the state of the reservoir dam at different time points.

8. The reservoir dam safety monitoring method based on multi-source information according to claim 6 is characterized in that: The S203 includes: S301, classifying height information of the dam body from the panoramic model according to the spatial layout in the panoramic model; S302, adding the collected multi-source information features to the classified panoramic model to form a panoramic monitoring model of the dam status; S303, collecting panoramic monitoring models in historical time and training boundary point safety hazard prediction models; S304, predicting the safety hazards of the dam body at the boundary points within half a cycle, and displaying the safety hazard prediction situation in a dynamic three-dimensional image within half a cycle.

9. The reservoir dam safety monitoring method based on multi-source information according to claim 8, characterized in that: The S301, grading the height information of the dam body, includes: dividing the dam body into three levels: bottom, middle and top according to the height information of the dam body; if the monitoring equipment is installed below the reservoir water level, the bottom includes the dam foundation and the dam toe part, the middle part is the main load-bearing part of the dam body, and the top includes the dam crest and the wave-breaking wall; if the monitoring equipment is installed above the reservoir water level, the bottom is the part above the reservoir water level line.

10. The reservoir dam safety monitoring method based on multi-source information according to claim 8, characterized in that: The S301 further includes: S401, partitioning the graded panoramic model, and marking the boundary and position of each partition according to the determined partition standard; S402, determining whether there is a focus area according to the partitioned panoramic model; S403, determining whether it is necessary to perform separate zone inspections on drone A and drone B and collect separate zone panoramic models according to the status of the key focus area; S404, training a safety hazard prediction model for partition boundary points according to the partition panoramic model characteristics.

Images