Hydropower Engineering Monitoring Method and System Based on UAV Inspection and 3D Modeling

Abstract

The present disclosure provides a method and system for hydropower project monitoring based on drone inspection and 3D modeling, which relates to the field of image recognition technology. The method includes: collecting color images, infrared images, and laser point cloud data; constructing a 3D point cloud model corresponding to the hydropower project area according to the laser point cloud data, and mapping pixel data and temperature data to the 3D point cloud model to generate a 3D scene model; performing a regional segmentation operation on the 3D scene model to obtain a target sub-model, and extracting features from the target sub-model to generate an image semantic feature vector and a heat distribution feature vector; performing anomaly recognition on the image semantic feature vector and the heat distribution feature vector through a pre-trained anomaly recognition model to obtain corresponding anomaly recognition results, and adding the anomaly recognition results to the spatial positions corresponding to the 3D scene model. This solution can improve the spatial expression ability of image data and the accuracy of anomaly recognition during the hydropower project monitoring process.

Description

Background Art

[0002] In the inspection and monitoring work of related hydropower projects, image information is mainly collected by manual shooting, fixed-point videography or aerial photography, and is displayed and analyzed in a two-dimensional form. Although such means can provide intuitive pictures, their processing methods usually only stay on the images themselves, and no spatial correspondence relationship is established between the image data and the on-site structure, making it difficult to perform subsequent anomaly localization.

[0003] On the other hand, in the related technology when performing image analysis, it mainly relies on manual observation and empirical judgment to identify abnormal phenomena. The operation process is time-consuming and easily affected subjectively. It not only has low efficiency and is prone to omission, but also is difficult to meet the actual needs of high-frequency inspections and multi-target parallel recognition. In a complex operation environment, the limitations of manual recognition are more obvious, affecting the safe operation and intelligent management level of hydropower projects.

[0004] Therefore, there is still room for improvement in the existing hydropower project monitoring technology in terms of the spatial expression of image data and the ability to identify anomalies. Summary of the Invention

[0005] The purpose of the embodiments of the present disclosure is to provide a hydropower project monitoring method based on drone inspection and three-dimensional modeling, a hydropower project monitoring system based on drone inspection and three-dimensional modeling, an electronic device, and a computer-readable storage medium, which can construct a spatially aligned monitoring view in an actual three-dimensional scene, and use an anomaly recognition model for discriminant analysis, thereby improving the spatial expression ability of image data and the accuracy of anomaly recognition during the hydropower project monitoring process.

[0006] Other features and advantages of the present disclosure will become apparent through the following detailed description, or be learned in part through the practice of the present disclosure.

[0007] According to the first aspect of the embodiments of the present disclosure, there is provided a hydropower project monitoring method based on drone inspection and three-dimensional modeling, including: using a drone to fly along a preset flight path in the hydropower project area, and collecting color images, infrared images and laser point cloud data; constructing a three-dimensional point cloud model corresponding to the hydropower project area according to the laser point cloud data, and mapping pixel data and temperature data to the three-dimensional point cloud model based on the pose parameters of the color image and the infrared image to generate a three-dimensional scene model; performing a region segmentation operation on the three-dimensional scene model to obtain a target sub-model, and respectively extracting features from the pixel data and the temperature data in the target sub-model to generate an image semantic feature vector and a heat distribution feature vector; performing anomaly recognition on the image semantic feature vector and the heat distribution feature vector through a pre-trained anomaly recognition model to obtain corresponding anomaly recognition results, and adding the anomaly recognition results to the corresponding spatial positions of the three-dimensional scene model.

[0008] In some exemplary embodiments of the present disclosure, based on the foregoing solution, the above-mentioned hydropower project monitoring method based on drone inspection and 3D modeling further includes: using the spatial position corresponding to the abnormal recognition result in the 3D scene model as an inspection point, establishing a correspondence between the inspection point and the abnormal recognition result, and generating an abnormal inspection path based on the inspection point; controlling the drone to perform image acquisition along the abnormal inspection path, and when reaching the inspection point corresponding to the preset abnormal recognition result, playing the warning voice information corresponding to the preset abnormal recognition result.

[0009] In some exemplary embodiments of the present disclosure, based on the foregoing solution, the generating of the abnormal inspection path based on the inspection point includes: determining a starting point according to the starting position of the drone, and constructing a node set according to the starting point and the inspection point; calculating the spatial distance between each node in the node set, and obtaining a plurality of connection edges according to the comparison result between the spatial distance and a preset distance threshold; extracting point cloud data from the spatial region corresponding to each connection edge in the 3D point cloud model; performing grid processing on the path point cloud data, and obtaining elevation change information and obstacle distribution information corresponding to each connection edge based on the height difference and point density characteristics within each grid; assigning a passage cost value to each connection edge according to the elevation change information and obstacle distribution information, and constructing the abnormal inspection path according to the passage cost value.

[0010] In some exemplary embodiments of the present disclosure, based on the foregoing solution, the constructing of the abnormal inspection path according to the passage cost value includes: establishing a path connection graph according to the node set and the plurality of connection edges; in the path connection graph, starting from the starting point, traversing all inspection points, and determining the node connection order with the minimum total passage cost value according to the passage cost value corresponding to each connection edge; constructing the abnormal inspection path according to the node connection order.

[0011] In some exemplary embodiments of the present disclosure, based on the foregoing solution, the mapping of the pixel data and the temperature data to the 3D point cloud model based on the pose parameters of the color image and the infrared image includes: obtaining the pose parameters of the color image and the infrared image, and determining a transformation matrix of the infrared image relative to the color image reference coordinate system based on the pose parameters; transforming the infrared image to the color image reference coordinate system according to the transformation matrix to generate an aligned infrared image; mapping the pixel data corresponding to the color image and the temperature data corresponding to the aligned infrared image to the 3D point cloud model according to the correspondence between the color image, the infrared image, and the laser point cloud data.

[0012] In some exemplary embodiments of the present disclosure, based on the foregoing solution, the generation process of the image semantic feature vector includes: converting the pixel data in the target sub-model into an image tensor; performing multiple convolution operations and downsampling operations on the image tensor to extract the hierarchical semantic features of the image tensor; performing feature compression on the hierarchical semantic features in the channel dimension to obtain a single semantic feature; performing global average pooling on the single semantic feature, and using the pooling result as the image semantic feature vector.

[0013] In some exemplary embodiments of the present disclosure, based on the foregoing solution, the generation process of the heat distribution feature vector includes: performing normalization processing on the temperature data to generate a normalized temperature map; in the normalized temperature map, calculating a thermal response mask based on the ratio of the local temperature peak to the surrounding temperature, and generating a heat response region; performing regional temperature consistency analysis on the heat response region to obtain the central average temperature, boundary gradient value, and internal temperature dispersion degree of the heat response region; constructing the heat distribution feature vector based on the central average temperature, boundary gradient value, and internal temperature dispersion degree.

[0014] In some exemplary embodiments of the present disclosure, based on the foregoing solution, the calculating a thermal response mask based on the ratio of the local temperature peak to the surrounding temperature in the normalized temperature map and generating a heat response region includes: extracting local temperature peak points in the normalized temperature map, and constructing a neighborhood window with a fixed size based on each peak point; calculating the average temperature in each neighborhood window, and calculating the ratio of the average temperature to the temperature value of the corresponding peak point; using the pixel points whose ratio exceeds a preset thermal response threshold as response points; performing regional aggregation processing on adjacent response points to generate a thermal response mask, and generating a corresponding heat response region based on the thermal response mask.

[0015] In some exemplary embodiments of the present disclosure, based on the foregoing solution, the performing anomaly recognition on the image semantic feature vector and the heat distribution feature vector through a pre-trained anomaly recognition model to obtain corresponding anomaly recognition results includes: inputting the image semantic feature vector into a first anomaly recognition model constructed by a convolutional neural network and a spatial attention mechanism to generate a first anomaly recognition result; inputting the heat distribution feature vector into a second anomaly recognition model constructed by a multi-layer perceptron network and a heat distribution feature encoding layer to generate a second anomaly recognition result; wherein, the first anomaly recognition result includes any one or more of slope deformation, concrete cracks, landslides, structural water seepage, collapses, guardrail damage, and floating objects on the water surface, and the second anomaly recognition result includes any one or more of personnel staying, swimming behavior, and abnormal high temperature points.

[0016] In some exemplary embodiments of the present disclosure, based on the foregoing solution, the step of inputting the image semantic feature vector into a first anomaly recognition model constructed by a convolutional neural network and a spatial attention mechanism to generate a first anomaly recognition result includes: performing multi-layer convolutional operations on the image semantic feature vector by using the convolutional neural network to extract a deep image feature map; adjusting the response weights of each region in the deep image feature map based on the spatial attention mechanism to generate an enhanced image feature map; and performing anomaly category determination on the enhanced image feature map to generate the first anomaly recognition result.

[0017] In some exemplary embodiments of the present disclosure, based on the foregoing solution, the step of inputting the heat distribution feature vector into a second anomaly recognition model constructed by a multi-layer perceptron network and a heat distribution feature encoding layer to generate a second anomaly recognition result includes: respectively inputting the central average temperature, the boundary gradient value, and the temperature dispersion in the heat distribution feature vector into the corresponding heat distribution feature encoding layer; performing normalization and scale alignment processing on the input data in the heat distribution feature encoding layer to obtain a three-channel feature vector; splicing the three-channel feature vectors in a preset order to generate a combined feature vector; inputting the combined feature vector into the multi-layer perceptron network for non-linear mapping processing, and generating the second anomaly recognition result based on the result of the non-linear mapping processing.

[0018] According to a second aspect of the embodiments of the present disclosure, there is provided a hydropower project monitoring system based on drone inspection and three-dimensional modeling, including: an image acquisition module, configured to fly a drone along a preset flight path in a hydropower project area and acquire color images, infrared images, and laser point cloud data; a three-dimensional reconstruction module, configured to construct a three-dimensional point cloud model corresponding to the hydropower project area according to the laser point cloud data, and map pixel data and temperature data to the three-dimensional point cloud model based on the pose parameters of the color image and the infrared image to generate a three-dimensional scene model; a vector extraction module, configured to perform regional segmentation operations on the three-dimensional scene model to obtain a target sub-model, and respectively extract features from the pixel data and the temperature data in the target sub-model to generate an image semantic feature vector and a heat distribution feature vector; and an anomaly recognition module, configured to perform anomaly recognition on the image semantic feature vector and the heat distribution feature vector through a pre-trained anomaly recognition model to obtain corresponding anomaly recognition results, and add the anomaly recognition results to spatial positions corresponding to the three-dimensional scene model.

[0019] According to a third aspect of the embodiments of the present disclosure, there is provided an electronic device, including: a processor; and a memory, having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the hydropower project monitoring method based on drone inspection and three-dimensional modeling as in the first aspect is implemented.

[0020] According to a fourth aspect of the embodiments of the present disclosure, there is provided a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the hydropower project monitoring method based on drone inspection and three-dimensional modeling as in the first aspect.

[0021] The technical solutions provided by the embodiments of the present disclosure may include the following beneficial effects:

[0022] In the hydropower project monitoring method based on drone inspection and three-dimensional modeling in the exemplary embodiments of the present disclosure, on the one hand, by controlling the drone to collect color images, infrared images and laser point cloud data in the hydropower project area along a preset path, compared with the image acquisition method that relies on manual or fixed camera methods, the image coverage range is improved. On the other hand, a three-dimensional point cloud model of the hydropower project area is constructed based on the laser point cloud data, and in combination with the pose parameters of the color image and the infrared image, the pixel data and temperature data in the image are mapped to the model, realizing the integrated modeling and visual association of image information and spatial structure, and solving the problem that the observation results cannot be spatially located in the related art, which is convenient for result presentation.

[0023] On the other hand, the generated three-dimensional scene model is regionally segmented, and feature extraction operations are respectively performed on the pixel data and temperature data in the target sub-model to obtain an image semantic feature vector and a thermal distribution feature vector, which can input the observation information into a pre-trained anomaly recognition model in a structured form. This method realizes the automatic recognition of hydropower project anomalies and solves the problems of the deficiencies of the traditional manual method in terms of recognition efficiency, accuracy and scene adaptability. Further, the recognition result is located at the corresponding spatial position in the three-dimensional scene model, making the display relationship between the recognition result and the actual position clearer and facilitating the quick understanding of the anomaly distribution. Thus, it can improve the spatial expression ability of image data and the accuracy of anomaly recognition to a certain extent during the hydropower project monitoring process.

[0024] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The accompanying drawings here are incorporated into the specification and constitute a part of this specification, showing embodiments consistent with the present disclosure, and are used together with the specification to explain the principles of the present disclosure. Obviously, the accompanying drawings in the following description are only some embodiments of the present disclosure. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0026] Figure 1Schematically shows a flowchart of a hydropower project monitoring method based on drone inspection and 3D modeling according to some embodiments of the present disclosure.

[0027] Figure 2 Schematically shows a system structure diagram to which the hydropower project monitoring method in the embodiments of the present disclosure can be applied.

[0028] Figure 3 Schematically shows a flowchart of generating an abnormal inspection path according to some embodiments of the present disclosure.

[0029] Figure 4 Schematically shows a flowchart of generating a heat distribution feature vector according to some embodiments of the present disclosure.

[0030] Figure 5 Schematically shows a process diagram of a hydropower project monitoring method according to some embodiments of the present disclosure.

[0031] Figure 6 Schematically shows a schematic diagram of an interaction interface of a hydropower project monitoring system according to some embodiments of the present disclosure.

[0032] Figure 7 Schematically shows a schematic diagram of a hydropower project monitoring system based on drone inspection and 3D modeling according to some embodiments of the present disclosure.

[0033] Figure 8 Schematically shows a schematic diagram of a computer system of an electronic device according to some embodiments of the present disclosure.

[0034] Figure 9 Schematically shows a schematic diagram of a computer-readable storage medium according to some embodiments of the present disclosure.

[0035] In the drawings, the same or corresponding reference numerals indicate the same or corresponding parts. Detailed Description of the Embodiments

[0036] Here, the exemplary embodiments will be described in detail, and the examples are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings indicate the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this specification. On the contrary, they are merely examples of devices and methods consistent with some aspects of this specification as detailed in the appended claims.

[0037] The terms used in this specification are for the purpose of describing particular embodiments only and are not intended to limit this specification. The singular forms "a", "said", and "the" used in this specification and the appended claims are also intended to include the plural forms unless the context clearly dictates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0038] Example embodiments will now be described more fully with reference to the accompanying drawings. However, the example embodiments can be implemented in various forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the example embodiments to those skilled in the art.

[0039] In addition, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of the embodiments of the present disclosure. However, those skilled in the art will recognize that the technical solutions of the present disclosure may be practiced without one or more of the specific details, or may be implemented using other methods, components, devices, steps, etc. In other cases, well-known methods, devices, implementations, or operations are not shown or described in detail to avoid obscuring aspects of the present disclosure.

[0040] In addition, the accompanying drawings are only schematic illustrations and are not necessarily drawn to scale. The block diagrams shown in the drawings are only functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0041] In the present example embodiment, first, a method for monitoring a hydropower project based on drone inspection and 3D modeling is provided. Figure 1 A flowchart of a method for monitoring a hydropower project based on drone inspection and 3D modeling according to some embodiments of the present disclosure is schematically shown. Refer to Figure 1 As shown, the method for monitoring a hydropower project based on drone inspection and 3D modeling may include the following steps:

[0042] Step S110, flying a drone along a preset flight path in the hydropower project area and collecting color images, infrared images, and laser point cloud data;

[0043] Step S120: Construct a three-dimensional point cloud model corresponding to the hydropower project area based on the laser point cloud data, and map the pixel data and temperature data to the three-dimensional point cloud model based on the pose parameters of the color image and the infrared image to generate a three-dimensional scene model;

[0044] Step S130: Perform a regional segmentation operation on the three-dimensional scene model to obtain the target sub-model, and respectively extract features from the pixel data and temperature data in the target sub-model to generate an image semantic feature vector and a thermal distribution feature vector;

[0045] Step S140: Perform anomaly recognition on the image semantic feature vector and the thermal distribution feature vector through a pre-trained anomaly recognition model to obtain the corresponding anomaly recognition result, and add the anomaly recognition result to the corresponding spatial position of the three-dimensional scene model.

[0046] According to the hydropower project monitoring method based on drone patrol and three-dimensional modeling in this exemplary embodiment, on the one hand, by controlling the drone to collect color images, infrared images and laser point cloud data in the hydropower project area along a preset path, compared with the image acquisition method that relies on manual or fixed camera methods, the image coverage range is improved. On the other hand, a three-dimensional point cloud model of the hydropower project area is constructed based on the laser point cloud data, and combined with the pose parameters of the color image and the infrared image, the pixel data and temperature data in the image are mapped to this model, realizing the integrated modeling and visual association of image information and spatial structure. It not only solves the problem that the observation results cannot be spatially located in the related technology, but also gives the image content a clear spatial attribution relationship, which is convenient for result presentation. On the other hand, the generated three-dimensional scene model is segmented into regions, and feature extraction operations are respectively performed on the pixel data and temperature data in the target sub-model to obtain an image semantic feature vector and a thermal distribution feature vector, which can input the observation information into the pre-trained anomaly recognition model in a structured form. This method realizes the automatic recognition of hydropower project anomalies and solves the problems of the deficiencies of the traditional manual method in terms of recognition efficiency, accuracy and scene adaptability. Further, the recognition result is located at the corresponding spatial position in the three-dimensional scene model, making the display relationship between the recognition result and the actual position clearer and facilitating a quick understanding of the anomaly distribution based on the model. Thus, it can improve the spatial expression ability of image data and the accuracy of anomaly recognition to a certain extent during the hydropower project monitoring process.

[0047] Next, the hydropower project monitoring method based on drone patrol and three-dimensional modeling in this exemplary embodiment will be further described.

[0048] Step S110: Use the drone to fly along a preset flight path in the hydropower project area and collect color images, infrared images and laser point cloud data.

[0049] Among them, the hydropower project area can represent the geographical space range where hydropower station engineering facilities are located, including but not limited to the reservoir storage area, the dam body of the dam, the diversion canal, the flood discharge channel, the slope, and the surrounding protection area, etc. The color image can represent two-dimensional image data obtained through a visible light image sensor, reflecting the color and texture information of the surface of the measured target. The infrared image can represent two-dimensional image data generated based on the thermal radiation intensity of the target surface collected through an infrared thermal imaging sensor, reflecting the temperature distribution characteristics of the measured object within different wavelength ranges, and can be used to judge states such as thermal anomalies, local heating, and seepage areas. The lidar point cloud data can represent a set of three-dimensional coordinates of a large number of spatial points within the target area obtained through a lidar device, and can be used to describe spatial structure characteristics such as terrain undulations, building outlines, and slope structures.

[0050] When collecting the above color images, infrared images, and lidar point cloud data, timestamp information and corresponding aircraft attitude parameters can be attached, and the spatio-temporal correspondence relationship between the data can be established through the flight track and sensor field-of-view parameters, so as to ensure the spatial synchronization and fusion feasibility of multi-source data in subsequent processing.

[0051] Step S120: Construct a three-dimensional point cloud model corresponding to the hydropower project area based on the lidar point cloud data, and map the pixel data and temperature data to the three-dimensional point cloud model based on the pose parameters of the color image and the infrared image to generate a three-dimensional scene model.

[0052] Among them, the three-dimensional point cloud model can represent a spatial structure expression model constructed based on the lidar point cloud data, consisting of multiple three-dimensional discrete points with spatial coordinate information. This model can reflect the outlines and terrain undulations of physical structures such as the dam body and slope within the hydropower project area. The pose parameters can represent the spatial position and orientation information of the image sensor relative to the unified reference coordinate system when collecting color image and infrared image data, and can include position vectors, translation matrices, and attitude angles or rotation matrices, which are used to establish the geometric mapping relationship between the image data and the three-dimensional space. The pixel data can represent the RGB values of the image reflecting the color and texture of the surface of the target area obtained through the color image. The temperature data can represent the thermal radiation intensity value corresponding to each image pixel point collected through the infrared image, which is used to characterize the temperature distribution state of the target. The three-dimensional scene model can represent a fusion model formed by projecting the pixel data in the color image and the temperature data in the infrared image onto the three-dimensional point cloud model based on the pose parameters respectively. On the basis of maintaining the original point cloud geometric structure, this model attaches visual texture and thermal radiation attributes to each spatial point, realizing the fusion expression of multi-modal data in the unified spatial coordinate system.

[0053] Step S130: Perform an area segmentation operation on the three-dimensional scene model to obtain a target sub-model, and respectively extract features from the pixel data and temperature data in the target sub-model to generate an image semantic feature vector and a thermal distribution feature vector.

[0054] Among them, the target sub-model can represent a local spatial structure model obtained by region division, structure segmentation, or specified spatial range extraction on the basis of the three-dimensional scene model. This target sub-model corresponds to a specific monitoring object area, such as the dam surface, high slope area, spillway boundary, etc. It is intercepted from the original three-dimensional scene model and retains its point cloud structure, pixel attributes, and temperature attributes. The image semantic feature vector can represent a multi-dimensional vector obtained through feature extraction operations based on the pixel data contained in the target sub-model, and is used to express the semantic attributes of the image content within this area. The thermal distribution feature vector can represent a multi-dimensional vector formed by characterizing the temperature distribution state of the local area for the temperature data contained in the target sub-model.

[0055] In this embodiment, by performing an area segmentation operation on the three-dimensional scene model, the entire monitoring area can be divided into multiple spatial structure units, enabling subsequent analysis to focus on sub-regions with clear geographical attribution and structural significance, reducing the redundancy of overall data processing while enhancing the recognition pertinence of key regions.

[0056] Step S140: Use a pre-trained anomaly recognition model to perform anomaly recognition on the image semantic feature vector and the thermal distribution feature vector to obtain corresponding anomaly recognition results, and add the anomaly recognition results to the corresponding spatial positions of the three-dimensional scene model.

[0057] Among them, the anomaly recognition model can represent a discriminant model used to analyze and judge the image semantic feature vector and the thermal distribution feature vector to identify whether there are abnormal targets or states. The anomaly recognition result can represent the discriminant information output after the anomaly recognition model processes the input image semantic feature vector and thermal distribution feature vector, and is used to characterize whether there are structural anomalies, thermal anomalies, or external interferences, etc. in the target area. The anomaly recognition result can include but is not limited to anomaly types, confidence scores, etc. Adding the anomaly recognition result to the attribute fields of the corresponding spatial points or regions in the three-dimensional scene model enables the recognition information to be presented in a graphical or annotated manner in the structural model, thereby enhancing the locatability and visualization of the anomaly analysis results.

[0058] Figure 2Schematically shows a system structure diagram of a system to which the above-mentioned hydropower project monitoring method based on drone inspection and 3D modeling can be applied. This method can be specifically implemented under the system architecture shown in the figure. The system includes: a drone hangar, a path planning unit, a monitoring and warning system, a 3D model construction unit, an intelligent inspection analysis unit, and a user terminal.

[0059] In this embodiment, the drone hangar is used to store and manage the drone devices to be deployed, and fly in response to the inspection path sent by the monitoring and warning system. The drone is equipped with a color image sensor, an infrared thermal imaging device, and a lidar device, and flies in the hydropower project area according to the inspection path generated by the path planning unit, collects color images, infrared images, and lidar point cloud data of the target area, and transmits the inspection images and lidar point cloud data back to the monitoring and warning system. The monitoring and warning system receives the data transmitted by the drone and transmits it to the 3D model construction unit. The 3D model construction unit first constructs a 3D point cloud model corresponding to the hydropower project area based on the lidar point cloud data, and then maps the pixel data and temperature data to the 3D point cloud model based on the pose parameters of the color image and the infrared image to generate a 3D scene model. This 3D scene model is sent to the monitoring and warning system as the spatial basis for subsequent analysis.

[0060] The monitoring and warning system continues to perform a region segmentation operation on the 3D scene model to obtain a target sub-model, and respectively extracts features from the pixel data and temperature data in the target sub-model to generate an image semantic feature vector and a heat distribution feature vector. The extracted feature vectors are input into a pre-trained anomaly recognition model to perform anomaly recognition on the image semantic feature vector and the heat distribution feature vector, and finally obtain the corresponding anomaly recognition result. A monitoring report is generated according to the anomaly recognition result, and the anomaly recognition result is added to the spatial position corresponding to the 3D scene model and a warning message is generated. The monitoring and warning system sends the color image and the infrared image to the intelligent inspection analysis unit to generate a corresponding inspection report. The intelligent inspection analysis unit can send the inspection report to the monitoring and warning system for display. At the user end, the operator can receive the monitoring report, inspection report, and warning message sent by the monitoring and warning system through the human-machine interface, and can input control commands through the interaction interface and send them back to the monitoring and warning system to trigger subsequent drone re-inspection, path adjustment, or warning broadcast operations.

[0061] In some embodiments, the above-mentioned hydropower project monitoring method based on drone patrol and 3D modeling further includes the following technical steps: taking the spatial position corresponding to the anomaly recognition result in the 3D scene model as an inspection point, establishing the corresponding relationship between the inspection point and the anomaly recognition result, and generating an abnormal inspection path based on the inspection point; controlling the drone to perform image acquisition along the abnormal inspection path, and when reaching the inspection point corresponding to the preset anomaly recognition result, playing the warning voice information corresponding to the preset anomaly recognition result.

[0062] Among them, the inspection point refers to the spatial target point to be focused on rechecking determined according to the spatial position corresponding to the anomaly recognition result in the 3D scene model. Each inspection point has a one-to-one or one-to-many corresponding relationship with at least one anomaly recognition result, which is used to indicate the key stop or attention position of the drone in the subsequent abnormal inspection path. Generating an abnormal inspection path based on the inspection point and controlling the drone to perform image acquisition operations according to this path enables the drone patrol task to focus on covering high-risk areas, improving the scheduling efficiency of drone patrol and the pertinence of task response. When reaching a specific inspection point, playing the warning voice information matching the corresponding anomaly type can actively inform the on-site abnormal state during the drone patrol, enhancing the real-time and automation capabilities of on-site risk disposal.

[0063] In the specific implementation process, based on the spatial distribution information of multiple inspection points, combined with their associated anomaly types, recognition confidence levels or spatial priorities, use the path planning algorithm to generate an abnormal inspection path covering all inspection points. This abnormal inspection path can be defined as starting from the current parking position or the nearest charging point of the drone, passing through each inspection point in turn, and meeting scheduling constraints such as the shortest flight distance and obstacle avoidance safety. Subsequently, control the drone to perform image acquisition along the abnormal inspection path, record the flight state and image data in real time during the flight, and when reaching each inspection point corresponding to the preset anomaly recognition result, play the warning voice information corresponding to the preset anomaly recognition result. The warning voice information can include a fixed voice template or a broadcast instruction corresponding to the content of the anomaly type, which is used for on-site broadcast warnings for personnel approaching illegally, illegal behaviors or sudden thermal abnormal situations.

[0064] In some embodiments, the above-mentioned generation of the abnormal inspection path based on the inspection point can be realized through the following steps S310 to S350, which specifically include:

[0065] Step S310, determine the starting point according to the starting position of the drone, and construct a node set according to the starting point and the inspection point.

[0066] Step S320, calculate the spatial distance between each node in the node set, and obtain a plurality of connection edges according to the comparison result between the spatial distance and the preset distance threshold.

[0067] Specifically, the acquired node set includes a starting point and multiple inspection points, wherein the starting point can be the current location of the drone or the preset mission take-off point, and the inspection point is a spatial position determined in the three-dimensional scene model according to the abnormality recognition result. In the path generation stage, the distance between the three-dimensional spatial coordinates of any two nodes in the node set is calculated. The spatial distance can be solved based on the three-dimensional Euclidean distance formula to measure the direct connectivity between nodes. In order to avoid redundant connections and too long flight paths, the spatial distances between all calculated node pairs are compared one by one with the preset distance threshold. When the spatial distance between any node pair is less than or equal to the preset distance threshold, the node pair is determined to be connectable, and a connecting edge is established between the two, thereby generating a connecting edge set representing the pre-selected flight segment.

[0068] Step S330, performing point cloud extraction on the spatial area corresponding to each connecting edge in the three-dimensional point cloud model to obtain path point cloud data.

[0069] The path point cloud data may represent a point cloud subset extracted from the spatial region corresponding to the connecting edge based on the three-dimensional point cloud model. The process of determining the spatial region is as follows: taking the line between the starting node and the ending node of the connecting edge as the central axis, setting a fixed width, height and safety margin along the axis direction, and constructing a three-dimensional spatial region surrounding the connecting edge. The spatial region may be in the form of a cuboid, cylinder or other suitable structure to limit the range in which the point cloud data can be extracted.

[0070] Step S340, gridding the path point cloud data, and obtaining the elevation change information and obstacle distribution information corresponding to each connecting edge based on the height difference and point density characteristics in each grid.

[0071] Among them, the height difference value can represent the difference in vertical coordinate values between the maximum point and the minimum point in each grid unit corresponding to the path point cloud data, which is used to reflect the terrain undulation or structural height change in the area. The point density feature can represent the number of point clouds contained in a unit grid, which is used to characterize the complexity of the spatial structure or the aggregation of obstacles. The elevation change information can represent the set of height difference values extracted in each grid based on the path point cloud data, reflecting the terrain undulation and slope characteristics along the path segment. The obstacle distribution information can refer to the expression of the concentration or distribution trend of obstacles that may exist in the spatial area based on the statistical results of the point density features in the path point cloud data.

[0072] In specific implementation, a regular three-dimensional grid structure can be constructed within the spatial region corresponding to the connection edge. The path point cloud data is divided into multiple grid cells according to its spatial coordinates, and each grid corresponds to a fixed spatial range. For each grid cell, based on the point cloud data it contains, the height difference of the points within the grid is calculated, that is, the difference between the maximum height coordinate value and the minimum height coordinate value, which is used to reflect the vertical undulation of this region. At the same time, the number of points within the grid is counted to determine the point density characteristic of the grid. Further, the height differences of all grid cells are obtained to construct the elevation change information corresponding to this connection edge; at the same time, based on the extraction results of the point density characteristics of each grid, the obstacle distribution information corresponding to this connection edge is generated, which is used to depict the structural complexity or the obstacle distribution trend along this path segment.

[0073] Step S350, assign a passage cost value to each connection edge according to the elevation change information and the obstacle distribution information, and construct an abnormal inspection path according to the passage cost value.

[0074] Among them, the passage cost value can represent a numerical index given after comprehensively evaluating the passability of this path segment based on the elevation change information and the obstacle distribution information corresponding to the connection edge, and is used to measure the difficulty for the UAV to pass through this path segment when performing the abnormal inspection task. Denote the passage cost value as , and its calculation formula is as follows:

[0075]

[0076] Among them, represents the passage cost value of the corresponding connection edge; represents the normalized mean value of the elevation change information, reflecting the overall slope complexity of the path segment; represents the maximum point density value in the obstacle distribution information, which is used to measure the concentration degree of obstacles in the local space; represents the standard deviation of the height differences of all grids within the path segment, which is used to describe the discreteness and uneven undulation degree of the elevation change; , , respectively represent the elevation change weight, the obstacle risk weight, and the terrain disturbance weight.

[0077] In some embodiments, the construction of the abnormal inspection path according to the passage cost value in step S350 can be implemented through the following steps. Specifically, it can include: establishing a path connection graph according to the node set and multiple connection edges; in the path connection graph, starting from the starting point, traversing all inspection points, and determining the node connection order with the minimum total passage cost value according to the passage cost value corresponding to each connection edge; constructing an abnormal inspection path according to the node connection order.

[0078] In the specific implementation process, first, a path connection graph is constructed based on the node set and multiple connection edges, denoted as , where represents the node set, including the starting point and all inspection points; represents the set of connection edges that meet the connection conditions; is the weight set of the connection edges, and the weight is composed of the passage cost value, represents the node and the node the passage cost value between the connection edges.

[0079] In the path connection graph , with the starting point as the path starting point, traverse all inspection points except , and construct the optimal node access order , where , to minimize the following total cost function:

[0080]

[0081] Among them, represents the total cost value function of the abnormal inspection path; represents all permutation and combination sets starting from and including all inspection points; represents the th node and the th node in the path. The passage cost value between; represents the deflection angle of the path at the th turning point, defined as the angle amplitude between the current edge and the previous edge; represents the local elevation gradient change rate of the path on the connection edge where the node is located; is the turning penalty weight coefficient, used to suppress the oscillation of the flight path; is the height difference penalty weight coefficient, used to avoid paths with sharp elevation fluctuations, represents the number of inspection points participating in the path traversal except the starting point.

[0082] After obtaining the node connection order that makes reach the minimum value, the corresponding connection edges can be spliced in sequence according to this node connection order to construct an abnormal inspection path.

[0083] Next, the content in the above steps S110 to S140 will be described in detail.

[0084] In some embodiments, the pose parameters based on the color image and the infrared image in step S120 above can be achieved through the following technical process, mapping the pixel data and the temperature data to the three-dimensional point cloud model, which may specifically include: obtaining the pose parameters of the color image and the infrared image, and determining the transformation matrix of the infrared image relative to the reference coordinate system of the color image based on the pose parameters; transforming the infrared image to the reference coordinate system of the color image according to the transformation matrix to generate an aligned infrared image; mapping the pixel data corresponding to the color image and the temperature data corresponding to the aligned infrared image to the three-dimensional point cloud model according to the correspondence between the color image, the infrared image and the laser point cloud data.

[0085] Among them, the transformation matrix refers to the mathematical transformation relationship used to transform the image information in the infrared image coordinate system to the reference coordinate system of the color image. This transformation matrix is calculated from the pose parameters of the infrared image and the color image, and may include a combination of a rotation matrix and a translation vector. Among them, the rotation matrix is used to describe the attitude difference between the two camera coordinate systems, and the translation vector is used to describe the spatial position offset between the two. Through this transformation matrix, the geometric mapping of any point in the infrared image in the reference coordinate system of the color image can be realized. The aligned infrared image refers to the set of image data generated after mapping the pixel points or temperature data in the original infrared image to the reference coordinate system of the color image through the transformation matrix.

[0086] Further, define the transformation matrix as , and the acquisition process of the transformation matrix includes the following steps:

[0087] First, obtain the pose parameters of the corresponding cameras of the infrared image and the color image respectively. The pose parameters include a rotation matrix and a translation vector. Denote the pose of the infrared camera as , and the pose of the color camera as , where: represents the rotation matrix of the infrared camera; represents the translation vector of the infrared camera; represents the rotation matrix of the color camera; represents the translation vector of the color camera.

[0088] According to the transformation relationship between the poses of the two cameras, construct the transformation matrix from the infrared camera coordinate system to the color camera coordinate system:

[0089]

[0090] Among them: represents the direction transformation of the infrared coordinate system in the color image coordinate system; represents the translation transformation of the coordinate origin position difference; Represents the zero vector row of the homogeneous transformation matrix; the constant term 1 represents scale preservation in homogeneous coordinates.

[0091] In some embodiments, the process of generating the image semantic feature vector in step S130 includes: converting the pixel data in the target sub-model into an image tensor; performing multiple convolution operations and downsampling operations on the image tensor to extract hierarchical semantic features of the image tensor; performing feature compression on the hierarchical semantic features in the channel dimension to obtain a single semantic feature; performing global average pooling on the single semantic feature, and using the pooling result as the image semantic feature vector.

[0092] Among them, the image tensor can represent a multi-dimensional array structure formed by organizing the two-dimensional pixel data in the target sub-model according to its spatial structure and channel structure, including three dimensions of height, width, and channels, and is used as the data representation form input to the deep learning network. The downsampling operation can represent the process of compressing the spatial size of the image tensor during the feature extraction process, and is used to retain the backbone features while reducing the computational complexity. The hierarchical semantic features can represent the feature representations extracted at different network levels after performing multiple convolutions and downsampling operations on the image tensor, and this feature can reflect the gradually abstract information from low-level textures to high-level semantics in the image. The single semantic feature can represent the unified feature expression obtained by performing a fusion compression operation on multiple hierarchical semantic features along the channel dimension, and this feature reduces the number of channels while maintaining semantic integrity. Global average pooling can represent the operation of calculating the average value at all positions in the spatial dimension of the feature map, converting the input feature map into a single numerical vector, retaining the overall semantic distribution feature, and is used to generate the final image semantic feature vector.

[0093] In this embodiment, by converting the pixel data in the target sub-model into an image tensor and combining multiple convolution operations and downsampling operations to extract hierarchical semantic features, the spatial structure and semantic information in the image are expressed in multiple scales; further, a single semantic feature is generated through feature compression in the channel dimension, and a global average pooling operation is performed to obtain the image semantic feature vector, realizing the effective aggregation and structured expression of key semantic features in the image region, which helps to improve the discrimination ability and adaptability of the subsequent anomaly recognition model.

[0094] In specific implementation, for the pixel data in the target sub-model, first perform a structure conversion operation on it to reorganize it into an image tensor with a standard input structure, denoted as , the image tensor contains a three-dimensional array with a size of , represents the image height, represents the image width, represents the number of channels, which can include three color channels of red, green, and blue.

[0095] Next, for the image tensor Apply a series of convolutional operations and downsampling operations in sequence to extract the hierarchical semantic features of the image. The convolutional operation uses a set of learnable convolutional kernel sets acting on each layer. The feature map of the th layer can be expressed as:

[0096]

[0097] where, represents the hierarchical semantic features extracted by the th layer, is the input feature map of the previous layer, represents the convolutional operation, represents the convolutional kernel set of the th layer, represents the bias term, represents the downsampling function, such as the max pooling function, etc., represents the non-linear activation function, such as the ReLU function, etc.

[0098] After completing the multi-layer convolution and downsampling, combine the output feature maps of all layers into a multi-channel feature structure and compress it along the channel dimension to obtain a single semantic feature with a unified dimension . This compression process can be expressed as:

[0099]

[0100] where, represents the weighted coefficient of the channel features of the th layer, represents the channel normalization function, L represents the total number of channel layers.

[0101] Finally, perform a global average pooling operation on the single semantic feature to average and aggregate the values at all positions in the spatial dimension to obtain the final image semantic feature vector , and its expression is:

[0102]

[0103] where, represents the value of the compressed feature map at position , and are the height and width of the feature map respectively, represents the row coordinate index, represents the column coordinate index.

[0104] In some embodiments, refer toFigure 4 As shown, the generation of the thermal distribution feature vector can be achieved through steps S410 to S440, specifically including:

[0105] Step S410: Normalize the temperature data to generate a standardized temperature map. Among them, by normalizing the temperature data, the original temperature values are mapped to a unified numerical range, which can eliminate the scale deviation caused by factors such as the measurement range, environmental temperature difference, or device parameters of different infrared images, making the temperature comparison in each image frame consistent.

[0106] Step S420: In the standardized temperature map, calculate the thermal response mask based on the ratio of the local temperature peak to the surrounding temperature, and generate a thermal response region.

[0107] Among them, the local temperature peak can represent the pixel point with the highest temperature value within the neighborhood window range centered on a specified pixel in the standardized temperature map. The surrounding temperature can represent the set of temperatures of all pixels except the central pixel within the neighborhood window constructed around the local temperature peak point. The thermal response mask refers to a weight image mask generated based on the ratio relationship between the local temperature peak and its corresponding surrounding temperature, which is used to identify the pixel regions with significant temperature difference characteristics in the standardized temperature map. The thermal response region can represent the continuous spatial region marked as the high-response state in the thermal response mask, which usually has the local aggregation characteristic that the temperature is significantly higher than the surrounding environment.

[0108] In this step, by constructing the thermal response mask based on the ratio of the local temperature peak to the surrounding temperature in the standardized temperature map, the local thermal anomaly region can be effectively identified from the overall temperature distribution, avoiding the problems of missed detection or false detection of anomalies caused by improper setting of the global threshold. Further, the thermal response region extracted using the thermal response mask can accurately locate the spatial boundary of the local high-temperature region, which helps to improve the target focusing ability of subsequent thermal distribution feature extraction and the discrimination accuracy of the anomaly recognition model.

[0109] Step S430: Conduct regional temperature consistency analysis on the thermal response region to obtain the central average temperature, boundary gradient value, and internal temperature dispersion of the thermal response region.

[0110] Among them, the regional temperature consistency analysis can represent the processing process of statistical calculation and structural measurement of the distribution characteristics of each pixel temperature value within the thermal response region. The central average temperature can represent the average value of the temperature values of all pixels within the neighborhood window with a fixed scale near the regional geometric centroid in the thermal response region. The boundary gradient value can represent the change amplitude of the temperature value along the normal direction at the edge of the thermal response region. The internal temperature dispersion can represent the standard deviation or variance of the temperature values of all pixels within the thermal response region, which is used to measure the uniformity and fluctuation degree of the internal temperature distribution of this region.

[0111] Step S440: Construct a thermal distribution feature vector based on the central average temperature, boundary gradient value, and internal temperature dispersion.

[0112] In some embodiments, in the normalized temperature map, the thermal response mask is calculated based on the ratio of the local temperature peak to the surrounding temperature, and a thermal response region is generated. The specific technical steps are as follows: Extract local temperature peak points in the normalized temperature map, and construct a neighborhood window with a fixed size based on each peak point; Calculate the average temperature within each neighborhood window, and calculate the ratio of the average temperature to the temperature value of the corresponding peak point; Take the pixel points whose ratio exceeds the preset thermal response threshold as response points; Perform region aggregation processing on adjacent response points to generate a thermal response mask, and generate a corresponding thermal response region based on the thermal response mask.

[0113] Specifically, first identify local temperature peak points in the normalized temperature map. Traverse all pixel points in the normalized temperature map based on the sliding window method, and construct a neighborhood window with a fixed size at each central pixel position, denoted as where is the coordinate of the current candidate peak pixel.

[0114] Within each neighborhood window calculate the average temperature value of the surrounding pixels excluding the central pixel and perform a ratio operation with the temperature value of the central pixel of this neighborhood to obtain the local response intensity coefficient:

[0115]

[0116] where represents the local response intensity coefficient, is a small constant introduced to prevent the denominator from being zero.

[0117] Mark all peak pixels that satisfy as response points, where represents the preset thermal response threshold for filtering low-response or ambient background points. Then perform region aggregation processing on all the above response points, and use the connected component analysis method to merge spatially adjacent response points into continuous regions to generate a thermal response mask. This thermal response mask is a binary image that marks all high-response regions with local thermal anomaly characteristics in the current frame.

[0118] In some embodiments, perform region temperature consistency analysis on the thermal response region to obtain the central average temperature, boundary gradient value, and internal temperature dispersion of the thermal response region. The specific technical process is as follows:

[0119] Define the thermal response region as a continuous pixel set The temperature value of any pixel point within the region is denoted as , where .

[0120] First, taking the geometric center point of the thermal response region as a reference, a local window with a fixed size of is constructed in its vicinity . The average operation is performed on all pixel temperature values within this window to obtain the central average temperature:

[0121]

[0122] Wherein, represents the number of neighboring pixels, represents the central average temperature.

[0123] Next, based on the edge contour of the thermal response region, the boundary pixel set is extracted. For each boundary pixel , the adjacent pixel in its normal direction is selected, and the magnitude of the temperature gradient at this point is calculated.

[0124] By traversing all boundary pixels and taking the average of their gradient values, the boundary gradient value of the entire boundary is obtained:

[0125]

[0126] Wherein, represents the number of boundary pixels.

[0127] In addition, to quantify the stability of the temperature distribution within the thermal response region, the temperature value set of all pixel points within the region is statistically analyzed. Taking the overall average value as a reference, the temperature variance within the region is calculated:

[0128]

[0129] Wherein, represents the total number of pixels in the thermal response region, represents the temperature value, represents the temperature variance within the region, which can measure the uniformity and perturbation degree of the internal thermal field. The above three statistical indicators numerically model the spatial thermal structure of the thermal response region from the perspectives of thermal intensity, boundary transition, and internal consistency, providing accurate high-dimensional temperature feature support for subsequent construction of thermal distribution feature vectors and thermal anomaly identification.

[0130] In some embodiments, an anomaly recognition model is pre-trained to recognize anomalies in the image semantic feature vector and the heat distribution feature vector, and the corresponding anomaly recognition results are obtained. The specific technical process is as follows: The image semantic feature vector is input into the first anomaly recognition model constructed by a convolutional neural network and a spatial attention mechanism to generate a first anomaly recognition result; the heat distribution feature vector is input into the second anomaly recognition model constructed by a multi-layer perceptron network and a heat distribution encoding layer to generate a second anomaly recognition result; wherein, the first anomaly recognition result includes any one or more of slope deformation, concrete cracks, landslides, structural water seepage, collapses, guardrail damage, and floating objects on the water surface, and the second anomaly recognition result includes any one or more of personnel retention, swimming behavior, and abnormally high temperature points.

[0131] Among them, the first anomaly recognition model can represent a visual class anomaly recognition network structure constructed based on the image semantic feature vector. The second anomaly recognition model can represent a thermal field behavior class anomaly recognition network structure constructed based on the heat distribution feature vector. In other embodiments of the present disclosure, the first anomaly recognition result may further include types such as excessive water level, component damage, retaining wall cracking, water surface siltation, construction leftovers, dam body spalling, blocked passageways, and displacement of auxiliary structures, and the second anomaly recognition result may further include other types such as night retention behavior, overheating of electromechanical equipment, and temperature disturbances caused by animal retention at the inlet and outlet of a hydropower station.

[0132] In some embodiments, the image semantic feature vector is input into the first anomaly recognition model constructed by a convolutional neural network and a spatial attention mechanism to generate a first anomaly recognition result. The specific technical process is as follows: The convolutional neural network is used to perform multi-layer convolution operations on the image semantic feature vector to extract the deep image feature map; based on the spatial attention mechanism, the response weights of each region in the deep image feature map are adjusted to generate an enhanced image feature map; the enhanced image feature map is judged for anomaly categories to generate a first anomaly recognition result.

[0133] In the specific implementation process, to generate the first anomaly recognition result, first, the convolutional neural network is used to perform multi-layer convolution operations on the image semantic feature vector, and by continuously extracting the spatial features and semantic features of the image, a deep image feature map is constructed. This deep image feature map has the ability to express features in multiple scales and levels, and can effectively depict the geometric shape and texture changes of structural targets in complex scenarios.

[0134] After obtaining the deep image feature map, the response intensity of each spatial position therein is adaptively adjusted based on the spatial attention mechanism. Define the response score of each pixel position in the deep image feature map as , and the spatial attention weight is obtained through normalization calculation:

[0135]

[0136] Among them, represents the spatial attention weight at position in the deep feature map of the image, represents the response score generated by the attention mechanism, represents the traversal index for all spatial positions, represents the response score at position ( ) in the deep feature map of the image.

[0137] Apply the above spatial attention weight to the feature vectors at the corresponding positions in the deep feature map of the image, perform a point-by-point weighted fusion operation on them, and generate an enhanced image feature map. During this process, the regions in the image that are highly correlated with abnormal features will obtain enhanced expression, while the feature responses in the background or invalid regions will be relatively weakened.

[0138] Subsequently, perform a global average pooling operation on the enhanced image feature map in the spatial dimension to obtain a feature compression vector , and input it into the abnormal determination network to calculate the first abnormal recognition result through the following expression :

[0139]

[0140] Among them, represents the weight parameter of the fully connected classification layer, represents the classification bias, and Softmax represents the normalization function, which is used to map the output to a multi-class probability distribution.

[0141] In some embodiments, input the heat distribution feature vector into the second abnormal recognition model constructed by a multi-layer perceptron network and a heat distribution feature encoding layer to generate a second abnormal recognition result, which specifically includes the following technical processes: input the central average temperature, boundary gradient value, and temperature dispersion in the heat distribution feature vector into the corresponding heat distribution feature encoding layers respectively; perform normalization and scale alignment processing on the input data in the heat distribution feature encoding layers to obtain a three-channel feature vector; splice the three-channel feature vectors in a preset order to generate a combined feature vector; input the combined feature vector into the multi-layer perceptron network for non-linear mapping processing, and generate a second abnormal recognition result based on the result of the non-linear mapping processing.

[0142] Among them, the thermal distribution feature encoding layer can represent a feature processing structure used for normalizing and making the scales consistent for different physical dimensions in the thermal distribution feature vector. The three-channel feature vector can represent the normalization processing results of three groups output by the thermal distribution feature encoding layer, corresponding to the central average temperature, the boundary gradient value, and the temperature dispersion respectively. Each channel represents the expression information of an independent thermal attribute dimension. The multi-layer perceptron network can represent a feedforward neural network module containing multiple fully-connected hierarchical structures, and is used to perform high-order combination and non-linear feature mapping operations on the input feature vector. By introducing activation functions, hidden units, and deep connection relationships, this network enables the thermal distribution features to have a stronger classification boundary expression ability in the high-dimensional space. The non-linear mapping process can represent the process of applying non-linear transformation to the combined feature vector based on the activation function in the multi-layer perceptron network. By normalizing and aligning the scales of the central average temperature, the boundary gradient value, and the temperature dispersion, different thermal attributes are given a unified expression scale to avoid feature imbalance; splicing the processed three-channel features into a combined feature vector and inputting it into the multi-layer perceptron network for non-linear mapping helps to discover the associations between complex thermal distribution patterns and improve the recognition accuracy and stability of thermal anomalies such as personnel staying and equipment overheating.

[0143] In other embodiments of the present disclosure, after obtaining the first anomaly recognition result and the second anomaly recognition result, corresponding warning information can also be generated based on the first anomaly recognition result and the second anomaly recognition result.

[0144] Specifically, the spatial position, anomaly type, anomaly level, and recognition confidence corresponding to each recognition result can be extracted, and combined with the time tag and monitoring position of the current task to construct a structured warning candidate entry. During the warning generation process, matching judgment can be made according to the engineering structure type corresponding to the anomaly type and its location. For example, when the first anomaly recognition result is slope cracks or structural water seepage, the type of hydraulic structure to which its occurrence location belongs, such as a dam, a diversion canal, a powerhouse, etc., and the current inspection frequency can be combined to determine whether the triggering conditions for structural safety warnings are met; when the second anomaly recognition result is personnel staying, abnormal high-temperature points, or activities within a restricted area, the use classification of the monitoring area and the set threshold can be combined to judge whether the warning conditions for behavior safety or thermal risk are met. After the judgment result meets the corresponding warning conditions, warning information including warning level, triggering conditions, engineering matters, disposal suggestions, associated responsible persons, and spatial identifiers can be generated and loaded onto the interaction interface for display and confirmation.

[0145] Figure 5The process schematic diagram of the hydropower project monitoring method according to some embodiments of the present disclosure is schematically shown. As shown in the figure, in the hydropower project monitoring method, intelligent flight path planning is used to set a drone flight path that is complete in coverage and reasonable in obstacle avoidance according to the terrain range and structural distribution of the hydropower project area. The flight path information is sent to the drone storage area through the remote control system to achieve dynamic synchronization with the inspection task. The drone hangar responds to the received instruction, starts the standby drone and completes the initialization. Subsequently, the drone autonomously operates and flies according to the flight path. During the flight process, color images, infrared images and laser point cloud data are collected in sequence to form a multi-modal perception information source. The collected data is sent to the data processing software in real time through the data transmission channel.

[0146] Entering the data processing stage, first, data preprocessing is completed, including basic operations such as image alignment, point cloud denoising and infrared thermal map standardization. Multi-period data registration is used to perform spatial and temporal consistency alignment of the current inspection data and historical data to ensure a coherent observation of the change trend of the target area. Subsequently, slope disease hidden danger identification and monitoring operations are performed. A three-dimensional point cloud model of the hydropower project area is constructed based on the laser point cloud data, and combined with the pose parameters of the color image and the infrared image, the corresponding pixel data and temperature data in the image are mapped to the three-dimensional point cloud model to generate a three-dimensional scene model with spatial alignment attributes. This model is further divided into multiple target sub-models according to the structural boundaries, and the image semantic feature vectors and thermal distribution feature vectors are respectively extracted from the pixel data and temperature data within each sub-model. The above features are input into the pre-trained anomaly recognition model for classification and recognition, and the anomaly recognition results are output and marked at the corresponding spatial positions of the three-dimensional scene model to achieve the spatial correlation expression of image information and structural anomalies.

[0147] The data processing software summarizes the above recognition results and inspection process data to generate a report. After the report is generated, it is synchronized to the user side. The user side displays information including three-dimensional models, anomaly distributions, heat map responses and historical comparisons, and supports user interaction operations such as anomaly confirmation, re-inspection scheduling and feedback on path re-planning suggestions. The whole process is carried out based on the closed-loop of automatic drone collection, multi-source data fusion, model recognition and visualization interaction. Compared with the traditional method that relies on manual inspection and two-dimensional image judgment, it significantly improves the monitoring accuracy, timeliness and spatial expression ability of areas such as hydraulic structures and slope structures, and is applicable to the hydropower project inspection scenarios with complex operating environments and diverse structural types.

[0148] Figure 6A schematic diagram of an interaction interface of a hydropower project monitoring system according to some embodiments of the present disclosure is schematically shown. Specifically, the interaction interface of the hydropower project monitoring system provides a warning setting and editing function for parameter configuration and task management of risk events identified by the system. The interface includes multiple fields such as warning level, trigger condition, related matters, warning content, responsible person, status flag, status, and instruction issuance. The warning level is used to set the response level for the current anomaly, such as general warning or emergency warning. The trigger condition field records the specific data basis that triggers the warning, such as the water level exceeding the set threshold. The related matters indicate the engineering structure or operation task corresponding to the warning, such as dam safety management. The warning content is used to describe the abnormal phenomenon identified by the system and the recommended handling method, such as immediately evacuating personnel. The responsible person field designates the person responsible for handling the warning. The status flag is used to mark the processing stage of the warning, such as processed, unprocessed, etc., and the status field indicates whether the task has been started. The instruction input area is used to fill in the dispatching instruction or disposal order, which can be automatically generated by the system or manually input.

[0149] Further, in other embodiments of the present disclosure, after obtaining the image semantic feature vector and the thermal distribution feature vector, the image semantic feature vector and the thermal distribution feature vector can also be used as joint inputs and input into the same deep learning network for anomaly recognition. To improve the discriminative ability of multi-modal feature fusion, a fusion strategy combining structural prior guidance and modal cross-attention mechanism is adopted to enhance the collaborative expression ability between different modalities.

[0150] Specifically, first, for the current target sub-model to be recognized, according to its structural type, such as dam body, slope, water area, protective structure, and road, etc., a structural prior label vector is constructed, and it is mapped into a structural prior embedding vector through an embedding encoder as auxiliary information for model structure perception.

[0151] Subsequently, linear mapping and normalization processing are respectively performed on the image semantic feature vector and the thermal distribution feature vector to make their feature dimensions consistent and numerical ranges unified. After obtaining the aligned multi-modal features, a modal-guided cross-attention mechanism is constructed: a first attention channel is established with the image semantic feature as the main modality and the thermal distribution feature as the auxiliary modality, and a second attention channel is constructed with the image semantic feature as the auxiliary modality and the thermal distribution feature as the main modality. At the same time, the structural prior embedding vector is introduced into both attention channels as background semantic guidance.

[0152] Through the weighted summation and attention weight learning mechanism, multi-modal fusion features are extracted , which can be formally expressed as:

[0153]

[0154] Wherein, Represents the image semantic feature vector, Represents the heat distribution feature vector, Represents the structural prior embedding vector, Is a learnable parameter matrix.

[0155] After obtaining the multi-modal fusion features, input them into the parallel anomaly category branch prediction network. This parallel anomaly category branch prediction network contains multiple classification branches, and each branch corresponds to an anomaly category sub-domain, such as a structural anomaly branch, a heat anomaly branch, and a behavior anomaly branch, etc. Exemplarily, the structural anomaly branch can be used to identify anomalies such as concrete cracks, slope slips, structural water seepage, and collapses, mainly dominated by image semantic features and fusing a small amount of temperature anomaly information; the heat anomaly branch can be used to identify high-temperature concentration areas, illegal personnel staying, abnormal heat source approaching, etc., dominated by heat distribution features and combining image structure boundary information as an auxiliary; the behavior anomaly branch can be used to identify behavior-type targets such as wild swimming, fishing, and crossing the boundary, and it is necessary to combine image contours, movement trajectories, and local heat aggregation patterns. An independent modal attention adjustment structure is set inside each branch to assign different dominant weights to the image and heat features according to the modal sensitivity of a specific anomaly type. For example, in the structural anomaly branch, by increasing the weight of the image feature channels, the influence of the heat channels is weakened; while in the heat anomaly branch, it is the opposite.

[0156] Finally, each branch outputs the anomaly confidence under the corresponding category, generates the anomaly recognition result through the maximum probability or a set threshold, and writes the recognition result back to the corresponding spatial position in the 3D scene model.

[0157] It should be noted that although the steps of the method in the present disclosure are described in a specific order in the drawings, however, this does not require or imply that these steps must be executed in that specific order, or that all the shown steps must be executed to achieve the desired result. Additionally or alternatively, some steps can be omitted, multiple steps can be combined into one step for execution, and / or one step can be decomposed into multiple steps for execution, etc.

[0158] In addition, in this exemplary embodiment, a hydropower project monitoring system based on drone inspection and 3D modeling is also provided. Referring to Figure 7 As shown, the hydropower project monitoring system 700 based on drone inspection and 3D modeling includes: an image acquisition module 710, a 3D reconstruction module 720, a vector extraction module 730, and an anomaly recognition module 740. Among them:

[0159] The image acquisition module 710 can be used to fly a drone along a preset flight path in the hydropower project area and collect color images, infrared images, and laser point cloud data;

[0160] The 3D reconstruction module 720 can be used to construct a 3D point cloud model corresponding to the hydropower project area based on the lidar point cloud data, and map the pixel data and temperature data to the 3D point cloud model based on the pose parameters of the color image and the infrared image to generate a 3D scene model;

[0161] The vector extraction module 730 can be used to perform regional segmentation operations on the 3D scene model, obtain the target sub-model, and respectively extract features from the pixel data and temperature data in the target sub-model to generate an image semantic feature vector and a thermal distribution feature vector;

[0162] The anomaly recognition module 740 can be used to perform anomaly recognition on the image semantic feature vector and the thermal distribution feature vector through a pre-trained anomaly recognition model to obtain the corresponding anomaly recognition result, and add the anomaly recognition result to the corresponding spatial position of the 3D scene model.

[0163] The specific details of each module of the above hydropower project monitoring system based on UAV inspection and 3D modeling have been described in detail in the corresponding hydropower project monitoring method based on UAV inspection and 3D modeling, so they will not be elaborated here.

[0164] It should be noted that although several modules or units of the hydropower project monitoring system based on UAV inspection and 3D modeling are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of the two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0165] In addition, in the exemplary embodiments of the present disclosure, an electronic device capable of implementing the above hydropower project monitoring method based on UAV inspection and 3D modeling is also provided.

[0166] Those skilled in the art can understand that various aspects of the present disclosure can be implemented as a system, a method, or a program product. Therefore, various aspects of the present disclosure can be specifically implemented in the following forms, namely: a complete hardware embodiment, a complete software embodiment (including firmware, microcode, etc.), or an embodiment combining hardware and software aspects, which can be collectively referred to as "circuit", "module", or "system" here.

[0167] Next, refer to Figure 8 to describe the electronic device 800 according to the embodiments of the present disclosure. Figure 8 The illustrated electronic device 800 is only an example and should not impose any limitation on the functions and usage scope of the embodiments of the present disclosure.

[0168] As Figure 8As shown, the electronic device 800 is presented in the form of a general-purpose computing device. The components of the electronic device 800 may include, but are not limited to: at least one of the above-mentioned processing units 810, at least one of the above-mentioned storage units 820, a bus 830 connecting different system components (including the storage unit 820 and the processing unit 810), and a display unit 840.

[0169] Among them, the storage unit stores program code, and the program code can be executed by the processing unit 810, so that the processing unit 810 executes the steps according to various exemplary embodiments of the present disclosure described in the "Exemplary Method" section of the present specification above.

[0170] The storage unit 820 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 821 and / or a cache storage unit 822, and may further include a read-only storage unit (ROM) 823.

[0171] The storage unit 820 may also include a program / utility 824 having a set (at least one) of program modules 825. Such program modules 825 include, but are not limited to: an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include the implementation of a network environment.

[0172] The bus 830 may represent one or more of several types of bus structures, including a storage unit bus or a storage unit controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0173] The electronic device 800 can also communicate with one or more external devices 870 (such as a keyboard, a pointing device, a Bluetooth device, etc.), can also communicate with one or more devices that enable a user to interact with the electronic device 800, and / or can communicate with any device that enables the electronic device 800 to communicate with one or more other computing devices (such as a router, a modem, etc.). Such communication can be carried out through an input / output (I / O) interface 850. And, the electronic device 800 can also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 860. As shown in the figure, the network adapter 860 communicates with other modules of the electronic device 800 through the bus 830. It should be understood that although not shown in the figure, other hardware and / or software modules can be used in combination with the electronic device 800, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0174] From the description of the above embodiments, those skilled in the art can easily understand that the exemplary embodiments described herein can be implemented by software, or by a combination of software and necessary hardware. Therefore, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a portable hard drive, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.

[0175] In an exemplary embodiment of the present disclosure, there is also provided a computer-readable storage medium having a program product stored thereon that can implement the above methods of this specification. In some possible embodiments, various aspects of the present disclosure can also be implemented in the form of a program product, which includes program code. When the program product runs on a terminal device, the program code is used to cause the terminal device to execute the steps according to various exemplary embodiments of the present disclosure described in the above "Exemplary Method" section of this specification.

[0176] Reference Figure 9 As shown, a program product 900 for implementing the above-described method for monitoring a hydropower project based on drone inspection and 3D modeling according to an embodiment of the present disclosure is described. It can be a portable compact disc read-only memory (CD-ROM) and includes program code, and can run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited thereto. In this document, the readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0177] The program product can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can, for example, but not be limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0178] A computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which readable program code is carried. Such a propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. The readable signal medium may also be any readable medium other than a readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device.

[0179] The program code contained on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0180] The program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, etc., and also including conventional procedural programming languages such as the "C" language or similar programming languages. The program code may be executed entirely on the user's computing device, partially on the user's device, executed as a stand-alone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., by using an Internet service provider to connect through the Internet).

[0181] In addition, the above-mentioned drawings are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present disclosure, and are not for limiting purposes. It is easy to understand that the processes shown in the above-mentioned drawings do not indicate or limit the chronological order of these processes. Additionally, it is also easy to understand that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0182] Through the description of the above embodiments, those skilled in the art can easily understand that the example embodiments described herein can be implemented by software or by a combination of software and necessary hardware. Therefore, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which may be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which may be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.

[0183] Other embodiments of the present disclosure will be readily apparent to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include known common general knowledge or conventional technical means in the technical field not disclosed in the present disclosure. The specification and examples are only to be considered as exemplary, and the true scope and spirit of the present disclosure are pointed out by the claims.

[0184] It should be understood that the present disclosure is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present disclosure is only limited by the appended claims.

Claims

1. A monitoring method for hydropower projects based on drone inspection and 3D modeling, characterized in that Including: Utilize a drone to fly along a preset flight path within a hydropower project area and collect color images, infrared images, and laser point cloud data; Construct a three-dimensional point cloud model corresponding to the hydropower project area based on the laser point cloud data, and map pixel data and temperature data to the three-dimensional point cloud model based on the pose parameters of the color image and the infrared image to generate a three-dimensional scene model; Perform regional segmentation operations on the three-dimensional scene model to obtain target sub-models, and respectively extract features from the pixel data and the temperature data in the target sub-models to generate an image semantic feature vector and a heat distribution feature vector; Perform anomaly recognition on the image semantic feature vector and the heat distribution feature vector through a pre-trained anomaly recognition model to obtain corresponding anomaly recognition results, and add the anomaly recognition results to the spatial positions corresponding to the three-dimensional scene model; Use the spatial positions corresponding to the anomaly recognition results in the three-dimensional scene model as inspection points, establish the corresponding relationship between the inspection points and the anomaly recognition results, determine the starting point according to the starting position of the drone, and construct a node set based on the starting point and the inspection points; Calculate the spatial distances between the nodes in the node set, obtain a plurality of connection edges according to the comparison results of the spatial distances and a preset distance threshold; extract point cloud data from the spatial regions corresponding to each connection edge in the three-dimensional point cloud model, perform grid processing on the path point cloud data, and obtain the elevation change information and obstacle distribution information corresponding to each connection edge based on the height difference and point density characteristics within each grid; assign a passage cost value to each connection edge according to the elevation change information and the obstacle distribution information, and construct an abnormal inspection path according to the passage cost value; Control the drone to perform image acquisition along the abnormal inspection path, and play warning voice information corresponding to the preset anomaly recognition result when reaching the inspection point corresponding to the preset anomaly recognition result.

2. The hydroelectric engineering monitoring method based on drone inspection and 3D modeling according to claim 1, characterized in that The constructing the abnormal inspection path according to the passage cost value includes: Establish a path connection graph according to the node set and the plurality of connection edges; In the path connection graph, starting from the starting point, traverse all inspection points, and determine the node connection order with the smallest total passage cost value according to the passage cost value corresponding to each connection edge; Construct the abnormal inspection path according to the node connection order.

3. The hydroelectric engineering monitoring method based on drone inspection and 3D modeling according to claim 1, characterized in that, The mapping the pixel data and the temperature data to the three-dimensional point cloud model based on the pose parameters of the color image and the infrared image includes: Obtain the pose parameters of the color image and the infrared image, and determine the transformation matrix of the infrared image relative to the color image reference coordinate system based on the pose parameters; Transform the infrared image to the color image reference coordinate system according to the transformation matrix to generate an aligned infrared image; Map the pixel data corresponding to the color image and the temperature data corresponding to the aligned infrared image to the three-dimensional point cloud model according to the corresponding relationship between the color image, the infrared image, and the laser point cloud data.

4. The hydropower engineering monitoring method based on drone inspection and 3D modeling according to claim 1, characterized in that The generation process of the image semantic feature vector includes: Converting the pixel data in the target sub-model into an image tensor; Performing multiple convolutional operations and downsampling operations on the image tensor to extract hierarchical semantic features of the image tensor; Performing feature compression on the hierarchical semantic features in the channel dimension to obtain a single semantic feature; Performing global average pooling on the single semantic feature and using the pooling result as the image semantic feature vector.

5. The hydropower project monitoring method based on drone inspection and 3D modeling according to claim 1, characterized in that, The generation process of the heat distribution feature vector includes: Performing normalization processing on the temperature data to generate a normalized temperature map; In the normalized temperature map, calculating a thermal response mask based on the ratio of the local temperature peak to the surrounding temperature and generating a heat response region; Performing regional temperature consistency analysis on the heat response region to obtain the central average temperature, boundary gradient value, and internal temperature dispersion of the heat response region; Constructing the heat distribution feature vector based on the central average temperature, boundary gradient value, and internal temperature dispersion.

6. The hydroelectric engineering monitoring method based on drone inspection and 3D modeling according to claim 5, characterized in that, The step of calculating a thermal response mask based on the ratio of the local temperature peak to the surrounding temperature in the normalized temperature map and generating a heat response region includes: Extracting local temperature peak points in the normalized temperature map and constructing a neighborhood window with a fixed size based on each peak point; Calculating the average temperature within each neighborhood window and calculating the ratio of the average temperature to the temperature value of the corresponding peak point; Taking the pixel points with a ratio exceeding a preset thermal response threshold as response points; Performing regional aggregation processing on adjacent response points to generate a thermal response mask and generating a corresponding heat response region based on the thermal response mask.

7. The method for monitoring hydropower projects based on drone inspection and 3D modeling according to claim 1, characterized in that, The step of performing anomaly recognition on the image semantic feature vector and the heat distribution feature vector through a pre-trained anomaly recognition model to obtain corresponding anomaly recognition results includes: Inputting the image semantic feature vector into a first anomaly recognition model constructed by a convolutional neural network and a spatial attention mechanism to generate a first anomaly recognition result; Inputting the heat distribution feature vector into a second anomaly recognition model constructed by a multi-layer perceptron network and a heat distribution feature encoding layer to generate a second anomaly recognition result; Among them, the first anomaly recognition result includes any one or more of slope deformation, concrete cracks, landslides, structural water seepage, collapses, guardrail damage, and floating objects on the water surface, and the second anomaly recognition result includes any one or more of personnel retention, swimming behavior, and abnormal high temperature points.

8. The method for monitoring hydropower projects based on UAV inspection and 3D modeling according to claim 7, characterized in that, The step of inputting the image semantic feature vector into a first anomaly recognition model constructed by a convolutional neural network and a spatial attention mechanism to generate a first anomaly recognition result includes: Using a convolutional neural network to perform multi-layer convolutional operations on the image semantic feature vector to extract an image deep feature map; Adjusting the response weights of each region in the image deep feature map based on the spatial attention mechanism to generate an enhanced image feature map; Performing anomaly category determination on the enhanced image feature map to generate the first anomaly recognition result.

9. The hydroelectric engineering monitoring method based on drone inspection and three-dimensional modeling according to claim 7, characterized in that The step of inputting the heat distribution feature vector into a second anomaly recognition model constructed by a multi-layer perceptron network and a heat distribution feature encoding layer to generate a second anomaly recognition result includes: Input the central average temperature, boundary gradient value, and temperature dispersion in the thermal distribution feature vector into the corresponding thermal distribution feature encoding layer respectively; Perform normalization and scale alignment processing on the input data in the thermal distribution feature encoding layer to obtain a three-channel feature vector; Perform splicing processing on the three-channel feature vector in a preset order to generate a combined feature vector; Input the combined feature vector into the multi-layer perceptron network for non-linear mapping processing, and generate the second anomaly recognition result based on the result of the non-linear mapping processing.

10. A hydropower project monitoring system based on drone inspection and 3D modeling, which is used to implement the hydropower project monitoring method based on drone inspection and 3D modeling as described in any one of claims 1-9, and is characterized in that, Including: An image acquisition module, configured to fly along a preset flight path in the hydropower project area by using a drone, and acquire color images, infrared images, and laser point cloud data; A three-dimensional reconstruction module, configured to construct a three-dimensional point cloud model corresponding to the hydropower project area according to the laser point cloud data, and map pixel data and temperature data to the three-dimensional point cloud model based on the pose parameters of the color image and the infrared image to generate a three-dimensional scene model; A vector extraction module, configured to perform regional segmentation operations on the three-dimensional scene model, obtain a target sub-model, and respectively extract features from the pixel data and the temperature data in the target sub-model to generate an image semantic feature vector and a thermal distribution feature vector; An anomaly recognition module, configured to perform anomaly recognition on the image semantic feature vector and the thermal distribution feature vector through a pre-trained anomaly recognition model to obtain corresponding anomaly recognition results, and add the anomaly recognition results to the corresponding spatial positions of the three-dimensional scene model.

Images