Hydropower engineering monitoring method and system based on unmanned aerial vehicle patrol and three-dimensional modeling

Abstract

The invention provides a hydropower engineering monitoring method and system based on unmanned aerial vehicle patrol and three-dimensional modeling, and relates to the technical field of image recognition. The method comprises the following steps: acquiring a color image, an infrared image and laser point cloud data; constructing a three-dimensional point cloud model corresponding to the hydropower engineering area according to the laser point cloud data, and mapping the pixel data and the temperature data to the three-dimensional point cloud model to generate a three-dimensional scene model; performing region segmentation operation on the three-dimensional scene model to obtain a target sub-model, performing feature extraction on the target sub-model, and generating an image semantic feature vector and a heat distribution feature vector; and performing anomaly recognition on the image semantic feature vector and the heat distribution feature vector through a pre-trained anomaly recognition model to obtain a corresponding anomaly recognition result, and adding the anomaly recognition result to a spatial position corresponding to the three-dimensional scene model. According to the scheme, the spatial expression ability of the image data and the accuracy of anomaly recognition in the hydropower engineering monitoring process can be improved.

Description

Background Art

[0002] In the inspection and monitoring work of related hydropower projects, image information is mainly collected through manual shooting, fixed-point videography or aerial photography, and is displayed and analyzed in a two-dimensional form. Although such means can provide intuitive pictures, their processing methods usually only stay on the images themselves, without establishing a spatial correspondence between the image data and the on-site structure, making it difficult to perform subsequent anomaly localization.

[0003] On the other hand, in the related technology for image analysis, it mainly relies on manual observation and empirical judgment to identify abnormal phenomena. The operation process is time-consuming and easily affected subjectively. It not only has low efficiency and is prone to omissions, but also is difficult to meet the actual needs of high-frequency inspections and multi-target parallel recognition. In a complex operation environment, the limitations of manual recognition are more obvious, affecting the safe operation and intelligent management level of hydropower projects.

[0004] Therefore, there is still room for improvement in the existing hydropower project monitoring technology in terms of the spatial expression of image data and the ability to identify anomalies. Summary of the Invention

[0005] An object of the embodiments of the present disclosure is to provide a hydropower project monitoring method based on drone inspection and three-dimensional modeling, a hydropower project monitoring system based on drone inspection and three-dimensional modeling, an electronic device, and a computer-readable storage medium, which can construct a spatially aligned monitoring view in an actual three-dimensional scene, and use an anomaly recognition model for discriminant analysis, thereby improving the spatial expression ability of image data and the accuracy of anomaly recognition during the hydropower project monitoring process.

[0006] Other features and advantages of the present disclosure will become apparent through the following detailed description, or be learned in part through the practice of the present disclosure.

[0007] According to the first aspect of the embodiments of the present disclosure, there is provided a hydropower project monitoring method based on drone inspection and three-dimensional modeling, including: flying a drone along a preset flight path within a hydropower project area, and collecting color images, infrared images and laser point cloud data; constructing a three-dimensional point cloud model corresponding to the hydropower project area according to the laser point cloud data, and mapping pixel data and temperature data to the three-dimensional point cloud model based on the pose parameters of the color image and the infrared image to generate a three-dimensional scene model; performing a regional segmentation operation on the three-dimensional scene model to obtain a target sub-model, and respectively extracting features from the pixel data and the temperature data in the target sub-model to generate an image semantic feature vector and a heat distribution feature vector; performing anomaly recognition on the image semantic feature vector and the heat distribution feature vector through a pre-trained anomaly recognition model to obtain corresponding anomaly recognition results, and adding the anomaly recognition results to the corresponding spatial positions of the three-dimensional scene model.

[0008] In some exemplary embodiments of the present disclosure, based on the foregoing solution, the above-mentioned method for monitoring hydropower projects based on drone patrol and 3D modeling further includes: using the spatial position corresponding to the abnormal recognition result in the 3D scene model as an inspection point, establishing a correspondence between the inspection point and the abnormal recognition result, and generating an abnormal inspection path based on the inspection point; controlling the drone to collect images along the abnormal inspection path, and when reaching the inspection point corresponding to the preset abnormal recognition result, playing the warning voice information corresponding to the preset abnormal recognition result.

[0009] In some exemplary embodiments of the present disclosure, based on the foregoing solution, the generating the abnormal inspection path based on the inspection point includes: determining a starting point according to the starting position of the drone, and constructing a node set according to the starting point and the inspection point; calculating the spatial distance between each node in the node set, and obtaining a plurality of connection edges according to the comparison result between the spatial distance and a preset distance threshold; extracting point cloud data from the spatial region corresponding to each connection edge in the 3D point cloud model; performing grid processing on the path point cloud data, and obtaining the elevation change information and obstacle distribution information corresponding to each connection edge based on the height difference and point density characteristics within each grid; assigning a passage cost value to each connection edge according to the elevation change information and obstacle distribution information, and constructing the abnormal inspection path according to the passage cost value.

[0010] In some exemplary embodiments of the present disclosure, based on the foregoing solution, the constructing the abnormal inspection path according to the passage cost value includes: establishing a path connection graph according to the node set and the plurality of connection edges; in the path connection graph, starting from the starting point, traversing all inspection points, and determining the node connection order with the minimum total passage cost value according to the passage cost value corresponding to each connection edge; constructing the abnormal inspection path according to the node connection order.

[0011] In some exemplary embodiments of the present disclosure, based on the foregoing solution, the mapping the pixel data and temperature data to the 3D point cloud model based on the pose parameters of the color image and the infrared image includes: obtaining the pose parameters of the color image and the infrared image, and determining a transformation matrix of the infrared image relative to the color image reference coordinate system based on the pose parameters; transforming the infrared image to the color image reference coordinate system according to the transformation matrix to generate an aligned infrared image; mapping the pixel data corresponding to the color image and the temperature data corresponding to the aligned infrared image to the 3D point cloud model according to the correspondence between the color image, the infrared image, and the laser point cloud data.

[0012] In some exemplary embodiments of the present disclosure, based on the foregoing solution, the generation process of the image semantic feature vector includes: converting the pixel data in the target sub-model into an image tensor; performing multiple convolution operations and downsampling operations on the image tensor to extract the hierarchical semantic features of the image tensor; performing feature compression on the hierarchical semantic features in the channel dimension to obtain a single semantic feature; performing global average pooling on the single semantic feature, and using the pooling result as the image semantic feature vector.

[0013] In some exemplary embodiments of the present disclosure, based on the foregoing solution, the generation process of the heat distribution feature vector includes: performing normalization processing on the temperature data to generate a normalized temperature map; in the normalized temperature map, calculating a thermal response mask based on the ratio of the local temperature peak to the surrounding temperature, and generating a heat response region; performing regional temperature consistency analysis on the heat response region to obtain the central average temperature, boundary gradient value, and internal temperature dispersion degree of the heat response region; constructing the heat distribution feature vector based on the central average temperature, boundary gradient value, and internal temperature dispersion degree.

[0014] In some exemplary embodiments of the present disclosure, based on the foregoing solution, the calculating a thermal response mask based on the ratio of the local temperature peak to the surrounding temperature in the normalized temperature map and generating a heat response region includes: extracting local temperature peak points in the normalized temperature map, and constructing a neighborhood window with a fixed size based on each peak point; calculating the average temperature within each neighborhood window, and calculating the ratio of the average temperature to the temperature value of the corresponding peak point; using the pixel points with a ratio exceeding a preset thermal response threshold as response points; performing regional aggregation processing on adjacent response points to generate a thermal response mask, and generating a corresponding heat response region based on the thermal response mask.

[0015] In some exemplary embodiments of the present disclosure, based on the foregoing solution, the performing anomaly recognition on the image semantic feature vector and the heat distribution feature vector through a pre-trained anomaly recognition model to obtain corresponding anomaly recognition results includes: inputting the image semantic feature vector into a first anomaly recognition model constructed by a convolutional neural network and a spatial attention mechanism to generate a first anomaly recognition result; inputting the heat distribution feature vector into a second anomaly recognition model constructed by a multi-layer perceptron network and a heat distribution feature encoding layer to generate a second anomaly recognition result; wherein, the first anomaly recognition result includes any one or more of slope deformation, concrete cracks, landslides, structural water seepage, collapses, guardrail damage, and floating objects on the water surface, and the second anomaly recognition result includes any one or more of personnel retention, swimming behavior, and abnormal high temperature points.

[0016] In some exemplary embodiments of the present disclosure, based on the foregoing solution, the step of inputting the image semantic feature vector into a first anomaly recognition model constructed by a convolutional neural network and a spatial attention mechanism to generate a first anomaly recognition result includes: performing multi-layer convolutional operations on the image semantic feature vector by using the convolutional neural network to extract a deep image feature map; adjusting the response weights of each region in the deep image feature map based on the spatial attention mechanism to generate an enhanced image feature map; and performing anomaly category determination on the enhanced image feature map to generate the first anomaly recognition result.

[0017] In some exemplary embodiments of the present disclosure, based on the foregoing solution, the step of inputting the heat distribution feature vector into a second anomaly recognition model constructed by a multi-layer perceptron network and a heat distribution feature encoding layer to generate a second anomaly recognition result includes: respectively inputting the central average temperature, the boundary gradient value, and the temperature dispersion in the heat distribution feature vector into the corresponding heat distribution feature encoding layers; performing normalization and scale alignment processing on the input data in the heat distribution feature encoding layers to obtain a three-channel feature vector; splicing the three-channel feature vectors in a preset order to generate a combined feature vector; inputting the combined feature vector into the multi-layer perceptron network for non-linear mapping processing, and generating the second anomaly recognition result based on the result of the non-linear mapping processing.

[0018] According to a second aspect of the embodiments of the present disclosure, there is provided a hydropower project monitoring system based on drone inspection and three-dimensional modeling, including: an image acquisition module, configured to fly a drone along a preset flight path in a hydropower project area and acquire color images, infrared images, and laser point cloud data; a three-dimensional reconstruction module, configured to construct a three-dimensional point cloud model corresponding to the hydropower project area according to the laser point cloud data, and map pixel data and temperature data to the three-dimensional point cloud model based on the pose parameters of the color image and the infrared image to generate a three-dimensional scene model; a vector extraction module, configured to perform regional segmentation operations on the three-dimensional scene model to obtain a target sub-model, and respectively extract features from the pixel data and the temperature data in the target sub-model to generate an image semantic feature vector and a heat distribution feature vector; and an anomaly recognition module, configured to perform anomaly recognition on the image semantic feature vector and the heat distribution feature vector through a pre-trained anomaly recognition model to obtain corresponding anomaly recognition results, and add the anomaly recognition results to spatial positions corresponding to the three-dimensional scene model.

[0019] According to a third aspect of the embodiments of the present disclosure, there is provided an electronic device, including: a processor; and a memory, on which computer-readable instructions are stored, and when the computer-readable instructions are executed by the processor, the hydropower project monitoring method based on drone inspection and three-dimensional modeling as in the first aspect is implemented.

[0020] According to a fourth aspect of the embodiments of the present disclosure, there is provided a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, it implements the hydropower project monitoring method based on drone inspection and three-dimensional modeling as in the first aspect.

[0021] The technical solutions provided by the embodiments of the present disclosure may include the following beneficial effects: In the hydropower project monitoring method based on drone inspection and three-dimensional modeling in the exemplary embodiments of the present disclosure, on the one hand, by controlling the drone to collect color images, infrared images, and laser point cloud data in the hydropower project area along a preset path, compared with the image acquisition method that relies on manual or fixed camera methods, the image coverage range is improved. On the other hand, a three-dimensional point cloud model of the hydropower project area is constructed based on the laser point cloud data, and in combination with the pose parameters of the color image and the infrared image, the pixel data and temperature data in the image are mapped to the model, realizing the integrated modeling and visual association of image information and spatial structure, and solving the problem that the observation results cannot be spatially located in the related art, which is convenient for result presentation.

[0022] On the other hand, the generated three-dimensional scene model is regionally segmented, and feature extraction operations are respectively performed on the pixel data and temperature data in the target sub-model to obtain an image semantic feature vector and a thermal distribution feature vector, which can input the observation information into a pre-trained anomaly recognition model in a structured form. This method realizes the automatic recognition of hydropower project anomalies and solves the problems of the deficiencies of the traditional manual method in terms of recognition efficiency, accuracy, and scene adaptability. Further, the recognition result is located at the corresponding spatial position in the three-dimensional scene model, making the display relationship between the recognition result and the actual position clearer and facilitating the quick understanding of the anomaly distribution. Thus, it can improve the spatial expression ability of image data and the accuracy of anomaly recognition to a certain extent during the hydropower project monitoring process.

[0023] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The accompanying drawings here are incorporated into the specification and form a part of the specification, showing embodiments consistent with the present disclosure, and are used together with the specification to explain the principles of the present disclosure. Obviously, the accompanying drawings in the following description are only some embodiments of the present disclosure, and those of ordinary skill in the art can obtain other drawings without creative efforts based on these drawings.

[0025] Figure 1 A flowchart of the hydropower project monitoring method based on drone inspection and three-dimensional modeling according to some embodiments of the present disclosure is schematically shown.

[0026] Figure 2 Schematically shows a system structure diagram to which the hydropower project monitoring method in the embodiments of the present disclosure can be applied.

[0027] Figure 3 Schematically shows a flowchart of generating an abnormal inspection path according to some embodiments of the present disclosure.

[0028] Figure 4 Schematically shows a flowchart of generating a thermal distribution feature vector according to some embodiments of the present disclosure.

[0029] Figure 5 Schematically shows a process diagram of the hydropower project monitoring method according to some embodiments of the present disclosure.

[0030] Figure 6 Schematically shows a schematic diagram of an interaction interface of a hydropower project monitoring system according to some embodiments of the present disclosure.

[0031] Figure 7 Schematically shows a schematic diagram of a hydropower project monitoring system based on drone inspection and 3D modeling according to some embodiments of the present disclosure.

[0032] Figure 8 Schematically shows a schematic diagram of a computer system of an electronic device according to some embodiments of the present disclosure.

[0033] Figure 9 Schematically shows a schematic diagram of a computer-readable storage medium according to some embodiments of the present disclosure.

[0034] In the drawings, the same or corresponding reference numerals indicate the same or corresponding parts. Detailed implementation manners

[0035] Here, the exemplary embodiments will be described in detail, and the examples are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings indicate the same or similar elements. The implementation manners described in the following exemplary embodiments do not represent all implementation manners consistent with this specification. On the contrary, they are merely examples of devices and methods consistent with some aspects of this specification as detailed in the appended claims.

[0036] The terms used in this specification are for the purpose of describing particular embodiments only and are not intended to limit this specification. The singular forms "a", "the", and "said" used in this specification and the appended claims are also intended to include the plural forms unless the context clearly dictates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0037] Example embodiments will now be described more fully with reference to the accompanying drawings. However, the example embodiments can be implemented in various forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the example embodiments to those skilled in the art.

[0038] In addition, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of the embodiments of the present disclosure. However, those skilled in the art will realize that the technical solutions of the present disclosure can be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. may be used. In other cases, well-known methods, devices, implementations, or operations are not shown or described in detail to avoid obscuring aspects of the present disclosure.

[0039] In addition, the accompanying drawings are only schematic diagrams and are not necessarily drawn to scale. The block diagrams shown in the drawings are only functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0040] In the present exemplary embodiment, a method for monitoring hydropower projects based on drone inspection and 3D modeling is first provided. Figure 1 A flowchart of a method for monitoring hydropower projects based on drone inspection and 3D modeling according to some embodiments of the present disclosure is schematically shown. Refer to Figure 1 As shown, the method for monitoring hydropower projects based on drone inspection and 3D modeling may include the following steps: Step S110, flying a drone along a preset flight path within the hydropower project area and collecting color images, infrared images, and laser point cloud data; Step S120, constructing a 3D point cloud model corresponding to the hydropower project area based on the laser point cloud data, and mapping pixel data and temperature data to the 3D point cloud model based on the pose parameters of the color image and the infrared image to generate a 3D scene model; Step S130: Perform a regional segmentation operation on the three-dimensional scene model to obtain the target sub-model, and respectively extract features from the pixel data and temperature data in the target sub-model to generate an image semantic feature vector and a thermal distribution feature vector. Step S140: Use the pre-trained anomaly recognition model to perform anomaly recognition on the image semantic feature vector and the thermal distribution feature vector to obtain the corresponding anomaly recognition result, and add the anomaly recognition result to the corresponding spatial position of the three-dimensional scene model.

[0041] According to the hydropower project monitoring method based on UAV inspection and three-dimensional modeling in this exemplary embodiment, on the one hand, by controlling the UAV to collect color images, infrared images and laser point cloud data along the preset path in the hydropower project area, compared with the image collection method relying on manual or fixed camera methods, the image coverage range is improved. On the other hand, based on the laser point cloud data, a three-dimensional point cloud model of the hydropower project area is constructed, and combined with the pose parameters of the color image and the infrared image, the pixel data and temperature data in the image are mapped to the model, realizing the integrated modeling and visual association of image information and spatial structure. It not only solves the problem that the observation results cannot be spatially located in the related technology, but also gives the image content a clear spatial attribution relationship, which is convenient for result presentation. On the one hand, the generated three-dimensional scene model is segmented into regions, and feature extraction operations are respectively performed on the pixel data and temperature data in the target sub-model to obtain an image semantic feature vector and a thermal distribution feature vector, which can input the observation information into the pre-trained anomaly recognition model in a structured form. This method realizes the automatic recognition of hydropower project anomalies and solves the problems of the deficiencies of traditional manual methods in terms of recognition efficiency, accuracy and scene adaptability. Further, the recognition result is located at the corresponding spatial position in the three-dimensional scene model, making the display relationship between the recognition result and the actual position clearer, which is convenient for quickly understanding the anomaly distribution situation based on the model. Thus, it can improve the spatial expression ability of image data and the accuracy of anomaly recognition to a certain extent during the hydropower project monitoring process.

[0042] Next, the hydropower project monitoring method based on UAV inspection and three-dimensional modeling in this exemplary embodiment will be further described.

[0043] Step S110: Use the UAV to fly along the preset flight path in the hydropower project area and collect color images, infrared images and laser point cloud data.

[0044] Among them, the hydropower project area can represent the geographical space range where the hydropower station engineering facilities are located, including but not limited to the reservoir storage area, the dam body of the dam, the diversion canal, the flood discharge channel, the slope, and the surrounding protection area, etc. The color image can represent the two-dimensional image data reflecting the color and texture information of the surface of the measured target obtained by the visible light image sensor. The infrared image can represent the two-dimensional image data generated based on the thermal radiation intensity of the target surface collected by the infrared thermal imaging sensor, reflecting the temperature distribution characteristics of the measured object within different wavelength ranges, and can be used to judge the states such as thermal anomalies, local heating, and seepage areas. The lidar point cloud data can represent the three-dimensional coordinate set of a large number of spatial points in the target area obtained by the lidar device, and can be used to describe the spatial structure characteristics such as terrain undulations, building outlines, and slope structures.

[0045] When collecting the above color images, infrared images, and lidar point cloud data, timestamp information and corresponding aircraft attitude parameters can be attached, and the spatio-temporal correspondence relationship between the data can be established through the track and sensor field-of-view parameters, so as to ensure the spatial synchronization and fusion feasibility of the multi-source data in subsequent processing.

[0046] Step S120: Construct a three-dimensional point cloud model corresponding to the hydropower project area based on the lidar point cloud data, and map the pixel data and temperature data to the three-dimensional point cloud model based on the pose parameters of the color image and the infrared image to generate a three-dimensional scene model.

[0047] Among them, the three-dimensional point cloud model can represent the spatial structure expression model constructed based on the lidar point cloud data, which is composed of multiple three-dimensional discrete points with spatial coordinate information. This model can reflect the outlines and terrain undulations of physical structures such as the dam body and slope in the hydropower project area. The pose parameter can represent the spatial position and orientation information of the image sensor relative to the unified reference coordinate system when collecting color image and infrared image data, and can include the position vector, translation matrix, and attitude angle or rotation matrix, which are used to establish the geometric mapping relationship between the image data and the three-dimensional space. The pixel data can represent the RGB values of the image reflecting the color and texture of the surface of the target area obtained through the color image. The temperature data can represent the thermal radiation intensity value corresponding to each image pixel point collected through the infrared image, which is used to characterize the temperature distribution state of the target. The three-dimensional scene model can represent the fusion model formed by projecting the pixel data in the color image and the temperature data in the infrared image onto the three-dimensional point cloud model based on the pose parameters respectively. On the basis of maintaining the original point cloud geometric structure, this model attaches visual texture and thermal radiation attributes to each spatial point, realizing the fusion expression of multi-modal data in the unified spatial coordinate system.

[0048] Step S130: Perform a regional segmentation operation on the three-dimensional scene model to obtain a target sub-model, and respectively extract features from the pixel data and temperature data in the target sub-model to generate an image semantic feature vector and a thermal distribution feature vector.

[0049] Among them, the target sub-model can represent a local spatial structure model obtained by region division, structure segmentation, or specified spatial range extraction based on the three-dimensional scene model. This target sub-model corresponds to a specific monitoring object area, such as the surface of a dam, a high slope area, a spillway boundary, etc. It is intercepted from the original three-dimensional scene model and retains its point cloud structure, pixel attributes, and temperature attributes. The image semantic feature vector can represent a multi-dimensional vector obtained through feature extraction operations based on the pixel data contained in the target sub-model, and is used to express the semantic attributes of the image content within this area. The thermal distribution feature vector can represent a multi-dimensional vector formed by characterizing the temperature distribution state of the local area for the temperature data contained in the target sub-model.

[0050] In this embodiment, by performing a regional segmentation operation on the three-dimensional scene model, the entire monitoring area can be divided into multiple spatial structure units, enabling subsequent analysis to focus on sub-regions with clear geographical attribution and structural significance, reducing the redundancy of overall data processing while enhancing the targeting of key area recognition.

[0051] Step S140: Use a pre-trained anomaly recognition model to perform anomaly recognition on the image semantic feature vector and the thermal distribution feature vector to obtain corresponding anomaly recognition results, and add the anomaly recognition results to the corresponding spatial positions of the three-dimensional scene model.

[0052] Among them, the anomaly recognition model can represent a discriminant model used to analyze and judge the image semantic feature vector and the thermal distribution feature vector to identify whether there are abnormal targets or states. The anomaly recognition result can represent the discriminant information output after the anomaly recognition model processes the input image semantic feature vector and thermal distribution feature vector, and is used to characterize whether there are structural anomalies, thermal anomalies, or external interferences, etc. in the target area. The anomaly recognition result can include, but is not limited to, anomaly types, confidence scores, etc. Adding the anomaly recognition result to the attribute fields of the corresponding spatial points or regions in the three-dimensional scene model enables the recognition information to be presented in a graphical or annotated manner in the structural model, thereby enhancing the localizability and visualization of the anomaly analysis results.

[0053] Figure 2A system structure diagram that can apply the above - mentioned hydropower project monitoring method based on drone inspection and 3D modeling is schematically shown. This method can be specifically implemented under the system architecture as shown in the figure. The system includes: a drone library, a path planning unit, a monitoring and early warning system, a 3D model construction unit, an intelligent inspection analysis unit, and a user terminal.

[0054] In this embodiment, the drone library is used to store and manage the drone devices to be deployed, and fly in response to the inspection path sent by the monitoring and early warning system. The drone is equipped with a color image sensor, an infrared thermal imaging device, and a lidar device. It flies within the hydropower project area according to the inspection path generated by the path planning unit, collects color images, infrared images, and lidar point cloud data of the target area, and transmits the inspection images and lidar point cloud data back to the monitoring and early warning system. The monitoring and early warning system receives the data transmitted by the drone and transmits it to the 3D model construction unit. The 3D model construction unit first constructs a 3D point cloud model corresponding to the hydropower project area based on the lidar point cloud data, and then maps the pixel data and temperature data to the 3D point cloud model based on the pose parameters of the color image and the infrared image to generate a 3D scene model. This 3D scene model is sent to the monitoring and early warning system as the spatial basis for subsequent analysis.

[0055] The monitoring and early warning system continues to perform an area segmentation operation on the 3D scene model to obtain the target sub - model, and respectively extracts features from the pixel data and temperature data in the target sub - model to generate an image semantic feature vector and a thermal distribution feature vector. The extracted feature vectors are input into a pre - trained anomaly recognition model to perform anomaly recognition on the image semantic feature vector and the thermal distribution feature vector, and finally obtain the corresponding anomaly recognition results. A monitoring report is generated according to the anomaly recognition results, and the anomaly recognition results are added to the corresponding spatial positions of the 3D scene model and warning information is generated. The monitoring and early warning system sends the color image and the infrared image to the intelligent inspection analysis unit to generate the corresponding inspection report. The intelligent inspection analysis unit can send the inspection report to the monitoring and early warning system for display. At the user end, the operator can receive the monitoring report, inspection report, and warning information sent by the monitoring and early warning system through the human - machine interface, and can input control commands through the interaction interface and send them back to the monitoring and early warning system to trigger subsequent operations such as drone re - inspection, path adjustment, or warning broadcast.

[0056] In some embodiments, the above-mentioned hydropower project monitoring method based on drone inspection and 3D modeling further includes the following technical steps: taking the spatial position corresponding to the anomaly recognition result in the 3D scene model as an inspection point, establishing the corresponding relationship between the inspection point and the anomaly recognition result, and generating an anomaly inspection path based on the inspection point; controlling the drone to perform image acquisition along the anomaly inspection path, and playing the warning voice information corresponding to the preset anomaly recognition result when reaching the inspection point corresponding to the preset anomaly recognition result.

[0057] Among them, the inspection point refers to the spatial target point to be rechecked with emphasis determined according to the spatial position corresponding to the anomaly recognition result in the 3D scene model. Each inspection point has a one-to-one or one-to-many corresponding relationship with at least one anomaly recognition result, and is used to indicate the key stop or attention position of the drone in the subsequent anomaly inspection path. Generating an anomaly inspection path based on the inspection point and controlling the drone to perform image acquisition operations according to this path enables the drone inspection task to focus on covering high-risk areas, improving the scheduling efficiency of drone inspections and the pertinence of task responses. When reaching a specific inspection point, playing the warning voice information matching the corresponding anomaly type can actively inform the on-site abnormal state during the drone inspection, enhancing the real-time and automated capabilities of on-site risk handling.

[0058] In the specific implementation process, based on the spatial distribution information of multiple inspection points, combined with the associated anomaly types, recognition confidence levels or spatial priorities, use a path planning algorithm to generate an anomaly inspection path covering all inspection points. This anomaly inspection path can be limited to starting from the current parking position or the nearest charging point of the drone, passing through each inspection point in turn, and satisfying scheduling constraints such as the shortest flight distance and obstacle avoidance safety. Subsequently, control the drone to perform image acquisition along the anomaly inspection path, record the flight status and image data in real time during the flight, and play the warning voice information corresponding to the preset anomaly recognition result when reaching each inspection point corresponding to the preset anomaly recognition result. The warning voice information can include a fixed voice template or a broadcast instruction corresponding to the content of the anomaly type, and is used to broadcast warnings on-site for personnel approaching illegally, illegal behaviors or sudden thermal abnormal situations.

[0059] In some embodiments, the above-mentioned generation of the anomaly inspection path based on the inspection point can be implemented through the following steps S310 to S350, which specifically include: Step S310, determine the starting point according to the starting position of the drone, and construct a node set according to the starting point and the inspection points.

[0060] Step S320, calculate the spatial distances between the nodes in the node set, and obtain a plurality of connecting edges according to the comparison result between the spatial distances and the preset distance threshold.

[0061] Specifically, the acquired node set includes a starting point and multiple inspection points, wherein the starting point can be the current location of the drone or the preset mission take-off point, and the inspection point is a spatial position determined in the three-dimensional scene model according to the abnormality recognition result. In the path generation stage, the distance between the three-dimensional spatial coordinates of any two nodes in the node set is calculated. The spatial distance can be solved based on the three-dimensional Euclidean distance formula to measure the direct connectivity between nodes. In order to avoid redundant connections and too long flight paths, the spatial distances between all calculated node pairs are compared one by one with the preset distance threshold. When the spatial distance between any node pair is less than or equal to the preset distance threshold, the node pair is determined to be connectable, and a connecting edge is established between the two, thereby generating a connecting edge set representing the pre-selected flight segment.

[0062] Step S330, performing point cloud extraction on the spatial area corresponding to each connecting edge in the three-dimensional point cloud model to obtain path point cloud data.

[0063] The path point cloud data may represent a point cloud subset extracted from the spatial region corresponding to the connecting edge based on the three-dimensional point cloud model. The process of determining the spatial region is as follows: taking the line between the starting node and the ending node of the connecting edge as the central axis, setting a fixed width, height and safety margin along the axis direction, and constructing a three-dimensional spatial region surrounding the connecting edge. The spatial region may be in the form of a cuboid, cylinder or other suitable structure to limit the range in which the point cloud data can be extracted.

[0064] Step S340, gridding the path point cloud data, and obtaining the elevation change information and obstacle distribution information corresponding to each connecting edge based on the height difference and point density characteristics in each grid.

[0065] Among them, the height difference value can represent the difference in vertical coordinate values ​​between the maximum point and the minimum point in each grid unit corresponding to the path point cloud data, which is used to reflect the terrain undulation or structural height change in the area. The point density feature can represent the number of point clouds contained in a unit grid, which is used to characterize the complexity of the spatial structure or the aggregation of obstacles. The elevation change information can represent the set of height difference values ​​extracted in each grid based on the path point cloud data, reflecting the terrain undulation and slope characteristics along the path segment. The obstacle distribution information can refer to the expression of the concentration or distribution trend of obstacles that may exist in the spatial area based on the statistical results of the point density features in the path point cloud data.

[0066] In specific implementation, a regular three-dimensional grid structure can be constructed within the spatial region corresponding to the connection edge. The path point cloud data is divided into multiple grid cells according to its spatial coordinates, and each grid corresponds to a fixed spatial range. For each grid cell, based on the point cloud data it contains, the height difference of the points within the grid is calculated, that is, the difference between the maximum height coordinate value and the minimum height coordinate value, which is used to reflect the vertical undulation of the region. At the same time, the number of points within the grid is counted to determine the point density feature of the grid. Further, the height differences in all grid cells are obtained to construct the elevation change information corresponding to the connection edge; at the same time, based on the extraction results of the point density features of each grid, the obstacle distribution information corresponding to the connection edge is generated, which is used to depict the structural complexity or the obstacle distribution trend along this path segment.

[0067] Step S350: Assign a passage cost value to each connection edge according to the elevation change information and the obstacle distribution information, and construct an abnormal inspection path based on the passage cost value.

[0068] Among them, the passage cost value can represent a numerical index given after comprehensively evaluating the passability of this path segment based on the elevation change information and the obstacle distribution information corresponding to the connection edge, and is used to measure the difficulty for the unmanned aerial vehicle to pass through this path segment when performing the abnormal inspection task. Denote the passage cost value as , and its calculation formula is as follows: Among them, represents the passage cost value of the corresponding connection edge; represents the normalized mean value of the elevation change information, reflecting the overall slope complexity of the path segment; represents the maximum point density value in the obstacle distribution information, which is used to measure the concentration degree of obstacles in the local space; represents the standard deviation of the height differences of all grids within the path segment, which is used to describe the discreteness and uneven undulation degree of the elevation change; , , respectively represent the elevation change weight, the obstacle risk weight, and the terrain disturbance weight.

[0069] In some embodiments, the following steps can be used to implement constructing the abnormal inspection path according to the passage cost value in step S350. Specifically, it can include: establishing a path connection graph according to the node set and multiple connection edges; in the path connection graph, starting from the starting point, traversing all inspection points, and determining the node connection order with the minimum total passage cost value according to the passage cost value corresponding to each connection edge; constructing an abnormal inspection path according to the node connection order.

[0070] In the specific implementation process, first construct a path connection graph according to the node set and multiple connection edges, denoted as , where represents the set of nodes, including the starting point and all inspection points; represents the set of connecting edges that meet the connection conditions; is the set of weights of the connecting edges, and the weights are composed of the passing cost values, represents the node and the node the passing cost value of the connecting edge between them.

[0071] In the path connection graph , with the starting point as the starting point of the path, traverse all inspection points except , and construct the optimal node access order , where , to minimize the following total cost function: where represents the total cost value function of the abnormal inspection path; represents all permutation and combination sets with as the starting point and including all inspection points; represents the th node and the th node in the path the passing cost value between them; represents the deflection angle of the path at the th turning point, defined as the angle amplitude between the current edge and the previous edge; represents the local elevation gradient change rate of the path on the connecting edge where the node is located; is the turning penalty weight coefficient, used to suppress the oscillation of the flight path; is the height difference penalty weight coefficient, used to avoid paths with sharp elevation fluctuations,

[0072] After solving to obtain the node connection order that makes reach the minimum value, the corresponding connecting edges can be spliced in sequence according to this node connection order to construct an abnormal inspection path.

[0073] Next, the content in the above steps S110 to S140 will be described in detail.

[0074] In some embodiments, the pose parameters based on the color image and the infrared image in step S120 above can be achieved through the following technical process, mapping the pixel data and the temperature data to the three-dimensional point cloud model, which may specifically include: obtaining the pose parameters of the color image and the infrared image, and determining the transformation matrix of the infrared image relative to the reference coordinate system of the color image based on the pose parameters; transforming the infrared image to the reference coordinate system of the color image according to the transformation matrix to generate an aligned infrared image; mapping the pixel data corresponding to the color image and the temperature data corresponding to the aligned infrared image to the three-dimensional point cloud model according to the correspondence between the color image, the infrared image and the laser point cloud data.

[0075] Among them, the transformation matrix refers to the mathematical transformation relationship used to transform the image information in the infrared image coordinate system to the reference coordinate system of the color image. This transformation matrix is calculated from the pose parameters of the infrared image and the color image, and may include a combination of a rotation matrix and a translation vector. Among them, the rotation matrix is used to describe the attitude difference between the two camera coordinate systems, and the translation vector is used to describe the spatial position offset between the two. Through this transformation matrix, the geometric mapping of any point in the infrared image in the reference coordinate system of the color image can be realized. The aligned infrared image refers to the set of image data generated after mapping the pixel points or temperature data in the original infrared image to the reference coordinate system of the color image through the transformation matrix.

[0076] Furthermore, define the transformation matrix as , and the acquisition process of the transformation matrix includes the following steps: First, obtain the pose parameters of the corresponding cameras of the infrared image and the color image respectively. These pose parameters include a rotation matrix and a translation vector. Denote the pose of the infrared camera as , and the pose of the color camera as , where: represents the rotation matrix of the infrared camera; represents the translation vector of the infrared camera; represents the rotation matrix of the color camera; represents the translation vector of the color camera.

[0077] According to the transformation relationship between the two camera poses, construct the transformation matrix from the infrared camera coordinate system to the color camera coordinate system: where: represents the direction transformation of the infrared coordinate system in the color image coordinate system; represents the translation transformation of the coordinate origin position difference; represents the zero vector row of the homogeneous transformation matrix; the constant term 1 represents the scale preservation in homogeneous coordinates.

[0078] In some embodiments, the process of generating the image semantic feature vector in step S130 includes: converting the pixel data in the target sub-model into an image tensor; performing multiple convolution operations and downsampling operations on the image tensor to extract hierarchical semantic features of the image tensor; performing feature compression on the hierarchical semantic features in the channel dimension to obtain a single semantic feature; performing global average pooling on the single semantic feature, and using the pooling result as the image semantic feature vector.

[0079] Among them, the image tensor can represent a multi-dimensional array structure formed by organizing the two-dimensional pixel data in the target sub-model according to its spatial structure and channel structure, including three dimensions: height, width, and channel, and is used as the data expression form for input to the deep learning network. The downsampling operation can represent the process of compressing the spatial size of the image tensor during the feature extraction process, and is used to retain the backbone features while reducing the computational complexity. The hierarchical semantic features can represent the feature representations extracted at different network levels after performing multiple convolutions and downsampling operations on the image tensor, and this feature can reflect the gradually abstract information from low-level textures to high-level semantics in the image. The single semantic feature can represent the unified feature expression obtained by performing a fusion compression operation on multiple hierarchical semantic features along the channel dimension, and this feature reduces the number of channels while maintaining semantic integrity. Global average pooling can represent the operation of calculating the average value at all positions in the spatial dimension of the feature map, converting the input feature map into a single numerical vector, retaining the overall semantic distribution feature, and is used to generate the final image semantic feature vector.

[0080] In this embodiment, by converting the pixel data in the target sub-model into an image tensor and combining multiple convolution operations and downsampling operations to extract hierarchical semantic features, the spatial structure and semantic information in the image are expressed in multiple scales; further, a single semantic feature is generated through feature compression in the channel dimension, and a global average pooling operation is performed to obtain the image semantic feature vector, realizing the effective aggregation and structured expression of key semantic features in the image region, which helps to improve the discrimination ability and adaptability of the subsequent anomaly recognition model.

[0081] In a specific implementation, for the pixel data in the target sub-model, first perform a structure conversion operation on it to reorganize it into an image tensor with a standard input structure, denoted as , the image tensor contains a three-dimensional array with a size of , represents the image height, represents the image width, represents the number of channels, which can include three color channels: red, green, and blue.

[0082] Next, for the image tensor Apply multiple convolutional operations and downsampling operations in sequence to extract hierarchical semantic features of the image. The convolutional operation uses a set of learnable convolutional kernels Act on each layer. The feature map of the th layer can be expressed as: Where, Represents the hierarchical semantic features extracted in the th layer, Is the input feature map of the previous layer, Represents the convolutional operation, Represents the th layer of convolutional kernel set, Represents the bias term, Represents the downsampling function, such as the max pooling function, etc., Represents the non-linear activation function, such as the ReLU function, etc.

[0083] After completing multi-layer convolution and downsampling, combine the output feature maps of all layers into a multi-channel feature structure, and compress along the channel dimension to obtain a single semantic feature with a unified dimension . This compression process can be expressed as: Where, Represents the weighted coefficient of the th layer channel feature, Represents the channel normalization function, L Represents the total number of channel layers.

[0084] Finally, perform global average pooling operation on the single semantic feature , aggregate the values at all positions on the spatial dimension to obtain the final image semantic feature vector , and its expression is: Where, Represents the value of the compressed feature map at the position , And Are the height and width of the feature map respectively, Represents the row coordinate index, Represents the column coordinate index.

[0085] In some embodiments, as shown in Figure 4 , the generation of the heat distribution feature vector can be realized through steps S410 to S440, specifically including: Step S410, perform normalization processing on the temperature data to generate a standardized temperature map. Among them, by performing normalization processing on the temperature data, the original temperature values are mapped to a unified numerical interval, which can eliminate the scale deviation caused by factors such as the measurement range, environmental temperature difference, or device parameters of different infrared images, making the temperature comparison in each image frame consistent.

[0086] Step S420, in the standardized temperature map, calculate the thermal response mask based on the ratio of the local temperature peak to the surrounding temperature, and generate a thermal response region.

[0087] Among them, the local temperature peak can represent the pixel point with the highest temperature value within the neighborhood window range centered on a specified pixel in the standardized temperature map. The surrounding temperature can represent the set of temperatures of all pixels except the central pixel within the neighborhood window constructed around the local temperature peak point. The thermal response mask refers to a weight image mask generated based on the ratio relationship between the local temperature peak and its corresponding surrounding temperature, used to identify the pixel regions with significant temperature difference characteristics in the standardized temperature map. The thermal response region can represent the continuous spatial region marked as a high-response state in the thermal response mask, usually having the local aggregation characteristic that the temperature is significantly higher than the surrounding environment.

[0088] In this step, by constructing the thermal response mask based on the ratio of the local temperature peak to the surrounding temperature in the standardized temperature map, the local thermal anomaly region can be effectively identified from the overall temperature distribution, avoiding the problems of missed detection or false detection of anomalies caused by improper setting of the global threshold. Further, the thermal response region extracted using the thermal response mask can accurately locate the spatial boundary of the local high-temperature region, which helps to improve the target focusing ability of subsequent thermal distribution feature extraction and the discrimination accuracy of the anomaly recognition model.

[0089] Step S430, perform regional temperature consistency analysis on the thermal response region to obtain the central average temperature, boundary gradient value, and internal temperature dispersion of the thermal response region.

[0090] Among them, the regional temperature consistency analysis can represent the processing process of statistical calculation and structural measurement of the distribution characteristics of each pixel temperature value within the thermal response region. The central average temperature can represent the average value of the temperature values of all pixels within a neighborhood window with a fixed scale near the regional geometric centroid in the thermal response region. The boundary gradient value can represent the change amplitude of the temperature value along the normal direction at the edge of the thermal response region. The internal temperature dispersion can represent the standard deviation or variance of the temperature values of all pixels within the thermal response region, used to measure the uniformity and fluctuation degree of the internal temperature distribution in this region.

[0091] Step S440, construct a thermal distribution feature vector based on the central average temperature, boundary gradient value, and internal temperature dispersion.

[0092] In some embodiments, in the above standardized temperature map, a thermosensitive response mask is calculated based on the ratio of the local temperature peak to the surrounding temperature, and a thermal response region is generated. The specific technical steps are as follows: Extract local temperature peak points in the standardized temperature map, and construct a neighborhood window with a fixed size based on each peak point; Calculate the average temperature within each neighborhood window, and calculate the ratio of the average temperature to the temperature value of the corresponding peak point; Use the pixel points whose ratio exceeds the preset thermosensitive response threshold as response points; Perform region aggregation processing on adjacent response points to generate a thermosensitive response mask, and generate a corresponding thermal response region based on the thermosensitive response mask.

[0093] Specifically, first identify local temperature peak points in the standardized temperature map, traverse all pixel points in the standardized temperature map based on the sliding window method, and construct a neighborhood window with a fixed size at each central pixel position, denoted as where is the coordinate of the current candidate peak pixel.

[0094] Within each neighborhood window calculate the average temperature value of the surrounding pixels except the central pixel and perform a ratio operation with the temperature value of the central pixel of this neighborhood to obtain the local response intensity coefficient: where represents the local response intensity coefficient, is a small constant introduced to prevent the denominator from being zero.

[0095] Mark all peak pixels that satisfy as response points, where represents the preset thermosensitive response threshold, which is used to filter low-response or ambient background points. Then perform region aggregation processing on all the above response points, and use the connected component analysis method to merge spatially adjacent response points into continuous regions to generate a thermosensitive response mask. This thermosensitive response mask is a binary image that marks all the high-response regions with local thermal anomaly characteristics in the current frame.

[0096] In some embodiments, perform regional temperature consistency analysis on the thermal response region to obtain the central average temperature, boundary gradient value, and internal temperature dispersion of the thermal response region. The specific technical process is as follows: Define the thermal response region as a continuous pixel set , and record the temperature value of any pixel point in the region as , where .

[0097] First, take the geometric center point For reference, a local window with a fixed size of is constructed adjacent to it. The average operation is performed on all pixel temperature values within this window to obtain the central average temperature: where represents the number of neighboring pixels, represents the central average temperature.

[0098] Next, a set of boundary pixels is extracted based on the edge contour of the thermal response region. For each boundary pixel , its adjacent pixel in the normal direction is selected, and the magnitude of the temperature gradient at this point is calculated.

[0099] By traversing all boundary pixels and taking the average of their gradient values, the overall boundary gradient value of the boundary is obtained: where represents the number of boundary pixels.

[0100] In addition, to quantify the stability of the temperature distribution inside the thermal response region, the set of temperature values of all pixel points within the region is statistically analyzed. Taking the overall average value as the benchmark, the temperature variance within the region is calculated: where represents the total number of pixels in the thermal response region, represents the temperature value, represents the temperature variance within the region, which can measure the uniformity and perturbation degree of the internal thermal field. The above three statistical indicators numerically model the spatial thermal structure of the thermal response region from the perspectives of thermal intensity, boundary transition, and internal consistency, providing precise high-dimensional temperature feature support for subsequent construction of thermal distribution feature vectors and thermal anomaly identification.

[0101] In some embodiments, an anomaly recognition model is used to perform anomaly recognition on the image semantic feature vector and the thermal distribution feature vector to obtain corresponding anomaly recognition results. The specific technical process is as follows: Input the image semantic feature vector into a first anomaly recognition model constructed by a convolutional neural network and a spatial attention mechanism to generate a first anomaly recognition result; Input the thermal distribution feature vector into a second anomaly recognition model constructed by a multi-layer perceptron network and a thermal distribution encoding layer to generate a second anomaly recognition result; Among them, the first anomaly recognition result includes any one or more of slope deformation, concrete cracks, landslides, structural water seepage, collapses, guardrail damage, and floating objects on the water surface, and the second anomaly recognition result includes any one or more of personnel staying, swimming behavior, and abnormally high temperature points.

[0102] Among them, the first anomaly recognition model can represent a visual class anomaly recognition network structure constructed based on the image semantic feature vector. The second anomaly recognition model can represent a thermal field behavior class anomaly recognition network structure constructed based on the thermal distribution feature vector. In other embodiments of the present disclosure, the first anomaly recognition result may also include types such as water level exceeding the limit, component damage, retaining wall cracking, water surface siltation, construction leftovers, dam body spalling, passage obstruction, and displacement of auxiliary structures, and the second anomaly recognition result may also include other types such as night stay behavior, overheating of electromechanical equipment, and temperature disturbance caused by animal stay at the inlet and outlet of a hydropower station.

[0103] In some embodiments, inputting the image semantic feature vector into the first anomaly recognition model constructed by a convolutional neural network and a spatial attention mechanism to generate a first anomaly recognition result specifically includes the following technical process: Using the convolutional neural network to perform multi-layer convolutional operations on the image semantic feature vector to extract the deep image feature map; Based on the spatial attention mechanism, adjust the response weights of each region in the deep image feature map to generate an enhanced image feature map; Perform anomaly category determination on the enhanced image feature map to generate a first anomaly recognition result.

[0104] In the specific implementation process, to generate the first anomaly recognition result, first use the convolutional neural network to perform multi-layer convolutional operations on the image semantic feature vector, and continuously extract the spatial features and semantic features of the image to construct a deep image feature map. This deep image feature map has the ability to express features at multiple scales and levels, and can effectively depict the geometric shape and texture changes of structural targets in complex scenarios.

[0105] After obtaining the deep image feature map, adaptively adjust the response intensity of each spatial position based on the spatial attention mechanism. Define the response score of each pixel position in the deep image feature map as , and obtain the spatial attention weight through normalization calculation: Among them, represents the spatial attention weight at position in the deep feature map of the image, represents the response score generated by the attention mechanism, represents the traversal index for all spatial positions, represents the response score at position ( ) in the deep feature map of the image.

[0106] Apply the above spatial attention weight to the feature vectors at the corresponding positions in the deep feature map of the image, and perform a point-by-point weighted fusion operation on them to generate an enhanced image feature map. In this process, the regions in the image that are highly correlated with abnormal features will obtain enhanced expression, while the feature responses in the background or invalid regions will be relatively weakened.

[0107] Subsequently, perform a global average pooling operation on the enhanced image feature map in the spatial dimension to obtain a feature compression vector , and input it into the anomaly determination network, and calculate the first anomaly recognition result through the following expression : Among them, represents the weight parameter of the fully connected classification layer, represents the classification bias, and Softmax represents the normalization function, which is used to map the output to a multi-class probability distribution.

[0108] In some embodiments, input the heat distribution feature vector into the second anomaly recognition model constructed by a multi-layer perceptron network and a heat distribution feature encoding layer to generate a second anomaly recognition result, which specifically includes the following technical processes: input the central average temperature, boundary gradient value, and temperature dispersion in the heat distribution feature vector into the corresponding heat distribution feature encoding layers respectively; perform normalization and scale alignment processing on the input data in the heat distribution feature encoding layers to obtain a three-channel feature vector; splice the three-channel feature vectors in a preset order to generate a combined feature vector; input the combined feature vector into the multi-layer perceptron network for non-linear mapping processing, and generate a second anomaly recognition result based on the result of the non-linear mapping processing.

[0109] Among them, the thermal distribution feature encoding layer can represent a feature processing structure used to standardize and scale-align different physical dimensions in the thermal distribution feature vector. The three-channel feature vector can represent the normalized processing results of three groups output by the thermal distribution feature encoding layer, corresponding to the central average temperature, the boundary gradient value, and the temperature dispersion respectively. Each channel represents the expression information of an independent thermal attribute dimension. The multi-layer perceptron network can represent a feed-forward neural network module containing multiple fully-connected hierarchical structures, used to perform high-order combination and non-linear feature mapping operations on the input feature vector. By introducing activation functions, hidden units, and deep connection relationships, this network enables the thermal distribution features to have stronger classification boundary expression capabilities in the high-dimensional space. The non-linear mapping process can represent the process of applying non-linear transformation to the combined feature vector based on the activation function in the multi-layer perceptron network. By normalizing and scale-aligning the central average temperature, the boundary gradient value, and the temperature dispersion, different thermal attributes are given a unified expression scale to avoid feature imbalance; splicing the processed three-channel features into a combined feature vector and inputting it into the multi-layer perceptron network for non-linear mapping helps to uncover the correlations between complex thermal distribution patterns and improve the recognition accuracy and stability of thermal anomalies such as personnel retention and equipment overheating.

[0110] In other embodiments of the present disclosure, after obtaining the first anomaly recognition result and the second anomaly recognition result, corresponding warning information can also be generated based on the first anomaly recognition result and the second anomaly recognition result.

[0111] Specifically, the spatial location, anomaly type, anomaly level, and recognition confidence corresponding to each recognition result can be extracted, and combined with the time tag and monitoring location of the current task to construct a structured warning candidate entry. During the warning generation process, matching judgments can be made according to the anomaly type and the engineering structure type corresponding to its location. For example, when the first anomaly recognition result is slope fissure or structural water seepage, the type of hydraulic structure to which its occurrence location belongs, such as dam, diversion canal, powerhouse, etc., and the current inspection frequency can be combined to determine whether the trigger conditions for structural safety warning are met; when the second anomaly recognition result is personnel retention, abnormal high-temperature point, or activity behavior within the restricted area, the use classification of the monitoring area and the set threshold can be combined to judge whether the warning conditions for behavior safety or thermal risk are met. After the judgment result meets the corresponding warning conditions, warning information including warning level, trigger conditions, engineering matters, disposal suggestions, associated responsible persons, and spatial identifiers can be generated and loaded onto the interaction interface for display and confirmation.

[0112] Figure 5The process schematic diagram of the hydropower project monitoring method according to some embodiments of the present disclosure is schematically shown. As shown in the figure, in the hydropower project monitoring method, intelligent route planning is used to set a drone flight path that is complete in coverage and reasonable in obstacle avoidance according to the terrain range and structural distribution of the hydropower project area. The flight path information is sent to the drone storage area through the remote control system to achieve dynamic synchronization with the inspection task. The drone hangar responds to the received instruction, starts the standby drone and completes the initialization. Subsequently, the drone autonomously operates and flies according to the flight path. During the flight, color images, infrared images and laser point cloud data are collected in sequence to form a multi-modal perception information source. The collected data is sent to the data processing software in real time through the data transmission channel.

[0113] Entering the data processing stage, first, data preprocessing is completed, including basic operations such as image alignment, point cloud denoising and infrared thermal map standardization. Multi-period data registration is used to perform spatial and temporal consistency alignment of the current inspection data and historical data to ensure a coherent observation of the change trend of the target area. Subsequently, slope disease hidden danger identification and monitoring operations are performed. A three-dimensional point cloud model of the hydropower project area is constructed based on the laser point cloud data, and combined with the pose parameters of the color image and the infrared image, the corresponding pixel data and temperature data in the image are mapped to the three-dimensional point cloud model to generate a three-dimensional scene model with spatial alignment attributes. The model is further divided into multiple target sub-models according to the structural boundaries, and the image semantic feature vectors and thermal distribution feature vectors are respectively extracted from the pixel data and temperature data within each sub-model. The above features are input into a pre-trained anomaly recognition model for classification and recognition, and the anomaly recognition results are output and marked at the corresponding spatial positions of the three-dimensional scene model to achieve the spatial association expression of image information and structural anomalies.

[0114] The data processing software summarizes the above recognition results and inspection process data to generate a report. After the report is generated, it is synchronized to the user side. The user side displays information including three-dimensional models, anomaly distributions, heat map responses and historical comparisons, and supports user interaction operations such as anomaly confirmation, re-inspection scheduling and path re-planning suggestion feedback. The whole process is carried out based on the closed-loop of automatic drone collection, multi-source data fusion, model recognition and visualization interaction. Compared with the traditional method that relies on manual inspection and two-dimensional image judgment, it significantly improves the monitoring accuracy, timeliness and spatial expression ability of areas such as hydraulic structures and slope structures, and is applicable to the inspection scenarios of hydropower projects with complex operating environments and diverse structural types.

[0115] Figure 6Schematically shows a schematic diagram of the interaction interface of a hydropower project monitoring system according to some embodiments of the present disclosure. Specifically, the interaction interface of the hydropower project monitoring system provides a warning setting and editing function for parameter configuration and task management of risk events identified by the system. The interface includes multiple fields such as warning level, trigger condition, associated matters, warning content, responsible person, status flag, status, and instruction issuance. The warning level is used to set the response level for the current anomaly, such as general warning or emergency warning. The trigger condition field records the specific data basis that triggers the warning, such as the water level exceeding the set threshold. The associated matters indicate the engineering structure or operation task corresponding to the warning, such as dam safety management. The warning content is used to describe the abnormal phenomenon identified by the system and the recommended handling method, such as immediately evacuating personnel. The responsible person field designates the person responsible for handling the warning. The status flag is used to mark the processing stage of the warning, such as processed, unprocessed, etc., and the status field indicates whether the task has been started. The instruction input area is used to fill in dispatching instructions or handling commands, which can be automatically generated by the system or manually input.

[0116] Further, in other embodiments of the present disclosure, after obtaining the image semantic feature vector and the thermal distribution feature vector, the image semantic feature vector and the thermal distribution feature vector can also be used as joint inputs and input into the same deep learning network for anomaly recognition. To improve the discriminative ability of multi-modal feature fusion, a fusion strategy combining structural prior guidance and modal cross-attention mechanism is adopted to enhance the collaborative expression ability between different modalities.

[0117] Specifically, first, for the current target sub-model to be recognized, according to its structural type, such as dam body, slope, water area, protective structure, and road, etc., a structural prior label vector is constructed, and it is mapped to a structural prior embedding vector through an embedding encoder as auxiliary information for model structure perception.

[0118] Subsequently, linear mapping and normalization processing are respectively performed on the image semantic feature vector and the thermal distribution feature vector to make their feature dimensions consistent and numerical ranges unified. After obtaining the aligned multi-modal features, a modal-guided cross-attention mechanism is constructed: a first attention channel is established with the image semantic feature as the main modality and the thermal distribution feature as the auxiliary modality, and a second attention channel is constructed with the image semantic feature as the auxiliary modality and the thermal distribution feature as the main modality. At the same time, the structural prior embedding vector is introduced into both attention channels as background semantic guidance.

[0119] Through the weighted summation and attention weight learning mechanism, multi-modal fusion features are extracted , which can be formally expressed as: Among them, represents the image semantic feature vector, represents the thermal distribution feature vector, represents the structural prior embedding vector, is a learnable parameter matrix.

[0120] After obtaining the multi-modal fusion features, input them into the parallel anomaly category branch prediction network. This parallel anomaly category branch prediction network contains multiple classification branches, and each branch corresponds to an anomaly category sub-domain, such as a structural anomaly branch, a thermal anomaly branch, and a behavior anomaly branch, etc. Exemplarily, the structural anomaly branch can be used to identify anomalies such as concrete cracks, slope slips, structural water seepage, and collapses, mainly dominated by image semantic features and fusing a small amount of temperature anomaly information; the thermal anomaly branch can be used to identify high-temperature concentration areas, illegal personnel staying, and approaching of abnormal heat sources, etc., dominated by the thermal distribution features and combining the image structure boundary information as an auxiliary; the behavior anomaly branch can be used to identify behavior-type targets such as wild swimming, fishing, and crossing the boundary, etc., and it is necessary to combine the image contour, motion trajectory, and local heat aggregation pattern. An independent modal attention adjustment structure is set inside each branch to assign different dominant weights to the image and thermal features according to the modal sensitivity of a specific anomaly type. For example, in the structural anomaly branch, by increasing the weight of the image feature channels, the influence of the thermal channel is weakened; while in the thermal anomaly branch, it is the opposite.

[0121] Finally, each branch outputs the anomaly confidence under the corresponding category, generates the anomaly recognition result through the maximum probability or a set threshold, and writes the recognition result back to the corresponding spatial position in the 3D scene model.

[0122] It should be noted that although the steps of the method in the present disclosure are described in a specific order in the drawings, this does not require or imply that these steps must be executed in this specific order, or that all the shown steps must be executed to achieve the desired result. Additionally or alternatively, some steps can be omitted, multiple steps can be combined into one step for execution, and / or one step can be decomposed into multiple steps for execution, etc.

[0123] In addition, in this exemplary embodiment, a hydropower project monitoring system based on drone inspection and 3D modeling is also provided. Referring to Figure 7 as shown, the hydropower project monitoring system 700 based on drone inspection and 3D modeling includes: an image acquisition module 710, a 3D reconstruction module 720, a vector extraction module 730, and an anomaly recognition module 740. Among them: The image acquisition module 710 can be used to fly a drone along a preset flight path in the hydropower project area and collect color images, infrared images, and laser point cloud data; The 3D reconstruction module 720 can be used to construct a 3D point cloud model corresponding to the hydropower project area based on the lidar point cloud data, and map the pixel data and temperature data to the 3D point cloud model based on the pose parameters of the color image and the infrared image to generate a 3D scene model; The vector extraction module 730 can be used to perform regional segmentation operations on the 3D scene model to obtain the target sub-model, and respectively extract features from the pixel data and temperature data in the target sub-model to generate an image semantic feature vector and a thermal distribution feature vector; The anomaly recognition module 740 can be used to perform anomaly recognition on the image semantic feature vector and the thermal distribution feature vector through a pre-trained anomaly recognition model to obtain the corresponding anomaly recognition result, and add the anomaly recognition result to the spatial position corresponding to the 3D scene model.

[0124] The specific details of each module of the above hydropower project monitoring system based on UAV inspection and 3D modeling have been described in detail in the corresponding hydropower project monitoring method based on UAV inspection and 3D modeling, so they will not be elaborated here.

[0125] It should be noted that although several modules or units of the hydropower project monitoring system based on UAV inspection and 3D modeling are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of the two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0126] In addition, in the exemplary embodiments of the present disclosure, an electronic device capable of implementing the above hydropower project monitoring method based on UAV inspection and 3D modeling is also provided.

[0127] Those skilled in the art can understand that various aspects of the present disclosure can be implemented as a system, method, or program product. Therefore, various aspects of the present disclosure can be specifically implemented in the following forms, namely: a complete hardware embodiment, a complete software embodiment (including firmware, microcode, etc.), or an embodiment combining hardware and software aspects, which can be collectively referred to as "circuitry", "module", or "system" here.

[0128] Next, refer to Figure 8 to describe the electronic device 800 according to the embodiments of the present disclosure. Figure 8 The illustrated electronic device 800 is merely an example and should not impose any limitation on the functions and usage scope of the embodiments of the present disclosure.

[0129] As Figure 8As shown, the electronic device 800 is presented in the form of a general-purpose computing device. The components of the electronic device 800 may include, but are not limited to: at least one of the above-mentioned processing units 810, at least one of the above-mentioned storage units 820, a bus 830 connecting different system components (including the storage unit 820 and the processing unit 810), and a display unit 840.

[0130] Among them, the storage unit stores program code, and the program code can be executed by the processing unit 810, so that the processing unit 810 executes the steps according to various exemplary embodiments of the present disclosure described in the "Exemplary Method" section of the present specification.

[0131] The storage unit 820 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 821 and / or a cache storage unit 822, and may further include a read-only storage unit (ROM) 823.

[0132] The storage unit 820 may also include a program / utility 824 having a set (at least one) of program modules 825. Such program modules 825 include, but are not limited to: an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include the implementation of a network environment.

[0133] The bus 830 may represent one or more of several types of bus structures, including a storage unit bus or a storage unit controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any bus structure in a variety of bus structures.

[0134] The electronic device 800 may also communicate with one or more external devices 870 (such as a keyboard, a pointing device, a Bluetooth device, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 800, and / or communicate with any device that enables the electronic device 800 to communicate with one or more other computing devices (such as a router, a modem, etc.). Such communication may be carried out through an input / output (I / O) interface 850. And, the electronic device 800 may also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 860. As shown in the figure, the network adapter 860 communicates with other modules of the electronic device 800 through the bus 830. It should be understood that although not shown in the figure, other hardware and / or software modules may be used in combination with the electronic device 800, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0135] Through the description of the above embodiments, those skilled in the art can easily understand that the exemplary embodiments described herein can be implemented by software or by a combination of software and necessary hardware. Therefore, the technical solutions according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, a USB flash drive, a portable hard drive, etc.) or on a network, including several instructions to enable a computing device (such as a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.

[0136] In an exemplary embodiment of the present disclosure, there is also provided a computer-readable storage medium, on which a program product capable of implementing the above method of the present specification is stored. In some possible embodiments, various aspects of the present disclosure can also be implemented in the form of a program product, which includes program code. When the program product runs on a terminal device, the program code is used to cause the terminal device to execute the steps according to various exemplary embodiments of the present disclosure described in the above "Exemplary Method" section of the present specification.

[0137] Reference Figure 9 As shown, a program product 900 for implementing the above-described method for monitoring a hydropower project based on UAV inspection and 3D modeling according to an embodiment of the present disclosure is described. It can be a portable compact disc read-only memory (CD-ROM) and includes program code, and can run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited thereto. In this document, a readable storage medium can be any tangible medium that contains or stores a program that can be used by or in combination with an instruction execution system, apparatus, or device.

[0138] The program product can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0139] A computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries readable program code. Such a propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. The readable signal medium may also be any readable medium other than a readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device.

[0140] The program code contained on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0141] The program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, etc., and also including conventional procedural programming languages such as the "C" language or similar programming languages. The program code may be executed entirely on the user's computing device, partially on the user's device, executed as a stand-alone software package, partially on the user's computing device and partially on a remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., by connecting through the Internet using an Internet service provider).

[0142] In addition, the above-mentioned drawings are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present disclosure, and are not for limiting purposes. It is easy to understand that the processes shown in the above-mentioned drawings do not indicate or limit the chronological order of these processes. Additionally, it is also easy to understand that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0143] Through the description of the above embodiments, those skilled in the art can easily understand that the exemplary embodiments described herein can be implemented by software or in the form of software combined with necessary hardware. Therefore, the technical solution according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, USB flash drive, mobile hard disk, etc.) or on a network, including several instructions to cause a computing device (which can be a personal computer, server, touch terminal, or network device, etc.) to execute the method according to the embodiments of the present disclosure.

[0144] Other embodiments of the present disclosure will be readily apparent to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include known or customary technical means in the art not disclosed herein. The specification and examples are only to be considered as exemplary, and the true scope and spirit of the present disclosure are pointed out by the claims.

[0145] It should be understood that the present disclosure is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present disclosure is only limited by the appended claims.

Claims

1. A hydropower project monitoring method based on drone inspection and three-dimensional modeling, characterized in that: include: Use drones to fly along a preset flight path in the hydropower project area and collect color images, infrared images and laser point cloud data; Constructing a three-dimensional point cloud model corresponding to the hydropower project area according to the laser point cloud data, and mapping pixel data and temperature data to the three-dimensional point cloud model based on the posture parameters of the color image and the infrared image to generate a three-dimensional scene model; Performing a region segmentation operation on the three-dimensional scene model to obtain a target sub-model, and performing feature extraction on the pixel data and the temperature data in the target sub-model respectively to generate an image semantic feature vector and a thermal distribution feature vector; Anomalies are identified on the image semantic feature vector and the thermal distribution feature vector through a pre-trained anomaly recognition model to obtain corresponding anomaly recognition results, and the anomaly recognition results are added to the corresponding spatial position of the three-dimensional scene model.

2. The hydropower project monitoring method based on drone inspection and three-dimensional modeling according to claim 1 is characterized in that: Also includes: The spatial position corresponding to the abnormal recognition result in the three-dimensional scene model is used as a patrol point, a corresponding relationship between the patrol point and the abnormal recognition result is established, and an abnormal patrol path is generated based on the patrol point; The drone is controlled to collect images along the abnormal inspection path, and when arriving at an inspection point corresponding to a preset abnormal recognition result, a warning voice message corresponding to the preset abnormal recognition result is played.

3. The hydropower project monitoring method based on drone inspection and three-dimensional modeling according to claim 2 is characterized in that: The generating of an abnormal inspection path based on the inspection point includes: Determine a starting point according to the starting position of the drone, and construct a node set according to the starting point and the inspection point; Calculating the spatial distance between each node in the node set, and obtaining a plurality of connecting edges according to a comparison result between the spatial distance and a preset distance threshold; Performing point cloud extraction on the spatial region corresponding to each of the connecting edges in the three-dimensional point cloud model to obtain path point cloud data; Gridding the path point cloud data, and obtaining elevation change information and obstacle distribution information corresponding to each connecting edge based on the height difference and point density characteristics in each grid; A travel cost value is allocated to each of the connecting edges according to the elevation change information and the obstacle distribution information, and the abnormal inspection path is constructed according to the travel cost value.

4. The hydropower project monitoring method based on drone inspection and three-dimensional modeling according to claim 3 is characterized in that: The constructing the abnormal inspection path according to the travel cost value includes: Establishing a path connection graph according to the node set and the plurality of connection edges; In the path connection graph, starting from the starting point, traversing all inspection points, and determining the node connection order with the smallest total cost value according to the travel cost value corresponding to each of the connection edges; The abnormal inspection path is constructed according to the node connection sequence.

5. The hydropower project monitoring method based on drone inspection and three-dimensional modeling according to claim 1 is characterized in that: The step of mapping pixel data and temperature data to the three-dimensional point cloud model based on the posture parameters of the color image and the infrared image includes: Acquire posture parameters of the color image and the infrared image, and determine a transformation matrix of the infrared image relative to a reference coordinate system of the color image based on the posture parameters; transforming the infrared image to the color image reference coordinate system according to the transformation matrix to generate an aligned infrared image; According to the correspondence between the color image, the infrared image and the laser point cloud data, the pixel data corresponding to the color image and the temperature data corresponding to the aligned infrared image are mapped to the three-dimensional point cloud model.

6. The hydropower project monitoring method based on drone inspection and three-dimensional modeling according to claim 1 is characterized in that: The generation process of the image semantic feature vector includes: Convert pixel data in the target sub-model into an image tensor; Performing multiple convolution operations and downsampling operations on the image tensor to extract hierarchical semantic features of the image tensor; Performing channel-dimensional feature compression on the hierarchical semantic features to obtain a single semantic feature; Perform global average pooling on the single semantic feature, and use the pooling result as the image semantic feature vector.

7. The hydropower project monitoring method based on drone inspection and three-dimensional modeling according to claim 1 is characterized in that: The generation process of the heat distribution feature vector includes: Normalizing the temperature data to generate a standardized temperature map; In the standardized temperature map, a thermal response mask is calculated based on the ratio of the local temperature peak to the surrounding temperature, and a thermal response area is generated; Performing regional temperature consistency analysis on the thermal response region to obtain the central average temperature, boundary gradient value and internal temperature dispersion of the thermal response region; The heat distribution feature vector is constructed based on the central average temperature, the boundary gradient value and the internal temperature dispersion.

8. The hydropower project monitoring method based on drone inspection and three-dimensional modeling according to claim 7 is characterized in that: In the standardized temperature map, a thermosensitive response mask is calculated based on the ratio of the local temperature peak to the surrounding temperature, and a thermal response area is generated, including: Extracting local temperature peak points in the standardized temperature map, and constructing a neighborhood window of a fixed size based on each of the peak points; Calculate the average temperature in each of the neighborhood windows, and calculate the ratio of the average temperature to the temperature value of the corresponding peak point; The pixel point whose ratio exceeds the preset thermal response threshold is regarded as the response point; Performing regional aggregation processing on adjacent response points to generate a thermal response mask, and generating a corresponding thermal response region based on the thermal response mask.

9. The hydropower project monitoring method based on drone inspection and three-dimensional modeling according to claim 1 is characterized in that: The pre-trained abnormality recognition model is used to perform abnormality recognition on the image semantic feature vector and the thermal distribution feature vector to obtain a corresponding abnormality recognition result, including: Inputting the image semantic feature vector into a first anomaly recognition model constructed by a convolutional neural network and a spatial attention mechanism to generate a first anomaly recognition result; Inputting the heat distribution feature vector into a second anomaly recognition model constructed by a multi-layer perception network and a heat distribution feature encoding layer to generate a second anomaly recognition result; Among them, the first abnormal identification result includes any one or more of slope deformation, concrete cracks, landslides, structural seepage, collapse, guardrail damage and floating objects on the water surface, and the second abnormal identification result includes any one or more of stranded personnel, swimming behavior and abnormally high temperature points.

10. The hydropower project monitoring method based on drone inspection and three-dimensional modeling according to claim 9 is characterized in that: The step of inputting the image semantic feature vector into a first anomaly recognition model constructed by a convolutional neural network and a spatial attention mechanism to generate a first anomaly recognition result includes: Using a convolutional neural network to perform multi-layer convolution operations on the image semantic feature vector to extract a deep feature map of the image; Adjusting the response weights of each region in the deep feature map of the image based on a spatial attention mechanism to generate an enhanced image feature map; An abnormality category determination is performed on the enhanced image feature map to generate the first abnormality recognition result.

11. The hydropower project monitoring method based on drone inspection and three-dimensional modeling according to claim 9 is characterized in that: The step of inputting the heat distribution feature vector into a second anomaly recognition model constructed by a multi-layer perception network and a heat distribution feature encoding layer to generate a second anomaly recognition result includes: Inputting the central average temperature, boundary gradient value and temperature dispersion in the thermal distribution feature vector into the corresponding thermal distribution feature encoding layer respectively; Normalizing and aligning the input data in the thermal distribution feature encoding layer to obtain a three-channel feature vector; The three-channel feature vectors are concatenated in a preset order to generate a combined feature vector; The combined feature vector is input into the multi-layer perception network for nonlinear mapping processing, and the second abnormality recognition result is generated based on the result of the nonlinear mapping processing.

12. A hydropower project monitoring system based on drone inspection and three-dimensional modeling, characterized in that: include: An image acquisition module is used to use the UAV to fly along a preset flight path in the hydropower project area and collect color images, infrared images and laser point cloud data; A three-dimensional reconstruction module, used to construct a three-dimensional point cloud model corresponding to the hydropower project area according to the laser point cloud data, and map pixel data and temperature data to the three-dimensional point cloud model based on the posture parameters of the color image and the infrared image to generate a three-dimensional scene model; A vector extraction module, used to perform a region segmentation operation on the three-dimensional scene model, obtain a target sub-model, and perform feature extraction on the pixel data and the temperature data in the target sub-model respectively to generate an image semantic feature vector and a thermal distribution feature vector; The anomaly recognition module is used to perform anomaly recognition on the image semantic feature vector and the thermal distribution feature vector through a pre-trained anomaly recognition model to obtain a corresponding anomaly recognition result, and add the anomaly recognition result to the spatial position corresponding to the three-dimensional scene model.

Images