Water conservancy project image recognition method, device and equipment for unmanned aerial vehicle inspection and medium

Abstract

The invention relates to the technical field of water conservancy projects. According to the water conservancy project image recognition method, device and equipment for unmanned aerial vehicle inspection and the medium, the method comprises the steps that a time sequence image stream on the surface of a water conservancy project is collected through a preset multispectral sensor carried by an unmanned aerial vehicle, and positioning data and attitude data of each frame of image in the time sequence image stream are synchronously obtained; based on a preset hydraulic engineering lightweight BIM model, feature extraction and spatial transformation processing are carried out on the time sequence image stream, and a projection relation is generated; in the registered fusion scene, defect identification processing is carried out on the time sequence image stream in a man-machine cooperation mode by utilizing a projection relation, and component associated defect information is generated; and on the basis of the component ID of the lightweight BIM model, performing space-time association processing on the component associated defect information to generate a structured database so as to achieve the technical effects of improving the image and BIM space registration precision, optimizing the man-machine collaborative recognition efficiency and enhancing the component-level defect space-time analysis capability.

Description

Technical Field

[0001] This invention relates to the field of water conservancy engineering technology, and in particular to a method, device, equipment and medium for image recognition of water conservancy projects inspected by unmanned aerial vehicles (UAVs). Background Technology

[0002] With the rapid development of drone and artificial intelligence technologies, the field of safety inspection of water conservancy projects is undergoing an intelligent transformation. Drones equipped with multispectral sensors can efficiently acquire time-series image data of facilities such as dams and dikes, providing a foundation for defect identification.

[0003] Traditional water conservancy engineering defect identification technology uses manual comparison of UAV images with 2D drawings for defect location processing. However, this method is difficult to achieve high-precision spatial registration between images and 3D BIM (Building Information Modeling) models, resulting in significant deviations in defect coordinate mapping. When facing the problem of identifying complex surface defects, traditional technologies rely solely on manual visual inspection or pure AI segmentation algorithms. However, manual methods are inefficient, and AI methods have a high false alarm rate and lack human-machine collaborative optimization mechanisms. In addition, existing methods ignore the spatiotemporal correlation analysis of component-level defects, resulting in historical defect data being discrete and fragmented, which cannot support the dynamic assessment and early warning of the project's health status. Summary of the Invention

[0004] Therefore, it is necessary to provide methods, devices, equipment and media for image recognition of water conservancy projects by drone inspection in response to the above-mentioned technical problems, so as to improve the accuracy of image registration with BIM space, optimize the efficiency of human-machine collaborative recognition, and enhance the spatiotemporal analysis capability of component-level defects.

[0005] Firstly, this application provides a method for image recognition of water conservancy projects inspected by unmanned aerial vehicles (UAVs), the method comprising:

[0006] The drone uses a pre-set multispectral sensor to collect a time-series image stream of the surface of the water conservancy project, and simultaneously acquires the positioning and attitude data of each frame in the time-series image stream.

[0007] Based on a pre-set lightweight BIM model of water conservancy project, feature extraction and spatial transformation processing are performed on time-series image streams to generate projection relationships.

[0008] In the fusion scene after registration, the projection relationship is used to perform defect identification processing on the time-series image stream through human-machine collaboration, and component-related defect information is generated;

[0009] Based on the component ID of the lightweight BIM model, spatiotemporal correlation processing is performed on the component-related defect information to generate a structured database.

[0010] In one embodiment, in the registered fusion scene, using projection relationships, defect identification processing is performed on the temporal image stream through human-machine collaboration to generate component-related defect information, including:

[0011] In response to a 3D bounding box selection operation on the surface of a lightweight BIM model, the selected area is processed by image defect feature extraction to generate manually annotated defect areas.

[0012] A pre-trained defect segmentation network is used to perform pixel-level semantic parsing of the temporal shadow stream to generate a defect semantic mask map.

[0013] By utilizing projection relationships, spatial coordinate mapping is performed on manually annotated defect areas and defect semantic masks to generate component-related defect information.

[0014] In one embodiment, spatial coordinate mapping is performed on manually annotated defect areas and defect semantic masks using projection relationships to generate component-associated defect information, including:

[0015] Spatial consistency verification is performed on manually annotated defect areas and defect semantic masks to generate registered defect data.

[0016] Based on the projection relationship, the registered defect data is transformed from pixels to BIM coordinates to generate a three-dimensional defect location set.

[0017] By optimizing the spatial topological relationships of components in a lightweight BIM model, component matching is performed on the set of three-dimensional defect locations to generate component-related defect information.

[0018] In one embodiment, in response to a 3D bounding box selection operation on the surface of a lightweight BIM model, image defect feature extraction processing is performed on the selected area to generate manually annotated defect areas, including:

[0019] Perform spatial coordinate analysis on the 3D bounding box selection operation to generate the bounding box spatial range;

[0020] Based on the projection relationship, the selected spatial range is subjected to two-dimensional image inverse processing to generate the target image region.

[0021] Texture-spectral joint feature analysis is performed on the target image region to generate an enhanced defect feature map;

[0022] In response to manual confirmation, the enhanced defect feature map is refined to generate manually annotated defect areas.

[0023] In one embodiment, based on a preset lightweight BIM model for water conservancy projects, feature extraction and spatial transformation processing are performed on the time-series image stream to generate projection relationships, including:

[0024] Radiometric distortion correction and illumination equalization are performed on the temporal image stream to generate preprocessed images;

[0025] Local invariant feature extraction is performed on the preprocessed image to generate a set of local invariant feature points;

[0026] Based on the surface mesh vertex set of the lightweight BIM model, iterative matching processing is performed on the local invariant feature point set to generate initial exterior orientation parameters.

[0027] The initial exterior orientation parameters are processed to remove mismatches, and optimized exterior orientation parameters are generated.

[0028] Perspective projection transformation is performed based on optimized exterior orientation parameters to generate projection relationships.

[0029] In one embodiment, based on the component ID of the lightweight BIM model, spatiotemporal correlation processing is performed on the component-related defect information to generate a structured database, including:

[0030] Based on the component ID, spatiotemporal clustering is performed on the component-related defect information to generate a component defect sequence;

[0031] Evolution trend analysis and spatial distribution pattern recognition of component defect sequences are performed to generate multidimensional defect evolution characteristics;

[0032] Based on the component topology relationship of the lightweight BIM model, the multidimensional defect evolution features are semantically associated to generate a defect knowledge graph.

[0033] Spatiotemporal indexing and relation mapping are performed on the defect knowledge graph to generate a structured database.

[0034] In one embodiment, the component-associated defect information is obtained using the following formula, including:

[0035]

[0036] in, Indicates component-related defect information. Represents a unique identifier for a component. This indicates that an aggregation operation is performed on all defect instances. Represents the coordinate standardization function. The generalized inverse operator representing the projection relation, Represents the two-dimensional homogeneous coordinates of the defect point. This represents the defect feature extraction function. This represents the characteristics of the defect area.

[0037] Secondly, this application also provides an image recognition device for water conservancy projects inspected by drones, the device comprising:

[0038] The UAV image acquisition module is used to acquire time-series image streams of the surface of water conservancy projects through a preset multispectral sensor carried by the UAV, and simultaneously acquire the positioning data and attitude data of each frame in the time-series image stream.

[0039] The projection relationship generation module is used to extract features and perform spatial transformation processing on time-series image streams based on a preset lightweight BIM model of water conservancy projects, and generate projection relationships.

[0040] The human-machine collaborative defect recognition module is used to perform defect recognition processing on the time-series image stream in the registered fusion scene by utilizing projection relationships and through human-machine collaboration, and to generate component-related defect information.

[0041] The structured database generation module is used to perform spatiotemporal correlation processing on component-related defect information based on the component ID of the lightweight BIM model, and generate a structured database.

[0042] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of any of the methods in the first aspect of this application.

[0043] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of any of the methods in the first aspect of this application.

[0044] The method, apparatus, equipment, and medium for image recognition of water conservancy projects inspected by unmanned aerial vehicles (UAVs) provided in this application acquire time-series image streams of the water conservancy project surface using a UAV equipped with a multispectral sensor, and simultaneously obtain positioning and attitude data to provide basic data support for subsequent processing. Based on a preset lightweight BIM model, feature extraction and spatial transformation are performed on the time-series image stream to generate projection relationships. This process effectively improves the spatial registration accuracy between images and the BIM model by associating image features with the BIM model.

[0045] In the fusion scenario after registration, defect identification of the time-series image stream is performed using projection relationships in a human-machine collaborative manner. This combines the accuracy of human judgment with the efficiency of AI processing, optimizing the efficiency of human-machine collaborative identification. Based on the component IDs of the lightweight BIM model, spatiotemporal correlation processing of component-related defect information is performed and a structured database is generated. This enables scattered defect data to form an organic association according to the component dimension, thereby enhancing the spatiotemporal analysis capability of component-level defects. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 This is a flowchart of an image recognition method for unmanned aerial vehicle (UAV) inspection of water conservancy projects according to one embodiment of the present invention.

[0048] Figure 2 In one embodiment of the present invention, in response to a three-dimensional bounding operation on the surface of a lightweight BIM model, image defect feature extraction processing is performed on the bounding area to generate a flowchart of manually annotated defect areas.

[0049] Figure 3 This is a structural diagram of a drone inspection image recognition device for water conservancy projects according to one embodiment of the present invention. Detailed Implementation

[0050] To make the above-mentioned objects, features, and advantages of this application more apparent and understandable, the specific embodiments of this application will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of this application. However, this application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the application. Therefore, this application is not limited to the specific embodiments disclosed below.

[0051] First, the application scenarios of the embodiments of this application are described. In the embodiments of this application, a method, device, equipment and medium for image recognition of water conservancy projects using drone inspection is provided, which is applicable to but not limited to the scenario of safety inspection of water conservancy projects. It can be used for the identification and data management of defects such as cracks on the surface of dams, seepage marks on dikes, and damage to components of water conservancy structures, providing technical support for daily maintenance, health status assessment and risk warning of the project.

[0052] The image recognition method, device, equipment and medium for water conservancy projects inspected by drones provided in this application embodiment can also be applied to other similar engineering scenarios, such as inspection of structural defects in hydropower plant buildings, identification of damage to irrigation canal facilities, and investigation of potential hazards in river and lake shoreline protection projects. This is only an example and does not limit the specific application scenarios.

[0053] like Figure 1 As shown, this application provides a method for image recognition of water conservancy projects inspected by unmanned aerial vehicles (UAVs). The method includes:

[0054] S101: The UAV uses a pre-set multispectral sensor to collect a time-series image stream of the surface of the water conservancy project, and simultaneously acquires the positioning and attitude data of each frame in the time-series image stream.

[0055] For example, the adaptation and debugging of the UAV and the preset multispectral sensor are completed to ensure that the sensor is stably mounted and completes data interaction configuration with the UAV flight control system. At the same time, the flight path of the UAV is planned according to the inspection range of the water conservancy project, and key parameters such as flight altitude and speed are determined. Then, the UAV and the preset multispectral sensor are started. The UAV flies according to the planned path, and the preset multispectral sensor continuously collects images of the surface of the water conservancy project to form a time-series image stream. During this process, the UAV positioning unit and attitude sensing unit acquire the positioning data and attitude data of each frame of image acquisition in real time, and the flight control system associates and stores the two types of data with the corresponding images to obtain an initial dataset including the time-series image stream of the water conservancy project surface, where each frame of the time-series image stream is synchronously associated with the corresponding positioning data and attitude data.

[0056] S102: Based on the preset lightweight BIM model of water conservancy project, feature extraction and spatial transformation processing are performed on the time series image stream to generate projection relationship.

[0057] For example, a pre-processed lightweight BIM model of a water conservancy project is used to confirm the completeness and accuracy of core information such as the spatial topological relationships of components and the vertex set of surface meshes. Radiometric distortion correction and illumination equalization preprocessing are performed on the time-series image stream to ensure image quality. Local invariant features are extracted from the pre-processed time-series image stream to form a set of local invariant feature points.

[0058] The initial exterior orientation parameters are determined by iteratively matching the set of locally invariant feature points with the surface mesh vertices of the lightweight BIM model of the water conservancy project. The initial exterior orientation parameters are then processed to remove mismatches, resulting in optimized exterior orientation parameters. Based on these optimized parameters, a perspective projection transformation is performed to establish a correspondence rule between the pixel coordinates of the time-series image stream and the spatial coordinates of the lightweight BIM model of the water conservancy project, thus generating a projection relationship.

[0059] S103: In the fusion scene after registration, the projection relationship is used to perform defect identification processing on the time-series image stream through human-machine collaboration, and component-related defect information is generated.

[0060] For example, the registered fusion scene is loaded and the generated projection relationship is invoked to complete the parameter configuration, ensuring that the pixel coordinates of the time-series image stream and the spatial coordinates of the lightweight BIM model of the water conservancy project can be accurately mapped. The human-machine collaborative process is initiated, and the operator manually selects suspected defect areas in the fusion scene. The system inversely maps these areas to the time-series image stream according to the projection relationship and extracts features to generate an enhanced defect feature map. After manual confirmation and refinement, manually annotated defect areas are formed. At the same time, a pre-trained defect segmentation network is used to parse the time-series image stream to generate a defect semantic mask map.

[0061] By utilizing projection relationships to perform spatial consistency verification on two types of defect areas, registered defect data is obtained. This data is then converted into spatial coordinates of a lightweight BIM model for hydraulic engineering to generate a 3D defect location set. Based on the spatial topological relationships of model components, the 3D defect location set is matched with the corresponding components, defect attribute information is supplemented, and component-related defect information is generated.

[0062] S104: Based on the component ID of the lightweight BIM model, perform spatiotemporal correlation processing on the component-related defect information to generate a structured database.

[0063] For example, the component IDs of the lightweight BIM model are extracted as core identifiers. Component-related defect information is categorized and collected according to component IDs, integrating component-related defect information from different time periods and batches corresponding to the same component ID. The spatiotemporal attributes of the grouped component-related defect information are parsed. The time dimension is used to extract and sort the collection timestamps, and the spatial dimension is used to extract the three-dimensional coordinates associated with the topological location of the components, generating a spatiotemporal attribute dataset of component defects.

[0064] Based on the aforementioned dataset, the temporal variation patterns and spatial distribution characteristics of defects are analyzed to generate spatiotemporal evolution information of component defects. Combining the component hierarchy relationships of the lightweight BIM model, metadata such as basic component parameters and defect classification labels are supplemented to construct an associated data structure. The field formats of the associated data structure are standardized, storage rules are defined, and a spatiotemporal index is established. Data writing configuration is completed, generating a structured database.

[0065] One embodiment of this application provides a method for image recognition of water conservancy projects during UAV inspections. This method uses a UAV equipped with a multispectral sensor to collect a time-series image stream of the water conservancy project surface and simultaneously acquires positioning and attitude data, providing fundamental data support for subsequent processing. Based on a preset lightweight BIM model, feature extraction and spatial transformation are performed on the time-series image stream to generate projection relationships. This process effectively improves the spatial registration accuracy between the image and the BIM model by associating image features with the BIM model.

[0066] In the fusion scenario after registration, defect identification of the time-series image stream is performed using projection relationships in a human-machine collaborative manner. This combines the accuracy of human judgment with the efficiency of AI processing, optimizing the efficiency of human-machine collaborative identification. Based on the component IDs of the lightweight BIM model, spatiotemporal correlation processing of component-related defect information is performed and a structured database is generated. This enables scattered defect data to form an organic association according to the component dimension, thereby enhancing the spatiotemporal analysis capability of component-level defects.

[0067] In one embodiment, in the registered fusion scene, using projection relationships, defect identification processing is performed on the temporal image stream through human-machine collaboration to generate component-related defect information, including:

[0068] (1) Responding to the three-dimensional selection operation on the surface of the lightweight BIM model, the selected area is processed by image defect feature extraction to generate manually annotated defect areas.

[0069] For example, in the registered fusion scenario, the system monitors interactive operations on the surface of the lightweight BIM model in real time. When a 3D selection operation is detected, the system immediately responds to the operation and parses the spatial range parameters covered by the 3D selection operation to determine the spatial coordinate boundary of the selected area on the lightweight BIM model surface. Then, the image feature extraction unit is invoked to inversely map the spatial coordinate boundary of the selected area on the lightweight BIM model surface to the time-series image stream based on the generated projection relationship, thereby locating the target image region in the time-series image stream corresponding to the selected area.

[0070] Multi-dimensional feature extraction is performed on the target image region, including texture features, spectral features, and grayscale variation features. A feature enhancement algorithm is then used to highlight potential defect features in the target image region, generating a preliminary defect feature map. This preliminary defect feature map is then pushed to a human interaction platform where humans confirm the defect contours and types, manually refine blurred defect boundaries, and generate manually annotated defect areas.

[0071] Among them, multi-dimensional feature extraction includes texture features, spectral features, and grayscale variation features, which are key visual information that can reflect the existence of defects extracted from the target image region of the time-series image stream. Feature enhancement algorithms are used to improve the recognition of the above key information.

[0072] (2) The temporal shadow stream is processed by pixel-level semantic parsing through a pre-trained defect segmentation network to generate a defect semantic mask.

[0073] For example, a pre-processed time-series image stream is acquired, and the time-series image stream is segmented into single-frame image data, ensuring that the resolution, color channels, and other parameters of each frame of image data meet the input requirements of the pre-trained defect segmentation network. The single-frame image data is then sequentially input into the pre-trained defect segmentation network. This network, through the feature patterns of common defects in water conservancy projects learned in the previous training, performs pixel-by-pixel scanning and semantic classification of the single-frame image data, determining whether each pixel belongs to a defect region and the corresponding defect type.

[0074] After completing the semantic parsing of all single-frame image data, the system integrates the semantic classification results of each frame to generate a frame-level defect semantic mask that includes the location of the defect area and the defect type identifier. Following the acquisition order of the temporal image stream, the frame-level defect semantic masks are concatenated to generate a defect semantic mask corresponding to the temporal image stream covering the entire inspection process.

[0075] The preprocessed temporal image stream refers to the image stream after operations such as radiometric distortion correction and illumination equalization. The pre-trained defect segmentation network is a neural network model with defect recognition capabilities, trained using a large amount of water conservancy project defect sample data. Common defects in water conservancy projects include cracks, leakage marks, and component damage.

[0076] (3) Using projection relationships, spatial coordinate mapping is performed on manually annotated defect areas and defect semantic mask maps to generate component-related defect information.

[0077] For example, the generated projection relationship is invoked to extract the pixel coordinate sets of manually annotated defect areas in the time-series image stream and the pixel coordinate sets in the defect semantic mask. These two sets of pixel coordinates are then converted into lightweight BIM model spatial coordinate sets using the projection relationship, resulting in the BIM spatial coordinate sets corresponding to the manually annotated defect areas and the defect semantic mask. Spatial overlap and consistency checks are performed on the two sets of BIM spatial coordinate sets. Coordinate data with overlap below a set threshold are removed, and valid coordinate data with matching spatial positions are retained to generate a unified defect spatial coordinate set.

[0078] Based on the component spatial topology of the lightweight BIM model, the system queries the component to which each coordinate point belongs in the defect spatial coordinate set and obtains the corresponding component ID. It then associates and binds information such as defect spatial coordinates, defect type, and acquisition time with the component ID, supplements the component's basic attribute information, and generates a complete component-related defect record. All component-related defect records are then organized according to a preset data structure specification, including field definitions, data type unification, and logical relationship verification, completing data writing and storage configuration to generate component-related defect information.

[0079] Among them, spatial overlap calculation is a key means to determine the spatial consistency of two sets of coordinate data, while the preset data structure specification is the basis for ensuring that the component-related defect records can be stored in an orderly manner and retrieved efficiently.

[0080] In one embodiment, spatial coordinate mapping is performed on manually annotated defect areas and defect semantic masks using projection relationships to generate component-associated defect information, including:

[0081] (1) Perform spatial consistency verification on manually annotated defect areas and defect semantic mask maps to generate registered defect data.

[0082] For example, the pixel coordinate ranges of manually annotated defect regions in the time-series image stream and the pixel coordinate ranges in the defect semantic mask are extracted to determine the corresponding pixel coordinate sets for each type of region. A spatial consistency check algorithm is then invoked to calculate the area ratio of the spatially overlapping region between the pixel coordinate sets of the manually annotated defect regions and the pixel coordinate sets of the defect semantic mask. Simultaneously, the geometric similarity of the defect contours in the two types of regions is compared, including the edge smoothness of the contours, the number of corners, and the overall shape matching degree.

[0083] If both the area ratio and geometric similarity of the spatially overlapping region reach the preset verification threshold, the two types of regions are determined to be spatially consistent, and the corresponding defect data is retained. If either indicator fails to reach the preset verification threshold, the differing regions are marked and invalid data is removed. Then, all the defect data that passed the verification are integrated to generate registration defect data.

[0084] Among them, the spatial consistency verification algorithm is the core tool for realizing spatial matching judgment of two types of defect areas. The preset verification threshold is a judgment standard set according to the accuracy requirements of defect identification in water conservancy projects, which is used to filter out valid defect data that match in both spatial location and shape.

[0085] (2) Based on the projection relationship, the registered defect data is transformed from pixels to BIM coordinates to generate a three-dimensional defect location set.

[0086] For example, the generated projection relationship and registration defect data are obtained, and the pixel coordinate information of each defect region is extracted from the registration defect data, including the pixel coordinates of the defect boundary and the pixel coordinates of feature points inside the defect. Based on the correspondence rules between the pixel coordinates of the temporal image stream and the spatial coordinates of the lightweight BIM model in the projection relationship, the extracted pixel coordinates of the defect region are input one by one into the projection transformation unit, and the pixel coordinates are converted into three-dimensional spatial coordinates in the coordinate system of the lightweight BIM model through the coordinate transformation algorithm.

[0087] During the conversion process, the accuracy of each converted 3D spatial coordinate is verified to ensure that the coordinate values ​​meet the spatial scale requirements of the lightweight BIM model and to avoid coordinate offset due to conversion errors. After the pixel coordinates of all defect areas have been converted and verified, the 3D spatial coordinates are classified and organized according to the defect category to generate a 3D defect location set that includes complete 3D spatial location information for each defect.

[0088] Among them, the projection transformation unit is a functional unit that performs the conversion from pixel coordinates to BIM coordinates, the coordinate transformation algorithm is a specific method for realizing coordinate mapping between different coordinate systems based on the projection relationship, and the accuracy verification is used to ensure the accuracy of the three-dimensional spatial coordinates after conversion.

[0089] (3) By using the component spatial topology relationship of the lightweight BIM model, component matching processing is performed on the three-dimensional defect location set to generate component-related defect information.

[0090] For example, a lightweight BIM model is acquired, and the spatial topology data of all components in the model is extracted, including the spatial location range of the components, the adjacency relationship between components, and the hierarchical structure of the components in the model. The three-dimensional spatial coordinates of each defect are extracted from the three-dimensional defect location set, and the three-dimensional spatial coordinates of each defect are compared with the spatial location range of each component in the lightweight BIM model to determine whether the three-dimensional spatial coordinates of the defect fall within the spatial location range of a certain component.

[0091] If the three-dimensional spatial coordinates of a defect fall within the spatial location range of a certain component, the component ID corresponding to that component is obtained, and the three-dimensional spatial coordinates, defect type, defect size, and other information of the defect are associated with that component ID. After all defects have completed component matching and information association, the associated data is standardized, the data format and field definitions are unified, and metadata information such as defect acquisition time and acquisition device number is supplemented. Finally, all associated data is integrated to generate component-associated defect information.

[0092] Among them, the component spatial topology relationship data of the lightweight BIM model is the basis for matching defects with components, the standardization process is used to ensure the data integrity and consistency of component-related defect information, and the metadata information provides supplementary background for subsequent defect data management and analysis.

[0093] like Figure 2 As shown, in response to a 3D bounding box selection operation on the surface of the lightweight BIM model, image defect feature extraction processing is performed on the selected area to generate manually annotated defect areas, including:

[0094] S201: Perform spatial coordinate analysis on the 3D bounding box selection operation to generate the bounding box spatial range.

[0095] For example, in the registered fusion scenario, when the system detects a 3D bounding box selection operation on the surface of the lightweight BIM model, it initiates the spatial coordinate analysis process. It calls the coordinate system parameters of the lightweight BIM model to extract the coordinates of multiple spatial control points selected by the user in the 3D bounding box selection operation, including the coordinates of the boundary vertices and internal reference points of the selected area. Through spatial geometric calculation methods, the discrete control point coordinates are fitted into a continuous spatial geometric region, determining the maximum and minimum spatial coordinate values ​​of this region in the lightweight BIM model coordinate system, clarifying the length, width, and height dimensions of the region, and generating the bounding box spatial range.

[0096] Among them, the coordinate system parameters of the lightweight BIM model are the benchmark for analyzing spatial coordinates, the spatial geometric calculation method is used to transform discrete control points into continuous spatial regions, and the coordinates of boundary vertices and internal reference points together ensure the accuracy of the selected spatial range.

[0097] S202: Based on the projection relationship, perform two-dimensional image inverse processing on the selected spatial range to generate the target image region.

[0098] For example, the generated projection relationship and selected spatial range are obtained. The mapping rules between the pixel coordinates of the time-series image stream and the spatial coordinates of the lightweight BIM model are extracted from the projection relationship, including the coordinate transformation matrix and parameter coefficients. All spatial coordinate points within the selected spatial range are substituted into the inverse mapping algorithm of the projection relationship, and the corresponding pixel coordinates in the time-series image stream are calculated one by one. All calculated pixel coordinates are filtered and sorted to determine the maximum and minimum values, thus delineating the rectangular image region in the time-series image stream corresponding to the selected spatial range. The integrity of this rectangular image region is verified to ensure that the image data within the region is complete and undistorted, generating the target image region.

[0099] Among them, the inverse mapping algorithm of projection relationship is the core of realizing the transformation from spatial coordinates to pixel coordinates, the coordinate transformation matrix and parameter coefficients are the key basis for mapping calculation, and integrity verification is used to ensure the image quality of the target image area.

[0100] S203: Perform texture-spectral joint feature analysis on the target image region to generate an enhanced defect feature map.

[0101] For example, image data of the target image region is acquired, and the texture-spectral joint feature analysis unit is invoked to extract texture features from the target image region. Methods such as gray-level co-occurrence matrix and local binary mode are used to capture the texture variation patterns in the image, identifying potential defective texture regions that differ significantly from the surrounding areas. Spectral features of the target image region are extracted. Based on spectral data from different bands acquired by a multispectral sensor, parameters such as the spectral reflectance and inter-band differences of pixels within the region are analyzed to screen out potential defective spectral regions with abnormal spectral characteristics.

[0102] The texture feature extraction results are fused with the spectral feature extraction results. By feature overlay and weight allocation, the feature information of potential defect areas is highlighted, and background interference information is suppressed to generate a preliminary defect feature map. The preliminary defect feature map is then subjected to contrast enhancement and noise filtering to further improve the clarity of the defect features, generating an enhanced defect feature map.

[0103] Among them, gray-level co-occurrence matrix and local binary mode are commonly used methods for extracting texture features. Spectral data from different bands of multispectral sensors are the basis for spectral feature analysis. Feature fusion processing is used to integrate two types of feature information to improve the accuracy of defect identification.

[0104] S204: In response to manual confirmation, refine the boundaries of the enhanced defect feature map and generate manually annotated defect areas.

[0105] For example, the enhanced defect feature map is pushed to the human interaction interface, and the system enters a human confirmation waiting state. Once the human completes the defect area confirmation operation in the interaction interface, including marking valid defect areas, eliminating falsely judged areas, and labeling the defect type, the system responds to the human confirmation operation and obtains the boundary information of the manually marked defect area. A boundary refinement algorithm is invoked to adjust the boundary of the manually marked defect area at the pixel level. Edge detection technology is used to identify the true edge pixels of the defect, correcting problems such as boundary offset and jagged edges that may occur during the manual marking process.

[0106] The boundaries of the refined defect areas are smoothed to ensure that the boundary contours are continuous and regular. The refined defect areas are then associated with and stored with information such as the corresponding defect type and manual confirmation time to generate manually annotated defect areas.

[0107] Among them, the human interaction interface is the operation platform for manual defect confirmation and marking, the boundary refinement algorithm and edge detection technology are used to improve the accuracy of the defect area boundary, and the smoothing process is used to optimize the integrity of the defect area contour.

[0108] In one embodiment, based on a preset lightweight BIM model for water conservancy projects, feature extraction and spatial transformation processing are performed on the time-series image stream to generate projection relationships, including:

[0109] (1) Perform radiation distortion correction and illumination equalization on the time-series image stream to generate preprocessed images.

[0110] For example, a time-series image stream of the surface of a water conservancy project is acquired by a preset multispectral sensor mounted on a UAV. A radiation distortion correction algorithm is called to eliminate image grayscale deviations caused by factors such as sensor optical system errors and electronic noise based on the radiation response characteristic parameters of the multispectral sensor, so that the grayscale values ​​of each pixel in the time-series image stream can accurately reflect the actual radiation information of the surface of the water conservancy project.

[0111] Next, illumination equalization processing is initiated to analyze the differences in illumination intensity distribution among different frames in the time-series image stream. Histogram equalization or adaptive contrast enhancement methods are used to adjust the brightness and contrast of the images to ensure that the images under different acquisition periods and illumination conditions maintain a consistent illumination level, thus avoiding interference from uneven illumination to subsequent feature extraction and generating preprocessed images.

[0112] Among them, the radiation response characteristics of multispectral sensors are the core basis for radiation distortion correction, and histogram equalization and adaptive contrast enhancement methods are commonly used techniques to achieve illumination equalization.

[0113] (2) Perform local invariant feature extraction on the preprocessed image to generate a set of local invariant feature points.

[0114] For example, a preprocessed image is acquired, and a local invariant feature extraction algorithm is invoked to construct a multi-scale space for the preprocessed image. Key points in the image, including corner points, edge points, and texture-dense points, are detected at different scales. For each detected key point, feature information such as gradient direction and gradient magnitude within its neighborhood is calculated, and a feature descriptor with rotation invariance and scale invariance is constructed to ensure that the key point features remain stable when the image is rotated, scaled, or subjected to changes in illumination.

[0115] The feature descriptors of all key points are organized and stored to generate a set of locally invariant feature points, including key point coordinates and feature descriptors. This provides a feature basis for subsequent matching with time-series image streams.

[0116] Among them, multi-scale space construction is used to comprehensively detect key points at different scales, and rotation-invariant and scale-invariant feature descriptors are key attributes to ensure the stability of local invariant feature point sets.

[0117] (3) Based on the surface mesh vertex set of the lightweight BIM model, the local invariant feature point set is iteratively matched to generate the initial exterior orientation parameters.

[0118] For example, a pre-defined vertex set of the surface mesh of a lightweight BIM model for water conservancy projects is extracted. This vertex set contains the vertex coordinate information of all mesh cells on the surface of the lightweight BIM model, accurately reflecting the surface geometry of the model. The generated set of locally invariant feature points and the vertex set of the surface mesh of the lightweight BIM model are imported into a feature matching unit. A nearest neighbor matching algorithm is used to calculate the similarity between the feature descriptor of each feature point in the locally invariant feature point set and the features of each vertex in the surface mesh vertex set.

[0119] Initial matching pairs are selected based on similarity thresholds. The initial matching pairs are iteratively verified using a random sampling consensus algorithm to eliminate erroneous matching pairs caused by factors such as noise and occlusion. At the same time, the relative attitude and position parameters between the time-series image stream and the lightweight BIM model of the water conservancy project are estimated to generate initial exterior orientation parameters.

[0120] Among them, the surface mesh vertex set of the lightweight BIM model of water conservancy project is the geometric benchmark for feature matching. The nearest neighbor matching algorithm is used to calculate feature similarity, and the random sampling consensus algorithm is used to improve matching accuracy and estimate exterior orientation parameters.

[0121] (4) Perform mismatch removal on the initial exterior orientation parameters to generate optimized exterior orientation parameters.

[0122] For example, the generated initial exterior orientation parameters are obtained, and the matching residuals corresponding to the initial exterior orientation parameters are analyzed. That is, the deviation values ​​between the local invariant feature point set and the surface mesh vertex set after mapping the local invariant feature point set to the coordinate system of the lightweight BIM model of water conservancy engineering according to the initial exterior orientation parameters are analyzed. A residual threshold is set, and exterior orientation parameter data with matching residuals greater than the residual threshold are marked as suspected mismatch data.

[0123] A robust estimation method is employed to recalculate and verify suspected mismatched data to determine whether they are genuine mismatches. After removing confirmed mismatched data, the remaining exterior orientation parameters are optimized by adjusting the parameter values ​​using the least squares method to minimize the overall matching residuals. This ensures that the exterior orientation parameters accurately reflect the spatial relationship between the temporal image stream and the lightweight BIM model of the water conservancy project, generating optimized exterior orientation parameters.

[0124] Among them, the matching residual is an important indicator for judging the accuracy of the exterior orientation parameters. The residual threshold is used to filter suspected mismatch data. Robust estimation methods and least squares method are the core technologies for realizing mismatch elimination and parameter optimization.

[0125] (5) Perform perspective projection transformation based on optimized exterior orientation parameters to generate projection relationships.

[0126] For example, the generated optimized exterior orientation parameters are obtained, which include interior orientation elements (such as focal length and principal point coordinates) and exterior orientation elements (such as position and attitude angle) of the time-series image stream. Based on the principle of perspective projection, a mathematical relationship between the optimized exterior orientation parameters and coordinate transformation is established, clarifying the mapping rules between pixel coordinates in the time-series image stream and spatial coordinates of the lightweight BIM model of the water conservancy project.

[0127] By substituting optimized exterior orientation parameters into the perspective projection transformation formula, the conversion process from pixel coordinates to BIM spatial coordinates is derived, including coordinate projection, scale transformation, and coordinate system alignment. The conversion process is then validated by selecting feature points from a set of locally invariant feature points and converting their pixel coordinates to BIM spatial coordinates using this process. These coordinates are then compared with the surface mesh vertex set of the lightweight BIM model for water conservancy projects. After confirming that the conversion accuracy meets the requirements, the mapping rules and conversion process are solidified, and projection relationships are generated.

[0128] Among them, the interior orientation element and the exterior orientation element are the key parameters of perspective projection transformation. The principle of perspective projection is the theoretical basis for establishing coordinate mapping relationship. The transformation accuracy verification is used to ensure the reliability of the projection relationship.

[0129] In one embodiment, based on the component ID of the lightweight BIM model, spatiotemporal correlation processing is performed on the component-related defect information to generate a structured database, including:

[0130] (1) Based on the component ID, spatiotemporal clustering is performed on the component-related defect information to generate a component defect sequence.

[0131] For example, the component IDs of all components in the lightweight BIM model are extracted, and the component-related defect information is grouped by component ID to ensure that all component-related defect information corresponding to the same component ID is grouped into the same data group. Within each data group, the time and spatial attributes of the component-related defect information are extracted. The time attributes include the time of first defect discovery and the time of each inspection, and the spatial attributes include the three-dimensional coordinates of the defect in the lightweight BIM model and the defect's impact range.

[0132] A spatiotemporal clustering algorithm is used to sort the component-related defect information under the same component ID in chronological order. At the same time, the defects are clustered according to the proximity of spatial coordinates, and defects belonging to the same evolution process are grouped into one category to form a defect set arranged on the time axis, generating a component defect sequence.

[0133] Among them, the spatiotemporal clustering algorithm is the core tool for sorting defect information by time and clustering it spatially. Time attributes and spatial attributes are the key basis for clustering, and component ID is the unique identifier that ensures the accuracy of data grouping.

[0134] (2) Perform evolution trend analysis and spatial distribution pattern recognition on the component defect sequence to generate multidimensional defect evolution characteristics.

[0135] For example, a sequence of component defects is generated, and an evolution trend analysis is performed on each component defect sequence over time. Temporal features such as changes in the number of defects, the rate of size growth, and the severity level are extracted. The development pattern of defects over time is identified through trend fitting methods, such as uniform expansion, accelerated deterioration, or stabilization. Simultaneously, spatial distribution pattern identification is performed, analyzing spatial characteristics such as the distribution density, clustering areas, and expansion direction of defects on the component surface. Spatial statistical methods are used to summarize the spatial distribution type of defects, such as single-point clustering, linear distribution, or planar diffusion.

[0136] By integrating the evolution trend characteristics of the time dimension with the distribution pattern characteristics of the spatial dimension, and classifying and organizing them according to the preset feature dimension framework (such as time characteristics, spatial characteristics, and attribute characteristics), a feature set containing multi-dimensional information is formed, generating multi-dimensional defect evolution characteristics.

[0137] Among them, the trend fitting method is used to capture the temporal evolution of defects, the spatial statistical method is used to identify the spatial distribution pattern of defects, and the preset feature dimension framework is the structural basis for integrating multi-dimensional features.

[0138] (3) Based on the component topology relationship of the lightweight BIM model, the multidimensional defect evolution features are semantically associated to generate a defect knowledge graph.

[0139] For example, the component topology of the lightweight BIM model is obtained, including structural information such as dependency, connection, and adjacency relationships between components. The generated multidimensional defect evolution features are associated with the corresponding component IDs, and the associated components of each component are found based on the component topology. The correlation between the multidimensional defect evolution features of associated components is analyzed, such as the consistency of defect types, the synchronicity of evolution trends, and the correlation of spatial distribution, establishing semantic connections between defect features of different components. Using knowledge representation methods, component IDs, multidimensional defect evolution features, component topology relationships, and semantic connections are transformed into structured knowledge units. Nodes represent components and defect features, and edges represent the relationships between them, generating a defect knowledge graph.

[0140] In lightweight BIM models, the component topology is the foundation for establishing component relationships. Knowledge representation methods are a technical means of transforming defect information into a graph structure, and semantic connections reflect the inherent relationships between the defect characteristics of different components. Each component's associated components include parent components, child components, and adjacent components.

[0141] (4) Spatiotemporal indexing and relation mapping are performed on the defect knowledge graph to generate a structured database.

[0142] For example, the generated defect knowledge graph is obtained, and the spatiotemporal attributes of all nodes and edges in the graph are analyzed. The temporal attributes include the generation time of the knowledge unit and the occurrence time of the associated event, while the spatial attributes include the spatial location of the component and the spatial range of the defect. A spatiotemporal index is constructed based on the spatiotemporal attributes. The time index uses a time axis segmentation method to achieve fast querying by time range, and the spatial index uses a spatial grid division method to achieve fast positioning by spatial region.

[0143] The relationships in the defect knowledge graph are mapped, transforming semantic relationships into database-storable association rules, clarifying the reference and dependency relationships between different data entities. Following database design specifications, the node data, edge data, and association rules of the defect knowledge graph are converted into table structures, defining field types, constraints, and indexing methods, completing the data import operation, and generating a structured database.

[0144] Among them, spatiotemporal indexes are a key structure for improving database query efficiency, association rules are an important form of storing relationships between data entities, and database design specifications are the foundation for ensuring the standardization and usability of structured databases.

[0145] In one embodiment, the component-associated defect information is obtained using the following formula, including:

[0146]

[0147] in, Indicates component-related defect information. Represents a unique identifier for a component. This indicates that an aggregation operation is performed on all defect instances. Represents the coordinate standardization function. The generalized inverse operator representing the projection relation, Represents the two-dimensional homogeneous coordinates of the defect point. This represents the defect feature extraction function. This represents the characteristics of the defect area.

[0148] For example, for each defect instance, a unique component identifier is determined. This clarifies the component to which the defect belongs, laying the foundation for subsequent correlation. The two-dimensional homogeneous coordinates of the defect point are obtained. Using the generalized inverse operator of projection relations Inversely map it to three-dimensional space to obtain three-dimensional coordinates, and then use the coordinate normalization function. The 3D coordinates are standardized to conform to a unified standard. Defect point domain features are then extracted. Using defect feature extraction function The feature was analyzed and calculated to extract key defect features. These features were then analyzed according to the component's unique identifier. The system performs an aggregation operation on all processed defect instances, integrating standardized three-dimensional spatial coordinates of the same component, extracted defect features, and other information to obtain component-related defect information.

[0149] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0150] In one embodiment, such as Figure 3 As shown, this application also provides an image recognition device 300 for unmanned aerial vehicle (UAV) inspection of water conservancy projects, the device 300 comprising:

[0151] The UAV image acquisition module 301 is used to acquire a time-series image stream of the surface of a water conservancy project through a preset multispectral sensor carried by the UAV, and simultaneously acquire the positioning data and attitude data of each frame of the time-series image stream.

[0152] The projection relationship generation module 302 is used to perform feature extraction and spatial transformation processing on the time-series image stream based on the preset lightweight BIM model of water conservancy projects, and generate projection relationships.

[0153] The human-machine collaborative defect recognition module 303 is used to perform defect recognition processing on the time-series image stream in the registered fusion scene by utilizing projection relationship and through human-machine collaboration, and generate component-related defect information.

[0154] The structured database generation module 304 is used to perform spatiotemporal correlation processing on component-related defect information based on the component ID of the lightweight BIM model to generate a structured database.

[0155] Specifically, the UAV image acquisition module 301 uses a pre-set multispectral sensor mounted on the UAV to acquire a time-series image stream of the surface of the water conservancy project, simultaneously acquiring the positioning and attitude data of each frame of the image to provide a spatiotemporal reference for subsequent processing. The projection relationship generation module 302, based on the lightweight BIM model of the water conservancy project, performs feature extraction and spatial transformation on the time-series image stream to generate projection relationships and establish coordinate mapping rules between the images and the model.

[0156] In the registered fusion scenario, the human-machine collaborative defect recognition module 303 utilizes projection relationships to extract and refine defect features through manual selection. Combined with a defect semantic mask generated by a pre-trained defect segmentation network, and through spatial coordinate mapping, it associates defects with components, generating component-associated defect information. The structured database generation module 304, based on component IDs, performs spatiotemporal clustering of component-associated defect information to generate component defect sequences. It analyzes their evolution trends and distribution patterns to obtain multidimensional defect evolution characteristics. Based on component topological relationships, it constructs a defect knowledge graph, and through spatiotemporal indexing and relationship mapping, generates a structured database, facilitating defect data management and analysis.

[0157] The human-machine collaborative defect identification module 303 is also used for:

[0158] In response to a 3D bounding box selection operation on the surface of a lightweight BIM model, the selected area is processed by image defect feature extraction to generate manually annotated defect areas.

[0159] A pre-trained defect segmentation network is used to perform pixel-level semantic parsing of the temporal shadow stream to generate a defect semantic mask map.

[0160] By utilizing projection relationships, spatial coordinate mapping is performed on manually annotated defect areas and defect semantic masks to generate component-related defect information.

[0161] The human-machine collaborative defect identification module 303 is also used for:

[0162] Spatial consistency verification is performed on manually annotated defect areas and defect semantic masks to generate registered defect data.

[0163] Based on the projection relationship, the registered defect data is transformed from pixels to BIM coordinates to generate a three-dimensional defect location set.

[0164] By optimizing the spatial topological relationships of components in a lightweight BIM model, component matching is performed on the set of three-dimensional defect locations to generate component-related defect information.

[0165] The human-machine collaborative defect identification module 303 is also used for:

[0166] Perform spatial coordinate analysis on the 3D bounding box selection operation to generate the bounding box spatial range;

[0167] Based on the projection relationship, the selected spatial range is subjected to two-dimensional image inverse processing to generate the target image region.

[0168] Texture-spectral joint feature analysis is performed on the target image region to generate an enhanced defect feature map;

[0169] In response to manual confirmation, the enhanced defect feature map is refined to generate manually annotated defect areas.

[0170] The projection relationship generation module 302 is also used for:

[0171] Radiometric distortion correction and illumination equalization are performed on the temporal image stream to generate preprocessed images;

[0172] Local invariant feature extraction is performed on the preprocessed image to generate a set of local invariant feature points;

[0173] Based on the surface mesh vertex set of the lightweight BIM model, iterative matching processing is performed on the local invariant feature point set to generate initial exterior orientation parameters.

[0174] The initial exterior orientation parameters are processed to remove mismatches, and optimized exterior orientation parameters are generated.

[0175] Perspective projection transformation is performed based on optimized exterior orientation parameters to generate projection relationships.

[0176] The structured database generation module 304 is also used for:

[0177] Based on the component ID, spatiotemporal clustering is performed on the component-related defect information to generate a component defect sequence;

[0178] Evolution trend analysis and spatial distribution pattern recognition of component defect sequences are performed to generate multidimensional defect evolution characteristics;

[0179] Based on the component topology relationship of the lightweight BIM model, the multidimensional defect evolution features are semantically associated to generate a defect knowledge graph.

[0180] Spatiotemporal indexing and relation mapping are performed on the defect knowledge graph to generate a structured database.

[0181] The human-machine collaborative defect identification module 303 is also used for:

[0182]

[0183] in, Indicates component-related defect information. Represents a unique identifier for a component. This indicates that an aggregation operation is performed on all defect instances. Represents the coordinate standardization function. The generalized inverse operator representing the projection relation, Represents the two-dimensional homogeneous coordinates of the defect point. This represents the defect feature extraction function. This represents the characteristics of the defect area.

[0184] In one embodiment, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0185] In one embodiment, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the above-described method embodiments.

[0186] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The components described as separate parts may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0187] The above-described embodiments are merely illustrative of several implementation methods of the embodiments of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the embodiments of this application, and these modifications and improvements all fall within the protection scope of the embodiments of this application.

Claims

1. A method for image recognition of water conservancy projects inspected by unmanned aerial vehicles (UAVs), characterized in that, The method includes: The drone uses a pre-set multispectral sensor to collect a time-series image stream of the surface of the water conservancy project, and simultaneously acquires the positioning data and attitude data of each frame in the time-series image stream. Based on a preset lightweight BIM model for water conservancy projects, feature extraction and spatial transformation processing are performed on the time-series image stream to generate projection relationships. In the fusion scene after registration, the projection relationship is used to perform defect identification processing on the temporal image stream through human-machine collaboration, and component-related defect information is generated; Based on the component IDs of the lightweight BIM model, spatiotemporal correlation processing is performed on the component-related defect information to generate a structured database. 2.The method of claim 1, wherein, In the fusion scene after registration, the projection relationship is used to perform defect identification processing on the temporal image stream through human-machine collaboration, generating component-related defect information, including: In response to a 3D bounding operation on the surface of the lightweight BIM model, image defect feature extraction processing is performed on the bounding area to generate manually annotated defect areas. The temporal video stream is subjected to pixel-level semantic parsing through a pre-trained defect segmentation network to generate a defect semantic mask map. Using the projection relationship, spatial coordinate mapping is performed on the manually annotated defect area and the defect semantic mask to generate the component-related defect information. 3.The method of claim 2, wherein, The step of using the projection relationship to perform spatial coordinate mapping processing on the manually annotated defect region and the defect semantic mask to generate the component-related defect information includes: Spatial consistency verification is performed on the manually annotated defect regions and the defect semantic mask to generate registered defect data. Based on the projection relationship, the registered defect data is transformed from pixels to BIM coordinates to generate a three-dimensional defect location set. By using the component spatial topology relationship of the lightweight BIM model, component matching processing is performed on the three-dimensional defect location set to generate the component-related defect information. 4.The method of claim 2, wherein, The response to the 3D bounding box operation on the surface of the lightweight BIM model involves extracting image defect features from the bounding box area and generating manually annotated defect areas, including: The three-dimensional bounding box operation is subjected to spatial coordinate analysis to generate the bounding box spatial range; Based on the projection relationship, the selected spatial range is subjected to two-dimensional image inverse processing to generate the target image region; The target image region is subjected to joint texture-spectral feature analysis to generate an enhanced defect feature map; In response to the manual confirmation operation, the enhanced defect feature map is subjected to boundary refinement processing to generate the manually annotated defect region.

5. The method of claim 1, wherein, The aforementioned lightweight BIM model for water conservancy projects, based on a pre-set model, performs feature extraction and spatial transformation processing on the time-series image stream to generate projection relationships, including: The time-series image stream is subjected to radiation distortion correction and illumination equalization processing to generate a preprocessed image; The preprocessed image is subjected to local invariant feature extraction processing to generate a set of local invariant feature points; Based on the surface mesh vertex set of the lightweight BIM model, the local invariant feature point set is iteratively matched to generate initial exterior orientation parameters. The initial exterior orientation parameters are subjected to mismatch elimination processing to generate optimized exterior orientation parameters; Based on the optimized exterior orientation parameters, perspective projection transformation is performed to generate the projection relationship. 6.The method of claim 1, wherein, The component IDs based on the lightweight BIM model are used to perform spatiotemporal correlation processing on the component-related defect information to generate a structured database, including: Based on the component ID, spatiotemporal clustering is performed on the component-associated defect information to generate a component defect sequence; The evolution trend analysis and spatial distribution pattern recognition of the component defect sequence are performed to generate multidimensional defect evolution characteristics; Based on the component topology relationship of the lightweight BIM model, the multidimensional defect evolution features are semantically associated to generate a defect knowledge graph. The defect knowledge graph is subjected to spatiotemporal indexing and relation mapping to generate the structured database.

7. The method of claim 2, wherein the method further comprises: The component-related defect information is obtained using the following formula, including: in, Indicates component-related defect information. Represents a unique identifier for a component. This indicates that an aggregation operation is performed on all defect instances. Represents the coordinate standardization function. The generalized inverse operator representing the projection relation, Represents the two-dimensional homogeneous coordinates of the defect point. This represents the defect feature extraction function. This represents the characteristics of the defect area.

8. An image recognition device for water conservancy projects inspected by unmanned aerial vehicles (UAVs), characterized in that, The device includes: The UAV image acquisition module is used to acquire a time-series image stream of the surface of a water conservancy project through a preset multispectral sensor mounted on the UAV, and simultaneously acquire the positioning data and attitude data of each frame in the time-series image stream. The projection relationship generation module is used to perform feature extraction and spatial transformation processing on the time-series image stream based on a preset lightweight BIM model of water conservancy projects, and generate projection relationships. The human-machine collaborative defect identification module is used to perform defect identification processing on the temporal image stream in the registered fusion scene by means of the projection relationship and through human-machine collaboration, and generate component-related defect information. The structured database generation module is used to perform spatiotemporal correlation processing on the component-related defect information based on the component ID of the lightweight BIM model to generate a structured database.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the image recognition method for water conservancy projects inspected by unmanned aerial vehicles as described in any one of claims 1 to 7.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the image recognition method for water conservancy projects by unmanned aerial vehicle inspection as described in any one of claims 1 to 7.

Images