Image defect identification method, system and equipment based on unmanned aerial vehicle inspection, and storage medium

Abstract

The invention discloses an image defect identification method, system and device based on unmanned aerial vehicle inspection, and a storage medium, and the method comprises the steps: generating a space model of a wind turbine generator and a photovoltaic region based on the three-dimensional scanning data of the wind turbine generator and the photovoltaic region and a two-dimensional station base map, and building a three-dimensional space coordinate system based on the space model; marshalling information of the unmanned aerial vehicle unit is determined, the space model of the wind turbine generator and the space model of the photovoltaic area are segmented based on the marshalling information, a plurality of inspection subareas are obtained, and an inspection path is determined based on the inspection subareas and the marshalling information; acquiring multispectral image information acquired based on the inspection path, performing fault identification on the multispectral image, and determining fault features in the multispectral image information; and when the fault feature is the target fault feature, determining position information of the fault feature in the multispectral image information, and obtaining fault positioning information based on the position information and the three-dimensional space coordinate system. According to the invention, the inspection efficiency and the fault identification accuracy of the wind turbine generator and the photovoltaic area can be improved.

Description

Technical Field

[0001] The present invention relates to a method, system, device and storage medium for detecting defects in drone inspection images. Background Art

[0002] Wind turbines are often installed in remote areas or at high altitudes. Traditional manual inspections make it difficult to conduct a comprehensive and detailed inspection of all turbines at once, resulting in long inspection cycles and low efficiency. Furthermore, the workload is high for inspectors, and inspection results are easily affected by human factors, leading to insufficient detection accuracy and the omission of potential problems. Since manual inspections require downtime, this results in extended downtime for wind turbines, significantly impacting power generation. This is especially true during critical periods of wind farm operations, such as seasons with abundant wind resources, where downtime for inspections can significantly impact the economic performance of the wind farm. Manual inspections also require the mobilization of significant manpower and material resources, and the frequent mobilization of personnel leads to high inspection costs. Furthermore, paper records or unstructured electronic document recording methods are not conducive to subsequent data analysis and management, adding additional management costs.

[0003] Drones, with their flexibility and high-speed flight capabilities, can quickly reach wind turbine locations and capture images of key components using high-definition cameras. Combined with image recognition technology, these inspection images can be analyzed in real time to quickly identify defects in key blade components, such as cracks, wear, and foreign matter. This improves inspection efficiency and shortens fault detection and resolution time. Therefore, a defect recognition method, system, device, and storage medium based on drone inspection images are urgently needed to enable fault inspection of wind turbines. Summary of the Invention

[0004] In view of the above existing problems, the present invention is proposed. Therefore, the present invention provides a method, system, device and storage medium for identifying defects in drone inspection images to solve the problems mentioned in the background art.

[0005] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0006] In a first aspect, an embodiment of the present invention provides a method for identifying defects based on drone inspection images, comprising: generating a spatial model of the wind turbine and photovoltaic area based on three-dimensional scanning data of the wind turbine and photovoltaic area by a drone group and a two-dimensional site base map of the wind turbine and photovoltaic area, and establishing a three-dimensional spatial coordinate system based on the spatial model;

[0007] Determining grouping information of the drone group, segmenting the spatial model of the wind turbine group and the photovoltaic area based on the grouping information to obtain a plurality of inspection zones, and determining an inspection path based on the inspection zones and the grouping information;

[0008] Acquire multispectral image information collected based on the inspection path, perform fault identification on the multispectral image, and determine fault features in the multispectral image information;

[0009] When the fault feature is a target fault feature, position information of the fault feature in the multispectral image information is determined, fault location information is obtained based on the position information and the three-dimensional space coordinate system, and the fault location information is output.

[0010] As a preferred solution of the defect recognition method based on drone inspection images described in the present invention, wherein: generating a spatial model of the wind turbine group and photovoltaic area based on the three-dimensional scanning data of the wind turbine group and photovoltaic area by the drone group and the two-dimensional site base map of the wind turbine group and photovoltaic area, and establishing a three-dimensional spatial coordinate system based on the spatial model, includes:

[0011] Matching the three-dimensional scanning data of the wind turbine group and the photovoltaic area by the drone group with the scanning formation information of the drone group, and adding a scanning mark to the three-dimensional scanning data;

[0012] Matching the two-dimensional site base map of the wind turbine and photovoltaic area with the three-dimensional scan data corresponding to the scan mark to obtain a matching result, and rewriting the scan mark according to the matching result to obtain a model fragment mark;

[0013] Arranging the model fragment marks in order, and generating a spatial model by combining the three-dimensional description data corresponding to the model fragment marks and the two-dimensional station base map;

[0014] A three-dimensional space coordinate system is established based on the space model.

[0015] As a preferred solution of the method for defect recognition based on drone inspection images described in the present invention, the grouping information of the drone group is determined, and the spatial model of the wind turbine group and the photovoltaic area is segmented based on the grouping information to obtain multiple inspection zones, including:

[0016] Determining grouping information of the drone, the grouping information including group information and group member identity information;

[0017] Determine the number of members in each group according to the grouping information and the member identity information;

[0018] Determining inspection division weights based on the number of members in each group;

[0019] The spatial model of the wind turbine generator set and the photovoltaic area is divided according to the inspection division weights to obtain a plurality of inspection partitions that are consistent with the number of the groups.

[0020] As a preferred solution of the method for defect recognition based on drone inspection images of the present invention, determining the inspection path based on the inspection partition and the grouping information includes:

[0021] Determining the number of group members according to the grouping information, and dividing the group members into a first sub-group and a second sub-group, wherein the difference in the number of group members between the first sub-group and the second sub-group is no more than one;

[0022] Divide the inspection zone into an inspection matrix with an even number of rows and columns, and determine a first inspection starting point and a second inspection starting point of the inspection matrix, wherein the first inspection starting point corresponds to the first team grouping, and the second inspection starting point corresponds to the second team grouping;

[0023] Generating a first inspection path with the first inspection starting point and the first inspection direction, and generating a second inspection path with the second inspection starting point and the second inspection direction;

[0024] sequentially combining a plurality of the first inspection paths to obtain a first target inspection path;

[0025] A plurality of the second inspection paths are sequentially combined to obtain a second target inspection path.

[0026] The beneficial effect of this preferred technical solution is that the inspection path is determined based on the inspection partition and grouping information, which can ensure that the drone can cover all inspection areas efficiently and comprehensively.

[0027] As a preferred solution of the method for defect recognition based on drone inspection images described in the present invention, wherein: obtaining multispectral image information collected based on the inspection path, performing fault recognition on the multispectral image, and determining the fault characteristics in the multispectral image information include:

[0028] Obtaining visible light image information and infrared image information collected by the drone group based on the inspection path; fusing the visible light image with the infrared image to obtain a multispectral image;

[0029] Perform feature recognition on the multispectral image to determine the physical object features and temperature features in the spectral image; perform feature fusion on the physical object features and the temperature features to obtain a fused image of the photovoltaic station; perform hot spot detection and occlusion detection on the fused image to obtain hot spot detection results and foreign object occlusion results, and determine fault features based on the hot spot detection results and foreign object occlusion detection results.

[0030] As a preferred solution of the defect recognition method based on drone inspection images described in the present invention, wherein: performing hot spot detection and occlusion detection on the fused image to obtain hot spot detection results and foreign body occlusion detection results, and determining fault characteristics based on the hot spot detection results and foreign body occlusion detection results, includes:

[0031] Hot spot detection is performed on the fused image to determine a temperature-differentiated area in the fused image, and a temperature difference between the temperature-differentiated area and the maximum temperature of other temperature areas is determined; when the temperature difference is greater than a preset temperature difference, the temperature-differentiated area is determined to be a target hot spot area, and a hot spot detection result is generated based on the target hot spot area; occlusion detection is performed on the fused image in parallel to determine a shadow area in the fused image; when the shadow area has a preset shadow feature, the shadow area is determined to be a target shadow area, and a foreign object occlusion detection result is generated based on the target shadow area; when any one of the hot spot detection result and the foreign object occlusion detection result is a fault feature result, a fault feature is determined based on the hot spot detection result and / or the foreign object occlusion detection result.

[0032] As a preferred solution of the defect recognition method based on drone inspection images described in the present invention, when the fault feature is a target fault feature, determining the position information of the fault feature in the multispectral image information, obtaining fault location information based on the position information and the three-dimensional spatial coordinate system, and outputting the fault location information, the method includes:

[0033] If the fault feature is a single type of fault, then the target single type of fault location information is obtained based on the position information of the fault feature in the multispectral image and the three-dimensional spatial coordinate system;

[0034] If the fault feature is a multi-type fault, obtain multi-type fault location information and perform deduplication processing to eliminate repeated information of hot spots and occlusion features at the same position to obtain target multi-type fault location information;

[0035] The obtained single-type fault location information and multi-type fault location information are output to perform fault maintenance for wind turbines and photovoltaic areas.

[0036] The beneficial effect of this preferred technical solution is that wind turbines and photovoltaic areas are modeled by drone formation, and inspection routes are planned based on the modeling results, and multi-spectral information collected on the inspection routes is obtained for fault identification and positioning, which can improve the inspection efficiency and fault identification accuracy of wind turbines and photovoltaic areas.

[0037] In a second aspect, the present invention provides a defect recognition system based on drone inspection images, comprising:

[0038] a three-dimensional modeling module, configured to generate a spatial model of the wind turbine and photovoltaic area based on the three-dimensional scanning data of the wind turbine and photovoltaic area by the drone group and the two-dimensional site base map of the wind turbine and photovoltaic area, and to establish a three-dimensional spatial coordinate system based on the spatial model;

[0039] An inspection module is configured to determine grouping information of the drone group, segment the spatial model of the wind turbine group and the photovoltaic area based on the grouping information to obtain a plurality of inspection zones, and determine an inspection path based on the inspection zones and the grouping information;

[0040] a fault identification module, configured to obtain multispectral image information collected based on the inspection path, perform fault identification on the multispectral image, and determine fault features in the multispectral image information;

[0041] A fault location module is used to determine the position information of the fault feature in the multispectral image information when the fault feature is a target fault feature, obtain fault location information based on the position information and the three-dimensional space coordinate system, and output the fault location information.

[0042] In a third aspect, the present invention provides an electronic device, comprising:

[0043] memory and processor;

[0044] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the defect recognition method based on drone inspection images are implemented.

[0045] In a fourth aspect, the present invention provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the steps of the method for defect recognition based on drone inspection images.

[0046] Compared with existing technologies, the present invention offers the following benefits: Compared with traditional inspection methods, it can improve inspection efficiency, reduce labor costs, and enhance the safety and accuracy of inspection work. Furthermore, the system supports real-time data transmission and analysis, helping operators to understand equipment status and develop maintenance plans in a timely manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. Those skilled in the art can also derive other drawings based on these drawings without inventive effort. Among them:

[0048] Figure 1 This is a flowchart of a method, system, device, and storage medium for defect recognition based on drone inspection images according to an embodiment of the present invention;

[0049] Figure 2 A schematic diagram of an internal defect fault of a method, system, device, and storage medium for defect recognition based on drone inspection images according to an embodiment of the present invention;

[0050] Figure 3 A schematic diagram of a first fixed facility obstruction fault according to a method, system, device, and storage medium for defect recognition based on drone inspection images according to an embodiment of the present invention;

[0051] Figure 4 A schematic diagram of a second fixed facility obstruction fault according to a method, system, device, and storage medium for identifying defects based on drone inspection images according to an embodiment of the present invention;

[0052] Figure 5 A schematic diagram of a surface wear fault according to an embodiment of the present invention, including a method, system, device, and storage medium for defect recognition based on drone inspection images;

[0053] Figure 6 This is a schematic diagram of a local corrosion fault based on a drone inspection image defect recognition method, system, equipment and storage medium according to an embodiment of the present invention. DETAILED DESCRIPTION

[0054] To make the above-mentioned objects, features, and advantages of the present invention more clearly understood, the following detailed description of the specific embodiments of the present invention is given in conjunction with the accompanying drawings. It is obvious that the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in this field without creative work should fall within the scope of protection of the present invention.

[0055] Example 1

[0056] Reference Figure 1 , is an embodiment of the present invention, which provides a method for identifying defects based on drone inspection images, comprising:

[0057] S100: generating a spatial model of the wind turbine and the photovoltaic area based on the three-dimensional scanning data of the wind turbine and the photovoltaic area by the drone group and the two-dimensional site base map of the wind turbine and the photovoltaic area, and establishing a three-dimensional spatial coordinate system based on the spatial model;

[0058] S200: Determine the grouping information of the drone group, segment the spatial model of the wind turbine group and the photovoltaic area based on the grouping information to obtain multiple inspection zones, and determine the inspection path based on the inspection zones and the grouping information;

[0059] S300: Acquire multispectral image information collected based on the inspection path, perform fault identification on the multispectral image, and determine the fault characteristics in the multispectral image information;

[0060] S400: When the fault feature is a target fault feature, determine the position information of the fault feature in the multispectral image information, obtain fault location information based on the position information and the three-dimensional space coordinate system, and output the fault location information.

[0061] It should be noted that current methods for inspecting wind turbines and photovoltaic panels for defects rely primarily on manual inspections. This approach is inefficient, costly, and difficult to cover large power plant areas. Manual inspections are not only time-consuming and labor-intensive, but also difficult to ensure accuracy and consistency due to human factors. Furthermore, manual inspections pose safety risks for installations in complex terrain or other inaccessible locations. Therefore, traditional manual inspection methods are not suitable for the safe and efficient inspection of wind turbines and photovoltaic panels.

[0062] The present application provides a solution that can replace manual inspections with drone groups, can perform three-dimensional modeling of wind turbines and photovoltaic areas, determine the layout structure of wind turbines and photovoltaic areas, and can group drones based on the layout structure of wind turbines and photovoltaic areas. It can also enable drones to inspect wind turbines and photovoltaic areas from multiple angles at the same time, plan appropriate inspection routes, quickly and accurately obtain multi-spectral images of wind turbines and photovoltaic areas, perform fault identification on multi-spectral images, and determine the type and location of the fault by combining visible light and infrared light images. Compared with manual inspections, this solution can not only improve inspection efficiency, but also discover potential faults, accurately obtain the fault location, and achieve precise positioning.

[0063] As can be seen from the above embodiments, the present application generates a spatial model of the wind turbine and photovoltaic area based on the three-dimensional scanning data of the wind turbine and photovoltaic area by the drone group and the two-dimensional site base map of the wind turbine and photovoltaic area, and establishes a three-dimensional spatial coordinate system based on the spatial model, determines the grouping information of the drone group, divides the spatial model of the wind turbine and photovoltaic area based on the grouping information to obtain multiple inspection partitions, and determines the inspection path based on the inspection partitions and grouping information, obtains multispectral image information collected based on the inspection path, identifies faults on the multispectral image, determines the fault features in the multispectral image information, and when the fault features are target fault features, determines the position information of the fault features in the multispectral image information, obtains fault location information based on the position information and the three-dimensional spatial coordinate system, and outputs the fault location information. The wind turbine and photovoltaic area can be modeled by drone grouping, and the inspection path can be planned based on the modeling results. The multispectral information collected on the inspection path is obtained for fault identification and location, which can improve the inspection efficiency and fault identification accuracy of the wind turbine and photovoltaic area.

[0064] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication, and program execution capabilities, such as a tablet computer, personal computer, or mobile phone, or an electronic device capable of performing the aforementioned functions, such as a wind turbine generator system and a photovoltaic zone intelligent inspection device. This embodiment and the following embodiments will be described below using wind turbine generator systems and photovoltaic zone intelligent inspection devices as examples.

[0065] In the embodiment of the present application, step S100 generates a spatial model of the wind turbine and the photovoltaic area based on the three-dimensional scanning data of the wind turbine and the photovoltaic area by the drone group and the two-dimensional site base map of the wind turbine and the photovoltaic area, and establishes a three-dimensional spatial coordinate system based on the spatial model, including:

[0066] According to the three-dimensional scanning data of the wind turbine and photovoltaic area by the drone group and the scanning formation information of the drone group, scanning marks are added to the three-dimensional scanning data;

[0067] Matching the two-dimensional site base map of the wind turbine and photovoltaic area with the three-dimensional scan data corresponding to the scan mark to obtain a matching result, and rewriting the scan mark according to the matching result to obtain a model fragment mark;

[0068] Arrange the model fragment marks in order, and generate a spatial model by combining the three-dimensional description data corresponding to the model fragment marks with the two-dimensional station base map;

[0069] A three-dimensional space coordinate system is established based on the space model.

[0070] It should be noted that the scanning mark is used to identify the correspondence between the three-dimensional scanning data and the drone. Each drone can mark the collected three-dimensional scanning data to facilitate matching with the two-dimensional station base map.

[0071] In an optional embodiment, step S100 generates a spatial model of the wind turbine and the photovoltaic area based on the three-dimensional scanning data of the wind turbine and the photovoltaic area by the drone group and the two-dimensional site base map of the wind turbine and the photovoltaic area, and establishes a three-dimensional spatial coordinate system based on the spatial model, including:

[0072] Unmanned aerial vehicle (UAV) fleets are deployed within wind turbines and photovoltaic (PV) zones for safety inspections. First, the UAV fleet performs a 3D scan of the wind turbines and PV zones, acquiring structural information about the wind turbines and PV zones, including their outlines and the orientation of the panels. The UAV fleet can freely scan the wind turbines and PV panels in all directions, generating comprehensive scan information. Each UAV acquires a certain amount of data, and its trajectory is stored. This allows the data from each UAV to be aggregated and fused, combining data from multiple drones to create a complete scan of the wind turbines and PV zones. Simultaneously, a 2D base map of the wind turbines and PV zones is acquired. A spatial model of the wind turbines and PV zones is generated by combining the complete scan information with the 2D base map. A 3D coordinate system is then constructed, with a pre-defined point in the spatial model serving as the origin.

[0073] It should be noted that a drone group generally refers to a collection of multiple drones. In this embodiment, it can be used to inspect wind turbines and photovoltaic areas. Because multiple drones work together, the collected data is redundant, ensuring data accuracy. Wind turbines and photovoltaic areas refer to power generation facilities that use solar panels to directly convert sunlight into electricity.

[0074] It should be understood that three-dimensional scanning data is a set of data obtained by three-dimensional scanning equipment carried by drones (such as LiDAR or photogrammetry systems). These data can describe the three-dimensional shape and structure of wind turbines and photovoltaic areas in detail. They usually include a large amount of point cloud data, and each point contains its X, Y, and Z coordinates in space. These data can accurately reflect the physical characteristics and spatial layout of wind turbines and photovoltaic areas. The two-dimensional site base map refers to the projection of wind turbines and photovoltaic areas on the horizontal plane, which is usually a plan view obtained through surveying or satellite imagery. This map contains the plan layout information of wind turbines and photovoltaic areas, such as the positional relationship of component arrangement, roads, buildings, etc. The three-dimensional spatial coordinate system is established based on the spatial model of wind turbines and photovoltaic areas. It can accurately describe each point in the spatial model and is established with three mutually perpendicular X-axis, Y-axis and Z-axis, each axis representing a dimension in space.

[0075] In an optional embodiment, to facilitate drone group management, the drones within a drone group can be organized into groups. This embodiment divides the drone group into two teams, with the best-performing drone in each team serving as the leader. Within the same team, all other drones can exchange data with the leader drone, and the leader drone can collect and process information collected by other drones in the same team. Therefore, to manage the drone formations, all drones can be numbered. For example, the first drone formation can be numbered from A00 to An, and the second drone formation can be numbered from B00 to Bm, where A and B distinguish the formation groups, 00 represents the leader drone, and m and n represent the number of drones in the formation. When the drone formation performs a 3D scanning task to model wind turbines and photovoltaic areas, the 3D scan data of the wind turbines and photovoltaic areas can be matched with the drone formation's scanned data, and scan tags can be added to the 3D scan data. The 2D site basemap of the wind turbine and photovoltaic area is then matched with the 3D scan data corresponding to the scanned markers to obtain a matching result. The scanned markers are then rewritten based on the matching result to obtain model fragment markers. The model fragment markers are then arranged in the order they appear in the 2D site basemap. The 3D description data corresponding to the model fragment markers is then combined with the 2D site basemap to generate a spatial model. Based on the spatial model, a 3D spatial coordinate system is established to describe each point in the spatial model.

[0076] In the embodiment of the present application, in step S200, the grouping information of the drone group is determined, and the spatial model of the wind turbine group and the photovoltaic area is segmented based on the grouping information to obtain multiple inspection zones, including:

[0077] Determine the grouping information of the drones, which includes group information and team member identity information;

[0078] Determine the number of members in each group based on the grouping information and member identity information;

[0079] Determine inspection division weights based on the number of crew members in each group;

[0080] The spatial model of wind turbines and photovoltaic areas is segmented according to the inspection division weights to obtain multiple inspection partitions that are consistent with the number of groups.

[0081] It's important to note that grouping information refers to the organization and allocation of drones during missions. This includes the number of drones and their respective mission roles. Inspection zoning involves dividing the spatial model of wind turbines and photovoltaic areas into multiple zones, with each zone serving as an independent inspection unit. This zoning is typically based on the geographic layout of wind turbines and photovoltaic areas, equipment distribution, risk level, or other relevant factors. The inspection route refers to the specific flight path that drones follow during inspection missions. This route is determined based on the inspection zoning and grouping information to ensure efficient and comprehensive coverage of all inspection areas.

[0082] In an optional embodiment, the grouping information of the drone group is parsed to determine the current drone group formation and the composition of each formation. After understanding the composition of the drone group, the spatial model of the wind turbine group and the photovoltaic area is divided according to the drone grouping to obtain multiple inspection zones, where each inspection zone is directly related to the number of drones. After the drone inspection zones are divided, the number of drone formations or the number of drones included in the inspection zone can be determined, and the inspection path can be determined in combination with the number of drones and the size of the inspection zone.

[0083] For example, the grouping information of drones is determined, where the grouping information includes group information and member identity information. The grouping information is the formation number of the current drone group, and the member identity information is the information of the drones that make up the drone formation. That is, multiple drone sets are obtained from the drones, where different drone sets include several drone members, and each drone can only exist in one drone group. In the drone grouping information, it can be stored in the format of: <group, group number, drone identity information>, for example<C,02,3DH87MKA98W1> Therefore, the number of members in each group can be determined based on the grouping information and the member identity information, and the inspection division weight can be obtained based on the proportion of each group to the total number of drones. The weight division formula is:

[0084]

[0085] Among them, w i Assign weights to inspections, N iis the number of drones in the i-th group, N is the total number of drones in the drone group, and m is the total number of groups.

[0086] When dividing the wind turbine and photovoltaic area spatial models according to the inspection division weights, the weights of each drone group can be used for partitioning, which can be expressed as: S i =w i S, where S is the total area of the spatial model, S i The inspection zone for the i-th drone group is the same as the number of inspection zones. Therefore, the inspection zones can be dynamically changed as the number of drones in the group changes. It is worth noting that when dividing the inspection zones, the integrity of the inspection zones is maintained as much as possible, so that drone groups with higher weights are prioritized.

[0087] In the embodiment of the present application, determining the inspection path based on the inspection partition and grouping information in step S200 includes:

[0088] Determine the number of group members according to the grouping information, and divide the group members into a first sub-group group and a second sub-group group, wherein the difference in the number of group members between the first sub-group group and the second sub-group group is not greater than one;

[0089] Divide the inspection area into an inspection matrix with an even number of rows and columns, determine the first inspection starting point and the second inspection starting point of the inspection matrix, the first inspection starting point corresponds to the first team grouping, and the second inspection starting point corresponds to the second team grouping;

[0090] A first inspection path is generated based on a first inspection starting point and a first inspection direction, and a second inspection path is generated based on a second inspection starting point and a second inspection direction;

[0091] Sequentially combine the multiple first inspection paths to obtain a first target inspection path;

[0092] The plurality of second inspection paths are sequentially combined to obtain a second target inspection path.

[0093] It should be noted that the first squad grouping and the second squad grouping are detailed groupings within the same grouping, that is, the drones of the original one grouping are divided into multiple squad groups according to certain rules. When allocating, the average principle can be followed so that the number of drones in the first squad grouping and the second squad grouping is the same, or the difference is 1. For example, when the number of drone groups is 9, the number of drones in the first squad grouping and the second squad grouping is 4 and 5 respectively. If the number of drone groups is 10, the number of drones in the first squad grouping and the second squad grouping is 5. At the same time, the inspection path of the first squad grouping is the first inspection path, and the inspection path of the second squad grouping is the second inspection path. The first inspection path and the second inspection path are different inspection paths.

[0094] In an optional embodiment, determining an inspection path based on inspection zones and grouping information can first determine the number of group members based on the grouping information and divide the group members into a first sub-team group and a second sub-team group. The inspection zone is then divided into an inspection matrix with an even number of rows and columns. The first inspection starting point and the second inspection starting point of the inspection matrix are determined, with the first inspection starting point corresponding to the first sub-team group and the second inspection starting point corresponding to the second sub-team group. A first inspection path is generated based on the first inspection starting point and the first inspection direction, and a second inspection path is generated based on the second inspection starting point and the second inspection direction. The drone inspection area is divided into an 8×8 inspection matrix, with the red and blue areas representing the first and second inspection starting points, respectively, and the green areas representing the inspection endpoints of the first and second inspection paths. Since each sub-team group includes multiple drones, the inspection paths of each drone in each sub-team group are sequentially combined to obtain a target inspection path. Therefore, multiple first inspection paths can be sequentially combined to obtain a first target inspection path, and multiple second inspection paths can be sequentially combined to obtain a second target inspection path.

[0095] In the embodiment of the present application, step S300 acquires multispectral image information collected based on the inspection path, performs fault identification on the multispectral image, and determines fault features in the multispectral image information, including:

[0096] Obtain visible light image information and infrared image information collected by the drone group based on the inspection path; fuse the visible light image and the infrared image to obtain a multispectral image;

[0097] Perform feature recognition on multispectral images to determine the physical object features and temperature features in the spectral images; perform feature fusion on the physical object features and temperature features to obtain a fused image of the photovoltaic station; perform hot spot detection and occlusion detection on the fused image to obtain hot spot detection results and foreign object occlusion results, and determine the fault features based on the hot spot detection results and foreign object occlusion detection results.

[0098] In the embodiment of the present application, step S300 performs hot spot detection and occlusion detection on the fused image to obtain hot spot detection results and foreign object occlusion detection results, and determines the fault characteristics based on the hot spot detection results and foreign object occlusion detection results, including:

[0099] Perform hot spot detection on the fused image to determine the temperature divergence area in the fused image, and determine the temperature difference between the temperature divergence area and the maximum temperature of other temperature areas; when the temperature difference is greater than the preset temperature difference, determine the temperature divergence area as the target hot spot area, and generate the hot spot detection result based on the target hot spot area; perform occlusion detection on the fused image in parallel to determine the shadow area in the fused image; when the shadow area has a preset shadow feature, determine the shadow area as the target shadow area, and generate the foreign object occlusion detection result based on the target shadow area; when any of the hot spot detection result and the foreign object occlusion detection result is a fault feature result, determine the fault feature based on the hot spot detection result and / or the foreign object occlusion detection result.

[0100] In one optional embodiment, a drone flies along a pre-set inspection route, collecting visible light and infrared images of wind turbines and photovoltaic areas. These images are then fused to generate a multispectral image, demonstrating the multispectral image fusion process. During the fusion process, specific fusion weights are used to combine the visible light and infrared images to produce the fused multispectral image.

[0101] It should be noted that in this embodiment, the fusion weight is used to balance the contributions of the visible light image and the infrared image to the final image.

[0102] Furthermore, feature recognition is performed on the fused multispectral image to determine the physical and temperature characteristics of the image. Specifically, the physical image of the photovoltaic panel is fused with the infrared image features to generate a temperature distribution feature image of the photovoltaic panel. Subsequently, hot spot detection and occlusion detection are performed on this fused image to obtain hot spot detection results and foreign object occlusion results, respectively.

[0103] In an optional embodiment, hot spot detection relies primarily on a temperature curve generated from an infrared image, identifying areas with sudden temperature changes as hot spots. Occlusion detection, on the other hand, is achieved by detecting grayscale values in the image, comparing the grayscale values of adjacent areas. Areas with higher grayscale values are considered occluded.

[0104] Furthermore, the fault characteristics are determined based on the hot spot detection results and the foreign object occlusion detection results. The specific steps include: First, hot spot detection is performed on the fused image to identify the temperature variation area and calculate the maximum temperature difference between this area and other temperature areas. When the temperature difference exceeds a preset value, the temperature variation area is considered a target hot spot area, and a hot spot detection result is generated based on this. Simultaneously, occlusion detection is performed in parallel to identify the shadow area in the fused image. If the shadow area meets the preset shadow characteristics, it is considered a target shadow area, and a foreign object occlusion detection result is generated based on this.

[0105] Once fault characteristics are identified in the hot spot detection or foreign object occlusion detection results, these results are used to determine the fault characteristics. After determining the fault characteristics, their location information within the multispectral image information is further determined and combined with the three-dimensional spatial coordinate system to obtain fault location information. The target fault characteristics here refer to faults such as localized hot spots or shadows on the photovoltaic panel. Location information refers to the specific coordinates of these faults within the image. Fault location information is a combination of fault type and location information.

[0106] In another optional embodiment, the temperature difference area in the fused image is determined by analyzing the infrared image, according to the formula: a ={(x,y)|T(x,y)>T t};

[0107] Among them, T a is the set of temperature dissimilar regions, T(x,y) is the temperature value of point (x,y), T t is the preset temperature threshold.

[0108] Determine the temperature difference between the highest temperatures in the temperature-dissimilar area and other temperature areas. When the temperature difference is greater than the preset temperature difference, determine the temperature-dissimilar area as the target hot spot area. Generate a hot spot detection result based on the target hot spot area. The temperature difference is the maximum temperature difference, that is, the difference between the highest temperature in the dissimilar area and the lowest temperature in the remaining areas, expressed as:

[0109] ΔT max =max(T a )-min(T other )

[0110] Where, ΔT max is the maximum temperature difference, max(T a ) is the highest temperature in the temperature alienation area, min(T other ) is the lowest temperature in other areas.

[0111] When determining the hot spot area, the formula can be used:

[0112]

[0113] Among them, H is the target hot spot area set, T p is the preset temperature threshold.

[0114] Similarly, when determining the occlusion area, according to the formula:

[0115]

[0116] Among them, S t is the target shadow area, S pis the preset shadow area threshold.

[0117] When any one of the hot spot detection result and the foreign object occlusion detection result is a fault characteristic result, the fault characteristic is determined according to the hot spot detection result and / or the foreign object occlusion detection result.

[0118] In the embodiment of the present application, in step S400, when the fault feature is a target fault feature, determining the position information of the fault feature in the multispectral image information, obtaining fault location information based on the position information and the three-dimensional spatial coordinate system, and outputting the fault location information include:

[0119] If the fault feature is a single type of fault, the target single type fault location information is obtained based on the position information of the fault feature in the multispectral image and the three-dimensional spatial coordinate system;

[0120] If the fault feature is a multi-type fault, obtain multi-type fault location information and perform deduplication processing to eliminate the repeated information of hot spots and occlusion features at the same location to obtain the target multi-type fault location information;

[0121] The obtained single-type fault location information and multi-type fault location information are output to perform fault maintenance for wind turbines and photovoltaic areas.

[0122] It should be noted that in terms of fault type, a distinction is made between single hot spot faults, occlusion faults, and multi-type faults that include both. For single-type faults, the target single-type fault location information is obtained based on the position information of the fault feature in the multispectral image, combined with the three-dimensional spatial coordinate system. For multi-type faults, multi-type fault location information is first obtained, and then deduplication processing is performed on it to eliminate the repeated information of hot spot and occlusion features at the same location to obtain the target multi-type fault location information. Finally, the obtained single-type fault location information and multi-type fault location information are output to provide precise guidance for the maintenance of wind turbines and photovoltaic areas.

[0123] Embodiment 2 is an embodiment of the present invention. This embodiment differs from the first embodiment in that it provides a defect recognition system based on drone inspection images, including:

[0124] A 3D modeling module is used to generate a spatial model of the wind turbine and photovoltaic area based on the 3D scanning data of the wind turbine and photovoltaic area by the drone group and the 2D site base map of the wind turbine and photovoltaic area, and to establish a 3D spatial coordinate system based on the spatial model;

[0125] The inspection module is used to determine the grouping information of the UAV group, segment the spatial model of the wind turbine group and photovoltaic area based on the grouping information to obtain multiple inspection zones, and determine the inspection path based on the inspection zones and grouping information;

[0126] A fault identification module is used to obtain multispectral image information collected based on the inspection path, perform fault identification on the multispectral image, and determine the fault characteristics in the multispectral image information;

[0127] The fault location module is used to determine the position information of the fault feature in the multispectral image information when the fault feature is a target fault feature, obtain fault location information based on the position information and the three-dimensional space coordinate system, and output the fault location information.

[0128] Specifically, when executing, each module of the drone inspection image defect recognition system of this embodiment implements the steps of the drone inspection image defect recognition method in Example 1, for example:

[0129] In one embodiment, the defect recognition system based on drone inspection images can perform the following steps:

[0130] According to the three-dimensional scanning data of the wind turbine and photovoltaic area by the drone group and the scanning formation information of the drone group, scanning marks are added to the three-dimensional scanning data;

[0131] Matching the two-dimensional site base map of the wind turbine and photovoltaic area with the three-dimensional scan data corresponding to the scan mark to obtain a matching result, and rewriting the scan mark according to the matching result to obtain a model fragment mark;

[0132] Arrange the model fragment marks in order, and generate a spatial model by combining the three-dimensional description data corresponding to the model fragment marks with the two-dimensional station base map;

[0133] A three-dimensional space coordinate system is established based on the space model.

[0134] Determine the grouping information of the drones, which includes group information and team member identity information;

[0135] Determine the number of members in each group based on the grouping information and member identity information;

[0136] Determine inspection division weights based on the number of crew members in each group;

[0137] The spatial model of wind turbines and photovoltaic areas is segmented according to the inspection division weights to obtain multiple inspection partitions that are consistent with the number of groups.

[0138] Determine the number of group members according to the grouping information, and divide the group members into a first sub-group group and a second sub-group group, wherein the difference in the number of group members between the first sub-group group and the second sub-group group is not greater than one;

[0139] Divide the inspection area into an inspection matrix with an even number of rows and columns, determine the first inspection starting point and the second inspection starting point of the inspection matrix, the first inspection starting point corresponds to the first team grouping, and the second inspection starting point corresponds to the second team grouping;

[0140] A first inspection path is generated based on a first inspection starting point and a first inspection direction, and a second inspection path is generated based on a second inspection starting point and a second inspection direction;

[0141] Sequentially combine the multiple first inspection paths to obtain a first target inspection path;

[0142] The plurality of second inspection paths are sequentially combined to obtain a second target inspection path.

[0143] Obtain visible light image information and infrared image information collected by the drone group based on the inspection path; fuse the visible light image and the infrared image to obtain a multispectral image;

[0144] Perform feature recognition on multispectral images to determine the physical object features and temperature features in the spectral images; perform feature fusion on the physical object features and temperature features to obtain a fused image of the photovoltaic station; perform hot spot detection and occlusion detection on the fused image to obtain hot spot detection results and foreign object occlusion results, and determine the fault features based on the hot spot detection results and foreign object occlusion detection results.

[0145] Perform hot spot detection on the fused image to determine the temperature divergence area in the fused image, and determine the temperature difference between the temperature divergence area and the maximum temperature of other temperature areas; when the temperature difference is greater than the preset temperature difference, determine the temperature divergence area as the target hot spot area, and generate the hot spot detection result based on the target hot spot area; perform occlusion detection on the fused image in parallel to determine the shadow area in the fused image; when the shadow area has a preset shadow feature, determine the shadow area as the target shadow area, and generate the foreign object occlusion detection result based on the target shadow area; when any of the hot spot detection result and the foreign object occlusion detection result is a fault feature result, determine the fault feature based on the hot spot detection result and / or the foreign object occlusion detection result.

[0146] If the fault feature is a single type of fault, the target single type fault location information is obtained based on the position information of the fault feature in the multispectral image and the three-dimensional spatial coordinate system;

[0147] If the fault feature is a multi-type fault, obtain multi-type fault location information and perform deduplication processing to eliminate the repeated information of hot spots and occlusion features at the same location to obtain the target multi-type fault location information;

[0148] The obtained single-type fault location information and multi-type fault location information are output to perform fault maintenance for wind turbines and photovoltaic areas.

[0149] This embodiment further provides an electronic device applicable to a method for identifying defects in drone inspection images, including:

[0150] Memory and processor; the memory is used to store computer-executable instructions, and the processor is used to execute computer-executable instructions to implement the defect recognition method based on drone inspection images as proposed in the above embodiment.

[0151] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, the method for defect recognition based on drone inspection images proposed in the above embodiment is implemented.

[0152] The storage medium proposed in this embodiment and the method for implementing defect recognition based on drone inspection images proposed in the above embodiment belong to the same inventive concept. Technical details not fully described in this embodiment can be found in the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

[0153] Through the above description of the implementation methods, those skilled in the art can clearly understand that the present invention can be implemented with the help of software and necessary general-purpose hardware, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as a computer's floppy disk, read-only memory (ROM), random access memory (RAM), flash memory (FLASH), hard disk or optical disk, etc., including a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods of various embodiments of the present invention.

[0154] Example 3, reference Figures 2 to 6 , is an embodiment of the present invention, which uses inspection tests to identify image defects in wind turbines and photovoltaic areas to verify the beneficial effects of the present invention.

[0155] In this embodiment, drones equipped with high-definition cameras and infrared thermal imagers conduct comprehensive, comprehensive inspections of key wind turbine components, including blades, towers, and hubs. The drones fly autonomously in the complex and ever-changing wind farm environment, ensuring smooth inspections. Regardless of weather conditions, the drones can reliably perform their tasks, unaffected by inclement weather. During inspections, the drones' high-resolution cameras continuously record image data from the wind turbines and transmit this data in real time to a ground station or cloud server for analysis and processing.

[0156] Reference Figures 2 to 4 ,in Figure 2 The fault type detected by the drone inspection image is an internal defect, indicating that the component has internal defects such as broken grids and hidden cracks, forming hot spots. The fault number locked by the drone is NO.117, the location is: 3,6,1, GPS (East longitude, North latitude): 117.07458996,42.22204517.

[0157] in Figure 3 The fault type detected by the drone inspection image is fixed facility obstruction, indicating that the component surface is obscured by the shadow of the fixed object, resulting in a hot spot. The fault number locked by the drone is NO.118, location: 3,16,5, GPS (East longitude, North latitude): 117.07389727,42.22285599.

[0158] in Figure 4 The fault type detected by the drone inspection image is fixed facility obstruction, indicating that the component surface is blocked by the shadow of the fixed object, resulting in a hot spot. The fault number locked by the drone is NO.119, location: 3,16,5, GPS (East longitude, North latitude): 117.07389727,42.22285599.

[0159] Combine Figure 5 and Figure 6 It should be noted that this defect is usually difficult to be detected by the naked eye in the early stages, but through the method of the present invention, the inspection cycle is shortened from once a month to once a week, the defect detection rate is expected to increase by 30%, and the maintenance cost is reduced by 20%. It can be discovered and warned in time, which improves the efficiency and accuracy of the inspection work, ensures that measures can be taken at the early stage of the problem, effectively prevents the expansion of small problems, and ensures the stable operation of the wind turbine.

[0160] The present invention further improves inspection efficiency, reduces labor costs, and enhances the safety and accuracy of inspection work. Compared with traditional inspection methods, drone inspections do not require personnel to personally climb to high altitudes to perform operations, thereby reducing safety risks. The drone inspection method of the present invention can operate efficiently both during the day and at night, ensuring that the wind turbine generator set is always in the best working condition. In addition, the present invention also has the ability to transmit and analyze data in real time. Inspection data is quickly uploaded to the cloud server for storage and processing. Operation and maintenance personnel can view inspection results and analysis reports at any time through various terminal devices, and promptly understand the operating status and maintenance needs of the wind turbine generator set. It also supports multi-dimensional data analysis and visual display to help operation and maintenance personnel more intuitively grasp the health status of the equipment and formulate scientific maintenance plans.

[0161] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for defect recognition based on drone inspection images, characterized in that: include: Generate a spatial model of the wind turbine group and the photovoltaic area based on the three-dimensional scanning data of the wind turbine group and the photovoltaic area by the drone group and the two-dimensional site base map of the wind turbine group and the photovoltaic area, and establish a three-dimensional spatial coordinate system based on the spatial model; Determining grouping information of the drone group, segmenting the spatial model of the wind turbine group and the photovoltaic area based on the grouping information to obtain a plurality of inspection zones, and determining an inspection path based on the inspection zones and the grouping information; Acquire multispectral image information collected based on the inspection path, perform fault identification on the multispectral image, and determine fault features in the multispectral image information; When the fault feature is a target fault feature, position information of the fault feature in the multispectral image information is determined, fault location information is obtained based on the position information and the three-dimensional space coordinate system, and the fault location information is output.

2. The defect recognition method based on drone inspection images according to claim 1 is characterized in that: Generating a spatial model of the wind turbine generator set and the photovoltaic area based on the three-dimensional scanning data of the wind turbine generator set and the photovoltaic area by the drone group and the two-dimensional site base map of the wind turbine generator set and the photovoltaic area, and establishing a three-dimensional spatial coordinate system based on the spatial model, includes: Matching the three-dimensional scanning data of the wind turbine group and the photovoltaic area by the drone group with the scanning formation information of the drone group, and adding a scanning mark to the three-dimensional scanning data; Matching the two-dimensional site base map of the wind turbine and photovoltaic area with the three-dimensional scan data corresponding to the scan mark to obtain a matching result, and rewriting the scan mark according to the matching result to obtain a model fragment mark; Arranging the model fragment marks in order, and generating a spatial model by combining the three-dimensional description data corresponding to the model fragment marks and the two-dimensional station base map; A three-dimensional space coordinate system is established based on the space model.

3. The defect recognition method based on drone inspection images according to claim 2 is characterized in that: Determine the grouping information of the drone group, and segment the spatial model of the wind turbine group and the photovoltaic area based on the grouping information to obtain multiple inspection zones, including: Determining grouping information of the drone, the grouping information including group information and group member identity information; Determine the number of members in each group according to the grouping information and the member identity information; Determining inspection division weights based on the number of members in each group; The spatial model of the wind turbine generator set and the photovoltaic area is divided according to the inspection division weights to obtain a plurality of inspection partitions that are consistent with the number of the groups.

4. The defect recognition method based on drone inspection images according to claim 3 is characterized in that: Determining an inspection path based on the inspection partition and the grouping information includes: Determining the number of group members according to the grouping information, and dividing the group members into a first sub-group and a second sub-group, wherein the difference in the number of group members between the first sub-group and the second sub-group is no more than one; Divide the inspection zone into an inspection matrix with an even number of rows and columns, and determine a first inspection starting point and a second inspection starting point of the inspection matrix, wherein the first inspection starting point corresponds to the first team grouping, and the second inspection starting point corresponds to the second team grouping; Generating a first inspection path with the first inspection starting point and the first inspection direction, and generating a second inspection path with the second inspection starting point and the second inspection direction; sequentially combining a plurality of the first inspection paths to obtain a first target inspection path; A plurality of the second inspection paths are sequentially combined to obtain a second target inspection path.

5. The defect recognition method based on drone inspection images according to claim 4 is characterized in that: Acquiring multispectral image information collected based on the inspection path, performing fault identification on the multispectral image, and determining fault features in the multispectral image information, including: Obtaining visible light image information and infrared image information collected by the drone group based on the inspection path; fusing the visible light image with the infrared image to obtain a multispectral image; Perform feature recognition on the multispectral image to determine the physical object features and temperature features in the spectral image; perform feature fusion on the physical object features and the temperature features to obtain a fused image of the photovoltaic station; perform hot spot detection and occlusion detection on the fused image to obtain hot spot detection results and foreign object occlusion results, and determine fault features based on the hot spot detection results and foreign object occlusion detection results.

6. The defect recognition method based on drone inspection images according to claim 5 is characterized in that: The performing hot spot detection and occlusion detection on the fused image to obtain hot spot detection results and foreign object occlusion detection results, and determining fault characteristics according to the hot spot detection results and foreign object occlusion detection results, includes: Hot spot detection is performed on the fused image to determine a temperature-differentiated area in the fused image, and a temperature difference between the temperature-differentiated area and the maximum temperature of other temperature areas is determined; when the temperature difference is greater than a preset temperature difference, the temperature-differentiated area is determined to be a target hot spot area, and a hot spot detection result is generated based on the target hot spot area; occlusion detection is performed on the fused image in parallel to determine a shadow area in the fused image; when the shadow area has a preset shadow feature, the shadow area is determined to be a target shadow area, and a foreign object occlusion detection result is generated based on the target shadow area; when any one of the hot spot detection result and the foreign object occlusion detection result is a fault feature result, a fault feature is determined based on the hot spot detection result and / or the foreign object occlusion detection result.

7. The defect recognition method based on drone inspection images according to claim 6 is characterized in that: When the fault feature is a target fault feature, determining position information of the fault feature in the multispectral image information, obtaining fault location information based on the position information and the three-dimensional spatial coordinate system, and outputting the fault location information includes: If the fault feature is a single type of fault, then the target single type of fault location information is obtained based on the position information of the fault feature in the multispectral image and the three-dimensional spatial coordinate system; If the fault feature is a multi-type fault, obtain multi-type fault location information and perform deduplication processing to eliminate repeated information of hot spots and occlusion features at the same position to obtain target multi-type fault location information; The obtained single-type fault location information and multi-type fault location information are output to perform fault maintenance for wind turbines and photovoltaic areas.

8. A defect recognition system based on drone inspection images, characterized in that: include: a three-dimensional modeling module, configured to generate a spatial model of the wind turbine and photovoltaic area based on the three-dimensional scanning data of the wind turbine and photovoltaic area by the drone group and the two-dimensional site base map of the wind turbine and photovoltaic area, and to establish a three-dimensional spatial coordinate system based on the spatial model; An inspection module is configured to determine grouping information of the drone group, segment the spatial model of the wind turbine group and the photovoltaic area based on the grouping information to obtain a plurality of inspection zones, and determine an inspection path based on the inspection zones and the grouping information; a fault identification module, configured to obtain multispectral image information collected based on the inspection path, perform fault identification on the multispectral image, and determine fault features in the multispectral image information; A fault location module is used to determine the position information of the fault feature in the multispectral image information when the fault feature is a target fault feature, obtain fault location information based on the position information and the three-dimensional space coordinate system, and output the fault location information.

9. An electronic device comprising: memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the method for defect recognition based on drone inspection images as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the steps of the method for defect recognition based on drone inspection images as described in any one of claims 1 to 7.

Images