Photovoltaic module defect positioning navigation method and system

Abstract

The invention discloses a photovoltaic module defect positioning navigation method and system, and belongs to the technical field of photovoltaic power generation operation and maintenance. The method comprises the following steps: acquiring synchronously acquired panoramic electronic map data, image data and radar point cloud data of a station, and constructing a digital holographic model of the station; identifying a defect area of the photovoltaic module in the field station in the image data by using a pre-constructed defect identification model; acquiring the actual position of the defect area in the image data according to the space coordinates and the shooting parameters of the image acquisition equipment and the height data recorded by the radar point cloud data; identifying a passable area in the station according to the radar point cloud data, and according to the actual positions of the passable area and the defect area, planning by taking the shortest path as a target to obtain an operation and maintenance navigation path. According to the method, an optimized inspection route and a defect positioning result with relatively high precision can be provided, efficient planning of a maintenance path can be performed after defects are found, a closed loop of inspection scheduling and defect elimination work order generation is realized, and the inspection and defect elimination efficiency is improved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of photovoltaic power generation operation and maintenance, and particularly relates to a photovoltaic module defect positioning and navigation method and system. BACKGROUND

[0002] With the continuous development and wide application of photovoltaic power generation technology, the scale of photovoltaic power stations is gradually expanding, and the traditional manual inspection method has exposed a series of problems such as low efficiency, high missed detection rate, and inaccurate positioning, which has affected the operation and management efficiency of photovoltaic power stations. In the prior art, although unmanned aerial vehicle inspection is widely used in the inspection task of power stations, its main limitation is that it can only collect visible light and infrared images, lacks linkage with fault positioning, and is difficult to accurately identify and locate defects on photovoltaic modules. As for defect identification, although the traditional YOLO algorithm can perform target detection, its detection accuracy for small targets is low, especially for the identification of small faults such as hot spots.

[0003] In addition, in terms of positioning and navigation technology, the existing GPS positioning system has a large error, and can usually only achieve meter-level accuracy, which cannot meet the accurate positioning requirements of string level in photovoltaic power stations in actual application, resulting in low accuracy of positioning information and affecting the accuracy of fault positioning. In terms of operation and maintenance management, the current defect elimination process has not yet formed a closed loop, and the results of the maintenance process are difficult to automatically verify, which makes it difficult to timely confirm and feedback the state of the equipment after maintenance, and affects the overall operation and maintenance efficiency of the power station.

[0004] In summary, the defect detection efficiency of photovoltaic power stations is low, the positioning is not accurate, the maintenance path planning is missing, and the operation and maintenance closed-loop management is insufficient, which seriously restricts the intelligent operation and maintenance level of photovoltaic power stations. SUMMARY

[0005] The purpose of the present application is to provide a photovoltaic module defect positioning and navigation method and system, which can provide an optimized inspection route and a high-precision defect positioning result, and can efficiently plan a maintenance path after discovering a defect, realize a closed loop of inspection scheduling and defect elimination work order generation, and improve the inspection and defect elimination efficiency. The technical solution adopted by the present application is as follows.

[0006] In one aspect, the present application provides a photovoltaic module defect positioning and navigation method, comprising:

[0007] acquiring synchronously collected field station panoramic electronic map data, image data, and radar point cloud data, and constructing a field station digital holographic model;

[0008] based on the image data, identifying the defect area of the photovoltaic module in the field station by using a pre-constructed defect identification model;

[0009] According to the spatial coordinates and shooting parameters of the image acquisition device and the height data recorded by the radar point cloud data, the actual position of the defect area in the image data is obtained;

[0010] Based on the digital holographic model of the site, the passable area in the site is identified according to the radar point cloud data, and the operation and maintenance navigation path is planned according to the shortest path target based on the actual position of the passable area and the defect area.

[0011] Optionally, the construction of the digital holographic model of the site comprises: generating a site real scene three-dimensional model according to the synchronously collected site panoramic electronic map data and image data, generating and updating a site obstacle map in real time according to the radar point cloud data, and superimposing the site obstacle map and a string topological relationship diagram of the photovoltaic module to the site real scene three-dimensional model. The radar in the application comprises a laser radar and a millimeter wave radar, which are complementary to each other and can ensure the reliability of the inspection data.

[0012] Optionally, the image data comprises orthographic image data and infrared thermal imaging data collected by a unmanned aerial vehicle; and the defect area of the photovoltaic module comprises a hot spot area, a module missing area, a diode fault area, an abnormally low temperature area, a dirty area and a shelter area.

[0013] The pre-constructed defect recognition model comprises a multi-source data alignment module, a double-branch feature extraction network module, an SPPFAST module, a feature pyramid network module, a decoupling detection head, a Gaussian Wasserstein distance loss function module and a multi-modal weighted voting fusion module; wherein:

[0014] The multi-source data alignment module is used for receiving visible light image and infrared image data in multi-image data, realizing data alignment through a SIFT feature matching method, and synchronously transmitting the aligned visible light image and infrared image data to two CSPNet-Darknet53 network branches of a double-branch feature extraction network module to extract feature data of the visible light image and the infrared image respectively; the SPPFAST module is used for expanding the feature receptive field of the visible light image and the infrared image to optimize the feature data of the visible light image and the infrared image; the feature pyramid network module is used for performing feature extraction based on the optimized feature data of the visible light image and the infrared image to obtain multi-scale features corresponding to the visible light image and the infrared image respectively; the decoupled detection head is used for calculating defect category detection results and defect position detection results corresponding to different feature scales based on the multi-scale features; the Gaussian Wasserstein distance loss function module is used for calculating a loss function based on the defect detection results output by the decoupled detection head and sample true value labels in a defect recognition model training stage to optimize network parameters of other modules in the defect recognition model; and the multi-modal weighted voting fusion module is used for performing weighted fusion based on the defect category detection results and the defect position detection results corresponding to different feature scales to obtain final defect category and position detection results.

[0015] In the above scheme, the application adopts a CSPNet-Darknet53 structure as a backbone network, increases an SPPFAST module to improve a receptive field, constructs feature extraction and detection branches for visible light images and infrared images respectively, and fuses the detection results of the two branches through weighted fusion, which can significantly improve the detection accuracy of the final defects. In addition, the SIFT feature matching is used to align the pixel level of the visible light and infrared images, which can significantly reduce the time synchronization error; the detection head adopts a Decoupled Head, which separates the classification and regression branches; at the same time, a local attention mechanism can be introduced at the 4th level of the feature pyramid to improve the recall rate of pixel defects; and the Gaussian Wasserstein distance is used instead of the IoU to solve the problem of insensitivity to small target frame overlap calculation.

[0016] The optimization of the above defect detection model can better cope with the situation of “small targets, complementary shape and thermal characteristics, and complex background” in the photovoltaic module defect detection task.

[0017] Optionally, the Gaussian Wasserstein distance loss function module adopted in the defect recognition model training process uses a loss function represented as follows:

[0018]

[0019] wherein, represents a total loss function value. represents the Gaussian distance loss between the predicted defect bounding box and the real defect bounding box; represents the L1 loss between the predicted defect bounding box and the real defect bounding box , that is, the sum of absolute differences in each dimension of the center coordinates, width and height of the predicted defect bounding box and the real defect bounding box; and are weight parameters of the Gaussian distance loss and the L1 loss, respectively;

[0020] the Gaussian distance loss is represented as:

[0021] wherein, is a hyperparameter for adjusting the sensitivity of the loss function, is the distance between the Gaussian distribution corresponding to the predicted defect bounding box and the real defect bounding box, is the square of the L2 norm of

[0022] By optimizing the model parameters by integrating the L1 loss and the Gaussian distance loss, the characteristics that the L1 loss is more sensitive to small changes in the bounding box parameters and the advantages of the Gaussian distance loss in robustness and sample imbalance are effectively utilized, which can improve the precision, robustness and generalization ability of the model in the target detection task.

[0023] Optionally, the actual position of the defect area in the image data is obtained according to the spatial coordinates and the shooting parameters of the image acquisition device and the height data recorded by the radar point cloud data, and the actual position of the defect area in the image data comprises:

[0024] calculating the relative height of the position of the defect area; and taking the horizontal coordinate of the image acquisition device when shooting as the image center position coordinate and defining it as a reference point, and calculating the horizontal direction pixel deviation of the defect area relative to the reference point;

[0025] According to the relative height and the focal length parameter, the horizontal direction pixel deviation is converted into a position offset in the physical coordinate system to obtain the position coordinates of the defect area in the image data mapped to the panoramic electronic map.

[0026] Optionally, assuming that the spatial coordinates of the image acquisition device when shooting are , the pixel focal length in the shooting parameter is , the camera yaw angle is , the image center point pixel is , the defect area center point pixel is , and the height of the photographed defect position recorded by the radar point cloud data is , then the horizontal coordinate of the image center position is ;

[0027] The relative height of the location of the defect area is calculated , the formula is: ;

[0028] The horizontal pixel deviation of the defect area relative to the reference point is: ;

[0029] The horizontal pixel deviation is converted into the position offset in the physical coordinate system, and the formula is: ;

[0030] The calculation formula of the horizontal coordinate of the defect area mapped into the panoramic electronic map is:

[0031] ,

[0032] The actual position of the defect area in the image data is .

[0033] Optionally, the method further comprises: segmenting and identifying each photovoltaic module string in the image data, and assigning a unique ID and corresponding string GPS coordinates to each string according to the radar point cloud data, GPS data in the panoramic electronic map data, and GPS coordinates and shooting parameters when the unmanned aerial vehicle shoots the image.

[0034] The method for obtaining the actual position of the defect area in the image data further comprises: associating the defect area to the corresponding string ID and string GPS coordinates according to the position coordinates of the defect area mapped into the panoramic electronic map.

[0035] The above technical solution accurately calculates the image coordinates, radar point cloud elevation data, and device spatial pose, directly maps the pixel-level defects in the image to the geographic coordinate system, and the positioning accuracy can reach centimeter level, completely solving the problem of only seeing defects and being unable to accurately locate. By assigning a unique ID and GPS coordinates to each photovoltaic string, the system can automatically associate the identified defects to the specific string ID and position, so that the operation and maintenance personnel can obtain accurate data, and the issuance of operation and maintenance instructions has strong directivity.

[0036] Optionally, assigning a unique ID and corresponding string GPS coordinates to each string comprises:

[0037] The latitude and longitude coordinates of the string center point or key pixel point in the forward shooting image of the unmanned aerial vehicle are taken as the latitude and longitude coordinates of the string, and the coordinate offset of the GPS coordinates relative to the image center point is calculated, and the formula is:

[0038]

[0039] In the formula, a pixel offset of the center point or key pixel point of the group string relative to a center of the orthographic image, a height when the UAV takes the image, a pixel focal length, a resolution of the image;

[0040] The longitude and latitude coordinates of the group string are calculated by the following formula:

[0041]

[0042] wherein, and respectively represent longitude and latitude coordinates, and respectively are inverse projection function and projection function of Universal Transverse Mercator, which are used for conversion between geographical coordinates, i.e. longitude and latitude coordinates, and plane coordinates, i.e. UTM coordinates; a longitude and latitude coordinate when the UAV takes the image.

[0043] In a second aspect, the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the photovoltaic module defect positioning and navigation method according to the first aspect.

[0044] In a third aspect, the present application provides a photovoltaic module defect positioning and navigation system, which comprises a scheduling platform and a UAV inspection subsystem, and the UAV inspection subsystem comprises a UAV group carrying a camera device, a radar device and a positioning device;

[0045] The UAV inspection subsystem is configured to receive an inspection instruction from the scheduling platform, generate an inspection path according to the inspection instruction or follow an inspection path specified by the inspection instruction, inspect photovoltaic modules in the station, and transmit image data, radar point cloud data and positioning data collected during the inspection process to the scheduling platform.

[0046] The scheduling platform comprises an inspection scheduling module, a station model construction module, a defect identification module and a positioning and navigation module, wherein:

[0047] The inspection scheduling module is configured to obtain UAV state data, generate an inspection work order, send an inspection instruction to the UAV according to the inspection work order and the UAV state data, and receive image data, radar point cloud data and positioning data collected by the UAV during the inspection process.

[0048] The station model construction module is configured to construct a digital holographic model of the station based on the image data, the radar point cloud data and the positioning data. ​

[0049] The defect identification module is configured to identify a defect area of the photovoltaic module in the station based on the image data and using a pre-constructed defect identification model;

[0050] The positioning and navigation module is configured to obtain an actual position of the defect area in the image data according to the spatial coordinates and the shooting parameters of the image acquisition device and the height data recorded by the radar point cloud data, and to obtain a passable area in the station based on the digital holographic model of the station and the radar point cloud data, and to plan a maintenance navigation path based on the passable area and the actual position of the defect area.

[0051] Optionally, the photovoltaic module defect positioning and navigation system further comprises a maintenance mobile terminal configured to receive a maintenance instruction and maintenance navigation path data issued by the dispatching platform and guide a maintenance personnel to reach the photovoltaic module with defects based on the maintenance navigation path.

[0052] Advantages

[0053] The present application deeply integrates advanced unmanned aerial vehicle platforms, multi-source sensing technology, artificial intelligence image recognition, high-precision geographic information technology and path planning algorithms, and constructs a photovoltaic power station intelligent operation and maintenance solution integrating comprehensive perception, accurate diagnosis, accurate positioning and intelligent navigation, which can greatly improve the efficiency and safety of operation and maintenance, and through the provision of centimeter-level defect positioning and global optimal action guidelines, the operation and maintenance management level of photovoltaic power stations is improved to a new height, and has significant economic benefits and promotional value. BRIEF DESCRIPTION OF DRAWINGS

[0054] Figure 1 Fig. 1 shows a photovoltaic module defect positioning and navigation system according to an embodiment of the present application;

[0055] Figure 2 Fig. 2 shows a photovoltaic module defect positioning and navigation and operation scheduling system according to an embodiment of the present application;

[0056] Figure 3 Fig. 3 shows a defect identification model principle diagram according to an embodiment of the present application. DETAILED DESCRIPTION

[0057] The following is further described in combination with the drawings and specific embodiments.

[0058] Embodiment 1

[0059] This embodiment introduces a photovoltaic module defect positioning and navigation method, which can be implemented by a photovoltaic module operation and maintenance management platform or other dispatching servers for photovoltaic module inspection and operation and maintenance management, and refers to Figure 2 As shown in the figure, the method comprises:

[0060] Acquire the synchronously collected field station panoramic electronic map data, image data and radar point cloud data, and construct a field station digital holographic model;

[0061] Based on the image data, a pre-constructed defect recognition model is used to identify the defect area of the photovoltaic module in the field station in the image data;

[0062] According to the spatial coordinates and shooting parameters of the image acquisition device and the height data recorded by the radar point cloud data, the actual position of the defect area in the image data is obtained;

[0063] Based on the field station digital holographic model, the passable area in the field station is identified according to the radar point cloud data, and the actual position of the passable area and the defect area is used to plan the operation and maintenance navigation path with the shortest path as the target.

[0064] In combination Figure 1 , the above-mentioned scheme of the present application can be used in the photovoltaic module operation and maintenance system of the photovoltaic station, and the corresponding method is executed by the scheduling platform to realize the inspection scheduling and defect positioning navigation of the photovoltaic module in the field station, and then the operation and maintenance mobile terminal can be controlled to guide the operation and maintenance personnel to reach the defect photovoltaic module based on the operation and maintenance navigation path, complete the repair of the photovoltaic module, and ensure the operation reliability of the photovoltaic equipment.

[0065] The implementation of the photovoltaic module defect positioning navigation method of the present embodiment specifically involves the following contents.

[0066] I. Unmanned aerial vehicle inspection subsystem

[0067] In the present embodiment, the data acquisition work in the operation process of the photovoltaic module can be completed by the scheduling platform controlling the unmanned aerial vehicle inspection subsystem. The unmanned aerial vehicle inspection subsystem can be realized by a group of unmanned aerial vehicles carrying camera equipment, radar equipment, positioning equipment, etc. Each unmanned aerial vehicle is communicatively connected to the scheduling platform for interaction of photovoltaic module image data, radar data, positioning data and inspection scheduling instructions.

[0068] The scheduling platform can generate the optimal inspection route of the group of unmanned aerial vehicles according to the known field station layout to efficiently complete the inspection of all photovoltaic modules in the field station. The design of the inspection route of the group of unmanned aerial vehicles can refer to the prior art, which will not be described here.

[0069] The scheduling platform acquires the unmanned aerial vehicle state data, generates an inspection work order according to the inspection requirement, and sends the inspection instruction to the unmanned aerial vehicle according to the inspection work order and the unmanned aerial vehicle state data. The unmanned aerial vehicle performs field station inspection according to the received inspection instruction, and in the inspection process, visible light, infrared image and radar point cloud data are collected for the covered area of the inspection route, and the collected data are transmitted to the scheduling platform together with the time stamp, shooting parameter and coordinate information.

[0070] When receiving a large-scale inspection task, the scheduling platform can split the entire station into multiple inspection sub-regions according to all photovoltaic string point clouds and inspection tasks of the entire station, combine the real-time state of each UAV, and distribute sub-tasks to the entire UAV group and plan the start-end route of each UAV according to the real-time position, power and load type (visible light or infrared) of the available UAVs, so as to ensure that all UAVs complete the work almost simultaneously, maximize the overall operation efficiency, and reduce the total task time. This involves the joint solution of multi-agent task allocation and path planning, and the optimization solving process can be realized by referring to the prior art.

[0071] The image data in the embodiment includes orthographic image data and infrared thermal imaging data collected by the UAV, so as to support detection of multiple defect types of the photovoltaic component defect area, such as hot spots, component missing, diode failure, abnormally low temperature, dirt, and obstructions. The radar includes a laser radar and a millimeter wave radar, which are complementary to each other and can ensure the reliability of the inspection data.

[0072] II. Construction of digital holographic model of station

[0073] The embodiment uses the synchronously collected panoramic electronic map data, image data and radar point cloud data of the station to construct a digital holographic model of the station, specifically: generating a real scene three-dimensional model of the station according to the synchronously collected panoramic electronic map data and image data of the station, generating and updating a station obstacle map in real time according to the radar point cloud data, and superimposing the station obstacle map and a string topological relationship graph of the photovoltaic component on the real scene three-dimensional model of the station.

[0074] The digital holographic model of the station constructed above not only contains a three-dimensional real scene, but also integrates an obstacle map generated in real time by the radar point cloud, so that the planned operation and maintenance navigation path can dynamically avoid temporary accumulations, low hanging cables, newly grown vegetation and other ground or air obstacles, thereby ensuring the safety of personnel and vehicle travel.

[0075] Moreover, based on the data provided by the UAV inspection system, the digital holographic model and the obstacle map can be incrementally updated, that is, the new radar point cloud data collected each time is compared with the historical model to automatically identify newly added obstacles. This will help the scheduling platform to always optimize the operation and maintenance navigation path according to the latest station layout and obstacle distribution when planning the operation and maintenance navigation path, and realize the intelligentization of the operation and maintenance closed loop.

[0076] III. Defect recognition model

[0077] In view of the characteristics of "small target, complementary shape and thermal characteristics, and complex background" in photovoltaic defect detection, in order to realize reliable detection of the defect type and position of the photovoltaic component, the present application improves the existing small target recognition model, and refers toFigure 3 In the embodiment, the defect recognition model comprises a multi-source data alignment module, a double-branch feature extraction network module, an SPPFAST module, a feature pyramid network module, a decoupling detection head, a Gaussian Wasserstein distance loss function module, and a multi-modal weighted voting fusion module. Specifically,

[0078] The multi-source data alignment module is configured to receive visible light images and infrared image data in a plurality of image data, align the data by using a SIFT feature matching method, and synchronously transmit the aligned visible light images and infrared image data to two CSPNet-Darknet53 network branches of the double-branch feature extraction network module to extract feature data of the visible light images and the infrared images, respectively. The SPPFAST module is configured to expand the feature receptive fields of the visible light images and the infrared images to optimize the feature data of the visible light images and the infrared images. The feature pyramid network module is configured to extract features based on the optimized feature data of the visible light images and the infrared images to obtain multi-scale features corresponding to the visible light images and the infrared images, respectively. The decoupling detection head is configured to calculate defect category detection results and defect position detection results corresponding to different feature scales based on the multi-scale features. The Gaussian Wasserstein distance loss function module is configured to calculate a loss function based on the defect detection results output by the decoupling detection head and sample true value labels in a defect recognition model training stage to optimize network parameters of other modules in the defect recognition model. The multi-modal weighted voting fusion module is configured to perform weighted fusion based on the defect category detection results and the defect position detection results corresponding to different feature scales to obtain final defect category and position detection results.

[0079] In the training stage of the defect recognition model, the LabelImg or VGG Image Annotator tool is used to label photovoltaic component defect image samples containing a plurality of defect types, and then the training set and the test set are divided and input to the built defect recognition model network.

[0080] To realize efficient optimization of the model, the Gaussian Wasserstein distance is improved in the loss function part in the embodiment. Specifically, the loss function used by the Gaussian Wasserstein distance loss function module in the defect recognition model training process is represented as follows:

[0081]

[0082] wherein, Ltotal represents a total loss function value; Lgaussian represents a Gaussian distance loss between a predicted defect bounding box and a real defect bounding box; Lgaussian represents a Gaussian distance loss between a predicted defect bounding box and a real defect bounding box; and the actual defect bounding box The L1 loss between the predicted and actual defect bounding boxes is the sum of the absolute differences between the predicted and actual defect bounding boxes in terms of center coordinates, width, and height. and These are the weight parameters for Gaussian distance loss and L1 loss, respectively;

[0083] Gaussian distance loss Represented as:

[0084] in, These are hyperparameters used to adjust the sensitivity of the loss function. To predict the distance between the defect bounding box and the Gaussian distribution corresponding to the true defect bounding box, for The square of the L2 norm.

[0085] By optimizing the model parameters by combining L1 loss and Gaussian distance loss, the model effectively utilizes the characteristic that L1 loss is sensitive to small changes in bounding box parameters, as well as the advantages of Gaussian distance loss in robustness and imbalanced sample conditions, which can improve the accuracy, robustness and generalization ability of the model in object detection tasks.

[0086] IV. Defect Area Location

[0087] To plan subsequent maintenance navigation paths, this embodiment obtains the actual location of the defective area in the image data based on the spatial coordinates and shooting parameters of the image acquisition device, as well as the height data recorded by the radar point cloud data. Specifically, this includes:

[0088] Calculate the relative height of the defect area; and use the horizontal coordinates of the image acquisition device when it takes the picture as the coordinates of the image center position, and define it as a reference point, and calculate the horizontal pixel deviation of the defect area relative to the reference point;

[0089] Based on the relative height and focal length parameters, the horizontal pixel deviation is converted into a position offset in the physical coordinate system, thus obtaining the position coordinates of the defect area in the image data mapped to the panoramic electronic map.

[0090] Specifically, let's assume the spatial coordinates of the image acquisition device during shooting are... The pixel focal length in the shooting parameters is The camera yaw angle is The center pixel of the image is The center pixel of the defect area is The radar point cloud data records the height of the photographed defect location. The horizontal coordinate of the image center is then... ;

[0091] Calculate the relative height of the location of the defect area , the formula is: ;

[0092] The horizontal pixel deviation of the defect area relative to the reference point is: ;

[0093] Convert the horizontal pixel deviation to the position offset in the physical coordinate system, the formula is: ;

[0094] The calculation formula for mapping the horizontal coordinate of the defect area in the panoramic electronic map is:

[0095] ,

[0096] The actual position of the defect area in the image data is .

[0097] The embodiment directly maps the pixel-level defects in the image to the geographic coordinate system by accurately solving the image coordinates, radar point cloud elevation data and device space pose, and the positioning accuracy can reach centimeter level, completely solving the problem of only seeing defects and being unable to accurately position.

[0098] In addition, the embodiment also segments and identifies each photovoltaic module string in the image data according to the image data collected by the unmanned aerial vehicle, and assigns a unique ID and corresponding string GPS coordinates to each string according to the radar point cloud data, GPS data in the panoramic electronic map data, GPS coordinates and shooting parameters when the unmanned aerial vehicle shoots the image, and associates the defect area to the corresponding string ID and string GPS coordinates according to the position coordinates of the defect area mapped in the panoramic electronic map.

[0099] The above assigns a unique ID and corresponding string GPS coordinates to each string, specifically including:

[0100] The latitude and longitude coordinates of the string center point or key pixel point in the orthographic image of the unmanned aerial vehicle are taken as the latitude and longitude coordinates of the string, and the coordinate offset of the GPS coordinates relative to the image center point is calculated, the formula is:

[0101]

[0102] In the formula, is the pixel offset of the string center point or key pixel point relative to the center of the orthographic image, is the height when the unmanned aerial vehicle shoots the image, is the pixel focal length, is the resolution of the image;

[0103] The latitude and longitude coordinates of the string are calculated using the following formula:

[0104] ,

[0105] in, and These represent longitude and latitude coordinates, respectively. and These are the inverse projection function and the orthographic projection function of the Universal Transverse Mercator (UTM) projection, used for the transformation between geographic coordinates (latitude and longitude coordinates) and planar coordinates (UTM coordinates); These are the latitude and longitude coordinates of the images captured by the drone.

[0106] Through the above operations, the system can automatically associate the identified defects with specific string IDs and locations, enabling maintenance personnel to obtain accurate data and making the issuance of maintenance commands highly targeted.

[0107] V. Operation and Maintenance Navigation Path Planning

[0108] After obtaining the location and type of the defect area and the corresponding GPS coordinates of the photovoltaic modules, the dispatch platform can consider the actual geographical location of all defect points and the passable areas within the site. With the goal of minimizing the total path, it can perform global maintenance navigation path optimization calculations to generate the optimal inspection or repair route that connects multiple defect points at once. It can also generate maintenance work orders and send them to the maintenance personnel's terminal APP, as well as send the maintenance navigation path to the maintenance mobile terminal to guide maintenance personnel to the photovoltaic modules that need maintenance. This significantly reduces the ineffective movement time of maintenance personnel within the site and improves the efficiency of each work assignment.

[0109] In addition, after the maintenance personnel have repaired the defective area, they can take a picture of the repaired area and send it back to the dispatch platform. The dispatch platform can use the defect recognition model to detect the feedback image and verify whether the defect has been eliminated. If the defect has been eliminated, an maintenance record can be generated and archived. If the defect has not been eliminated, the corresponding work order needs to be reissued to the maintenance personnel to prompt them to repair the defective location again.

[0110] Example 2

[0111] This embodiment describes a computer-readable storage medium storing a computer program thereon, characterized in that, when the computer program is executed by a processor, it implements the photovoltaic module defect location and navigation method as described in Embodiment 1.

[0112] Example 3

[0113] This embodiment describes a photovoltaic module defect location and navigation system, referring to... Figure 1As shown, the system includes a scheduling platform, a UAV inspection subsystem, and a mobile terminal for operation and maintenance, and can further include a terminal for operation and maintenance personnel;

[0114] The UAV inspection subsystem includes a UAV group carrying a camera device, a radar device, and a positioning device;

[0115] The UAV inspection subsystem is configured to receive inspection instructions from the scheduling platform, generate an inspection path according to the inspection instructions or follow an inspection path specified by the inspection instructions, inspect photovoltaic modules in the station, and transmit image data, radar point cloud data, and positioning data collected during the inspection process to the scheduling platform;

[0116] The scheduling platform includes an inspection scheduling module, a station model construction module, a defect identification module, and a positioning and navigation module, wherein:

[0117] The inspection scheduling module is configured to obtain UAV state data, generate an inspection work order, send inspection instructions to the UAV based on the inspection work order and the UAV state data, and receive image data, radar point cloud data, and positioning data collected by the UAV during the inspection process;

[0118] The station model construction module is configured to construct a digital holographic model of the station based on the image data, radar point cloud data, and positioning data;

[0119] The defect identification module is configured to identify defect areas of photovoltaic modules in the station in the image data using a pre-constructed defect identification model based on the image data;

[0120] The positioning and navigation module is configured to obtain the actual position of the defect area in the image data based on the spatial coordinates and shooting parameters of the image acquisition device and the height data recorded by the radar point cloud data, identify the passable area in the station based on the radar point cloud data and the digital holographic model of the station, and plan an operation and maintenance navigation path based on the actual positions of the passable area and the defect area, with the shortest path as the target;

[0121] The mobile terminal for operation and maintenance is configured to receive operation and maintenance instructions and operation and maintenance navigation path data issued by the scheduling platform, and guide the operation and maintenance personnel who have received the operation and maintenance work order to reach the photovoltaic module with defects based on the operation and maintenance navigation path.

[0122] The implementation of the functions of the above parts can refer to Figure 2 and the corresponding content of the method of Example 1, which will not be repeated.

[0123] In addition, in order to improve the operation and maintenance efficiency, the operation and maintenance mobile terminal can support the following functions on the basis of the above: offline map loading, displaying the photovoltaic station partition and string coding in a network-free environment; maintenance knowledge base calling, automatically pushing the processing scheme according to the defect type; job record uploading, including the photo, processing personnel and time information before and after maintenance; and the like.

[0124] In order to realize the overall management and visual monitoring of the scheduling side, the scheduling platform can further be provided with: a flight operation monitoring module for displaying the unmanned aerial vehicle position, the pan-tilt parameter and the task progress in real time through a man-machine interface; an alarm management module for comparing and analyzing the thermal infrared and visible light images based on an AI model and outputting the defect type alarm through the man-machine interface; a work order closed-loop management module for automatically generating the defect elimination work order and tracking the maintenance state; and the like.

[0125] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0126] The present application is described with reference to flowcharts and / or block diagrams of the method, device (system) and computer program product according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in one or more flows and / or blocks. Figure 1 The functions specified in one or more flows and / or blocks.

[0127] These computer program instructions can also be stored in a computer-readable memory capable of guiding the computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a product including instruction devices, which implement the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in one or more flows and / or blocks. Figure 1 The functions specified in one or more flows and / or blocks.

[0128] These computer program instructions can also be loaded into a computer or other programmable data processing devices, so that a series of operational steps are generated to realize the computer-implemented processes, and the instructions executed on the computer or other programmable devices provide a process for implementing the functions specified in the flowchart Figure 1 one flow or multiple flows and / or the functions specified in the block Figure 1 one flow or multiple flows and / or the functions specified in the block

[0129] The embodiments of the present application are described above with reference to the accompanying drawings, but the present application is not limited to the specific embodiments described above, and the specific embodiments described above are merely illustrative, but not restrictive, and those of ordinary skill in the art can make many forms under the inspiration of the present application without departing from the purpose of the present application and the scope protected by the claims, which are all within the protection of the present application.

Claims

1. A method for locating and navigating defects in photovoltaic modules, comprising: Acquire synchronously collected panoramic electronic map data, image data, and radar point cloud data of the station to construct a digital holographic model of the station; Based on the image data, a pre-built defect identification model is used to identify defective areas of photovoltaic modules within the field station in the image data. Based on the spatial coordinates and shooting parameters of the image acquisition device, as well as the height data recorded by the radar point cloud data, the actual location of the defect area in the image data is obtained. Based on the digital holographic model of the site, the passable areas within the site are identified according to radar point cloud data. Based on the actual locations of the passable areas and the defective areas, the operation and maintenance navigation path is planned with the shortest path as the objective.

2. The method according to claim 1, characterized in that, The construction of the digital holographic model of the power station includes: generating a real-scene 3D model of the power station based on synchronously collected panoramic electronic map data and image data; generating and updating a power station obstacle map in real time based on radar point cloud data; and overlaying the power station obstacle map and the string topology relationship layer of photovoltaic modules onto the real-scene 3D model of the power station.

3. The method according to claim 1, characterized in that, The image data includes orthophoto data and infrared thermal imaging data collected by the UAV; the defect areas of the photovoltaic module include hot spot areas, missing module areas, diode failure areas, abnormally low temperature areas, dirty areas, and obstruction areas. The pre-built defect identification model includes a multi-source data alignment module, a dual-branch feature extraction network module, a SPPFAST module, a feature pyramid network module, a decoupled detection head, a Gaussian Wasserstein distance loss function module, and a multimodal weighted voting fusion module; wherein: The multi-source data alignment module receives visible light and infrared image data from a majority of image data, aligns the data using the SIFT feature matching method, and synchronously transmits the aligned visible light and infrared image data to the two CSPNet-Darknet53 network branches of the dual-branch feature extraction network module to extract feature data from the visible light and infrared images respectively. The SPPFAST module expands the feature receptive field of the visible light and infrared images to optimize their feature data. The feature pyramid network module performs feature extraction based on the optimized feature data of the visible light and infrared images. Multi-scale features corresponding to visible light and infrared images are extracted. The decoupled detection head is used to calculate defect category detection results and defect location detection results corresponding to different feature scales based on the multi-scale features. The Gaussian Wasserstein distance loss function module is used to calculate the loss function based on the defect detection results output by the decoupled detection head and the sample ground truth labels during the defect recognition model training stage, and to optimize the network parameters of other modules in the defect recognition model. The multimodal weighted voting fusion module is used to perform weighted fusion based on the defect category detection results and defect location detection results corresponding to different feature scales to obtain the final defect category and location detection results.

4. The method according to claim 3, characterized in that Gaussian... The loss function used by the Wasserstein distance loss function module during the training of the defect identification model is expressed as follows: in, This represents the total loss function value; This represents the Gaussian distance loss between the predicted defect bounding box and the actual defect bounding box. Represents the predicted defect bounding box and the actual defect bounding box The L1 loss between the predicted and actual defect bounding boxes is the sum of the absolute differences between the predicted and actual defect bounding boxes in terms of center coordinates, width, and height. and These are the weight parameters for Gaussian distance loss and L1 loss, respectively; Gaussian distance loss Represented as: in, These are hyperparameters used to adjust the sensitivity of the loss function. To predict the distance between the defect bounding box and the Gaussian distribution corresponding to the true defect bounding box, for The square of the L2 norm.

5. The method according to claim 1, characterized in that, The step of obtaining the actual location of the defect area in the image data based on the spatial coordinates and shooting parameters of the image acquisition device, as well as the height data recorded by the radar point cloud data, includes: Calculate the relative height of the defect area; and use the horizontal coordinates of the image acquisition device when it takes the picture as the coordinates of the image center position, and define it as a reference point, and calculate the horizontal pixel deviation of the defect area relative to the reference point; Based on the relative height and focal length parameters, the horizontal pixel deviation is converted into a position offset in the physical coordinate system, thus obtaining the position coordinates of the defect area in the image data mapped to the panoramic electronic map.

6. The method according to claim 5, characterized in that, Assume the spatial coordinates of the image acquisition device at the time of shooting are: The pixel focal length in the shooting parameters is The camera yaw angle is The center pixel of the image is The center pixel of the defect area is The radar point cloud data records the height of the photographed defect location. The horizontal coordinate of the image center is then... ; Calculate the relative height of the defect area. The formula is: ; The horizontal pixel deviation of the defect area relative to the reference point is: ; The formula for converting horizontal pixel deviation into position offset in the physical coordinate system is: ; The formula for calculating the horizontal coordinates of the defect area mapped to the panoramic electronic map is as follows: ， The actual location of the defect area in the image data is then... .

7. The method according to claim 5, characterized in that, The method also includes: segmenting and identifying the strings of photovoltaic modules in the image data, and assigning a unique ID and corresponding string GPS coordinates to each string based on radar point cloud data, GPS data in panoramic electronic map data, GPS coordinates when the drone captured the image and shooting parameters; The acquisition of the actual location of the defect area in the image data also includes: associating the defect area with the corresponding string ID and its string GPS coordinates based on the location coordinates of the defect area mapped to the panoramic electronic map.

8. The method according to claim 7, characterized in that, Assign a unique ID and corresponding GPS coordinates to each string group, including: Using the latitude and longitude coordinates of the center point or key pixels of the sequence in the UAV orthophoto image as the latitude and longitude coordinates of the sequence, calculate the coordinate offset of its GPS coordinates relative to the center point of the image, using the following formula: In the formula, This refers to the pixel offset of the center point or key pixel of the cluster relative to the center of the orthophoto image. The altitude at which the drone captured the image. For pixel focal length, The resolution of the image; The latitude and longitude coordinates of the string are calculated using the following formula: ， in, and These represent longitude and latitude coordinates, respectively. and These are the inverse projection function and the orthographic projection function of the Universal Transverse Mercator (UTM) projection, used for the transformation between geographic coordinates (latitude and longitude coordinates) and planar coordinates (UTM coordinates); These are the latitude and longitude coordinates of the images captured by the drone.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the photovoltaic module defect location and navigation method as described in any one of claims 1 to 8.

10. A photovoltaic module defect location and navigation system, characterized in that, It includes a dispatch platform and a drone inspection subsystem, wherein the drone inspection subsystem includes a swarm of drones equipped with camera equipment, radar equipment, and positioning equipment; The UAV inspection subsystem is used to receive inspection instructions from the dispatch platform, generate inspection paths according to the inspection instructions or according to the inspection paths specified by the inspection instructions, inspect the photovoltaic modules in the site, and transmit the image data, radar point cloud data and positioning data collected during the inspection to the dispatch platform. The scheduling platform includes an inspection scheduling module, a site model construction module, a defect identification module, and a positioning and navigation module, wherein: The inspection scheduling module is used to acquire UAV status data, generate inspection work orders, and send inspection instructions to UAVs based on the inspection work orders and UAV status data, and receive image data, radar point cloud data and positioning data collected by UAVs during the inspection process. The site model building module is used to build a digital holographic model of the site based on the image data, radar point cloud data, and positioning data. The defect identification module is used to identify defect areas of photovoltaic modules in the field station based on the image data using a pre-built defect identification model. The positioning and navigation module is used to obtain the actual location of the defect area in the image data based on the spatial coordinates and shooting parameters of the image acquisition device and the height data recorded by the radar point cloud data; and based on the digital holographic model of the site, to identify the passable area in the site according to the radar point cloud data, and to plan the operation and maintenance navigation path with the shortest path as the objective based on the actual location of the passable area and the defect area.

Images