Fault location tracking method, device and electronic equipment for photovoltaic power station

Abstract

The application discloses a fault positioning and tracking method, device and electronic equipment of a photovoltaic power station, and belongs to the technical field of image processing. The fault positioning and tracking of the photovoltaic power station comprises the following steps: acquiring a plurality of inspection images; performing fault identification on the inspection images to obtain first pixel coordinates of the fault in the current image; determining a plurality of adjacent images in the plurality of inspection images based on the positioning coordinates of the current image; determining a mapping relationship of the pixel coordinates in the current image and the adjacent images; determining second pixel coordinates in the plurality of adjacent images based on the first pixel coordinates and the mapping relationship; and determining three-dimensional coordinates of the fault point based on the first pixel coordinates, the second pixel coordinates in the plurality of adjacent images and the camera pose. Through the pixel position prediction mode, the pixel coordinates of the same fault point in the plurality of images are determined, the repeatability of the photovoltaic scene texture can be adapted, the difficulty of fault point pairing can be eliminated, the positioning of the fault point can be accurately realized, and the tracking of the fault point is also completed at the same time.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of image processing, and particularly relates to a fault positioning and tracking method, device and electronic equipment for a photovoltaic power station. BACKGROUND

[0002] With the development of unmanned aerial vehicle technology, the operation and maintenance of photovoltaic power stations gradually changes from manual inspection in the past to intelligent detection of photovoltaic component faults by unmanned aerial vehicles carrying visible light or infrared cameras to collect images. This method greatly improves the efficiency of photovoltaic power station operation and maintenance and saves a large amount of human resources.

[0003] In related technologies, image registration is required for visible light images and infrared images, and then registration is performed for visible light images and electronic maps of power stations. However, in actual execution, due to the large differences in resolution, shooting range, and the like of visible light images and infrared images, the texture difference between photovoltaic components is small, the actual registration is difficult, and the positioning accuracy is low. SUMMARY

[0004] The present application aims to at least solve one of the technical problems existing in the prior art. To this end, the present application provides a fault positioning and tracking method, device and electronic equipment for a photovoltaic power station, which can accurately position the fault point of a photovoltaic component and also complete tracking of the fault point of the photovoltaic component.

[0005] In a first aspect, the present application provides a fault positioning and tracking method for a photovoltaic power station, comprising:

[0006] obtaining multiple inspection images of the photovoltaic power station by an unmanned aerial vehicle;

[0007] performing fault identification on the inspection images, and obtaining a first pixel coordinate of a fault in a current image in a case where it is determined that the current image has a fault;

[0008] determining multiple adjacent images in the multiple inspection images based on a positioning coordinate of the current image;

[0009] determining a mapping relationship of pixel coordinates between the current image and the adjacent images;

[0010] determining a second pixel coordinate in the multiple adjacent images based on the first pixel coordinate and the mapping relationship;

[0011] determining a three-dimensional coordinate of a fault point based on the first pixel coordinate, the second pixel coordinate in the multiple adjacent images, and camera poses of the current image and the multiple adjacent images.

[0012] The fault positioning and tracking method of the photovoltaic power station according to the application determines the pixel coordinates of the same fault point in multiple images in a pixel position prediction manner, can adapt to the repeatability of the photovoltaic scene texture, eliminates the difficulty of fault point pairing, accurately realizes the positioning of the fault point, and simultaneously completes the tracking of the fault point.

[0013] According to an embodiment of the application, the determining of the mapping relationship of the pixel coordinates in the current image and the adjacent image comprises:

[0014] For multiple adjacent images, the mapping relationship of the pixel coordinates in each adjacent image and the current image is independently determined.

[0015] According to an embodiment of the application, the independently determining of the mapping relationship of the pixel coordinates in each adjacent image and the current image for multiple adjacent images comprises:

[0016] Feature points in the current image and the adjacent image are extracted to obtain the feature points in the current image and the adjacent image.

[0017] The pixel coordinates of the feature points in the current image are taken as samples, and the pixel coordinates of the same-named points in the adjacent image are taken as sample labels to train a pixel position prediction model, and the trained pixel position prediction model is taken as the mapping relationship.

[0018] The determining of the second pixel coordinates in multiple adjacent images based on the first pixel coordinates and the mapping relationship comprises: inputting the first pixel coordinates into each pixel position prediction model to obtain the second pixel coordinates in multiple adjacent images output by each pixel position prediction model.

[0019] According to an embodiment of the application, before the determining of the three-dimensional coordinates of the fault point, the method further comprises:

[0020] The multiple inspection images are three-dimensionally reconstructed to obtain the corrected camera poses of each inspection image.

[0021] According to an embodiment of the application, the three-dimensionally reconstructing of the multiple inspection images to obtain the corrected camera poses of each inspection image comprises:

[0022] The multiple inspection images are three-dimensionally reconstructed to obtain three-dimensional point cloud information, and the three-dimensional point cloud information comprises the pixel coordinates and the three-dimensional coordinates of multiple points.

[0023] Based on the pixel coordinates and the three-dimensional coordinates of the feature points in the inspection image, the camera pose of the inspection image is corrected.

[0024] According to one embodiment of this application, determining the three-dimensional coordinates of the fault point based on the first pixel coordinates, the second pixel coordinates in the plurality of adjacent images, and the camera pose of the current image and the plurality of adjacent images includes:

[0025] Multiple collinearity equations are constructed using the first pixel coordinates, the second pixel coordinates in the multiple adjacent images, and the camera pose.

[0026] The coordinates of the intersection point of the multiple collinear equations are used as the three-dimensional coordinates of the fault point.

[0027] Secondly, this application provides a fault location and tracking device for a photovoltaic power plant, the device comprising:

[0028] The first receiving module is used to acquire multiple inspection images of the photovoltaic power station via drone;

[0029] The first processing module is used to identify faults in the multiple inspection images, and if it is determined that there is a fault in the current image, obtain the first pixel coordinates of the fault in the current image.

[0030] The second processing module is used to determine multiple adjacent images among the multiple inspection images based on the positioning coordinates of the current image;

[0031] The third processing module is used to determine the mapping relationship between the pixel coordinates of the current image and the neighboring images;

[0032] The fourth processing module is used to determine the coordinates of the second pixel in multiple adjacent images based on the first pixel coordinates and the mapping relationship;

[0033] The fifth processing module is used to determine the three-dimensional coordinates of the fault point based on the first pixel coordinates, the second pixel coordinates in the multiple adjacent images, and the camera pose of the current image and the multiple adjacent images.

[0034] The fault location and tracking device for photovoltaic power plants according to this application determines the pixel coordinates of the same fault point in multiple images by pixel position prediction. It can be adapted to photovoltaic scenarios, eliminate the difficulty of fault point pairing, and accurately locate the fault point. At the same time, it also completes the tracking of the fault point to eliminate duplicate statistics.

[0035] Thirdly, this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the fault location and tracking method for a photovoltaic power station as described in the first aspect above.

[0036] In a fourth aspect, the present application provides a non-transitory computer-readable storage medium having a computer program stored thereon, the computer program, when executed by a processor, implements the fault locating and tracking method of the photovoltaic power station according to the first aspect.

[0037] In a fifth aspect, the present application provides a chip, the chip comprising a processor and a communication interface, the communication interface and the processor being coupled, the processor being configured to run a program or an instruction, and implement the fault locating and tracking method of the photovoltaic power station according to the first aspect.

[0038] In a sixth aspect, the present application provides a computer program product comprising a computer program, the computer program, when executed by a processor, implements the fault locating and tracking method of the photovoltaic power station according to the first aspect.

[0039] Additional aspects and advantages of the present application will be in part apparent and in part pointed out hereinafter. BRIEF DESCRIPTION OF DRAWINGS

[0040] The above and / or additional aspects and advantages of the present application will become apparent and be readily appreciated from the following description, including the appended drawings.

[0041] Figure 1 FIG. 1 is one of flow diagrams of the fault locating and tracking method of the photovoltaic power station according to an embodiment of the present application;

[0042] Figure 2 FIG. 2 is another of flow diagrams of the fault locating and tracking method of the photovoltaic power station according to an embodiment of the present application;

[0043] Figure 3 FIG. 3 is a structural diagram of the fault locating and tracking device of the photovoltaic power station according to an embodiment of the present application;

[0044] Figure 4 FIG. 4 is a structural diagram of the electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0045] The technical solutions in the embodiments of the present application will be clearly described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art belong to the scope of protection of the present application.

[0046] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0047] The following description, in conjunction with the accompanying drawings, details the fault location and tracking method, fault location and tracking device, electronic equipment, and readable storage medium for photovoltaic power plants provided in this application, through specific embodiments and application scenarios.

[0048] Among them, the fault location and tracking method of photovoltaic power station can be applied to the terminal, which can be executed by the hardware or software in the terminal.

[0049] The fault location and tracking method for photovoltaic power plants provided in this application embodiment can be executed by an electronic device or a functional module or entity in an electronic device that can implement the fault location and tracking method for photovoltaic power plants. The electronic devices mentioned in this application embodiment include, but are not limited to, mobile phones, tablets, computers, cameras and wearable devices. The fault location and tracking method for photovoltaic power plants provided in this application embodiment will be described below using an electronic device as the execution subject as an example.

[0050] A photovoltaic power station consists of multiple photovoltaic panels, multiple photovoltaic modules on each photovoltaic panel (one cell of a photovoltaic panel is one photovoltaic module), and multiple photovoltaic modules connected together to form a photovoltaic string.

[0051] The fault location and tracking method of this photovoltaic power station can detect the three-dimensional coordinates of the faulty photovoltaic module for subsequent maintenance.

[0052] The fault location and tracking method of this photovoltaic power station can determine the pixel coordinates of the faulty photovoltaic module in each inspection image, and realize the tracking of the faulty photovoltaic module on multiple inspection images.

[0053] like Figure 1 As shown, the fault location and tracking method for this photovoltaic power station includes steps 110, 120, 130, 140, 150 and 160.

[0054] Step 110: Obtain multiple inspection images of the photovoltaic power station using a drone;

[0055] The UAV can be equipped with a camera, and the type of image to be detected varies according to the type of camera. For example, when the UAV is equipped with a visible light camera, the inspection image is a visible light image; when the UAV is equipped with an infrared camera, the inspection image is an infrared image.

[0056] The plurality of inspection images in this step can be a group of inspection images obtained after the UAV completes a flight inspection of the photovoltaic power station. When the UAV flies over the photovoltaic power station, the camera carried by the UAV can collect images of the photovoltaic power station below.

[0057] The adjacent inspection images have an overlap rate, which refers to the degree of image overlap between adjacent images or adjacent flight lines when the UAV takes pictures along the flight line. The former is referred to as the forward overlap, and the latter is referred to as the lateral overlap. The overlap rate is expressed as a percentage of the length of the image overlap part to the length of the image frame, such as a forward overlap of 60% and a lateral overlap of 40%.

[0058] The UAV is equipped with a positioning module, including but not limited to a GPS module, a Beidou module, etc. When the camera carried by the UAV takes inspection images, the inspection images contain the positioning information of the UAV at the time of shooting. The inspection images also contain the pose information of the camera at the time of shooting, which includes positioning information and attitude information.

[0059] The UAV can also be equipped with a clock module. When the camera carried by the UAV takes inspection images, the inspection images contain time information at the time of shooting.

[0060] In step 120, fault identification is performed on the inspection images. When it is determined that the current image has a fault, the first pixel coordinates of the fault in the current image are obtained.

[0061] After obtaining the plurality of inspection images in step 110, fault identification can be sequentially performed on each inspection image. When no fault is identified in an inspection image, fault identification is continued on the next inspection image. For the inspection image in which a fault is identified, the current image is defined, and the first pixel coordinates of the fault in the current image can be obtained.

[0062] In some examples, the bounding box pixel coordinates, the center pixel coordinates of the fault in the current image, and the type information of the fault can also be obtained.

[0063] For the current image in which a fault is identified, the position prediction of the fault in other images can be achieved through subsequent steps 130-160, and finally the image tracking of the fault photovoltaic module and the determination of the three-dimensional coordinates of the fault can be achieved.

[0064] In actual execution, the fault identification in step 120 can be completed by using a fault identification model.

[0065] Step 120, fault recognition is performed on the inspection image, and in a case where it is determined that the current image has a fault, first pixel coordinates of the fault in the current image are obtained, including: inputting the inspection image into a fault recognition model, and in a case where it is determined that the current image has a fault, obtaining first pixel coordinates of the fault in the current image output by the fault recognition model, the fault recognition model being trained by taking a sample image as a sample, and taking a pre-determined sample pixel coordinate corresponding to the sample image as a sample label.

[0066] Alternatively, the fault recognition model can also output a fault type, and correspondingly, step 120, fault recognition is performed on the inspection image, and in a case where it is determined that the current image has a fault, first pixel coordinates of the fault in the current image are obtained, including: inputting the inspection image into a fault recognition model, and in a case where it is determined that the current image has a fault, obtaining first pixel coordinates of the fault in the current image and the fault type output by the fault recognition model, the fault recognition model being trained by taking a sample image as a sample, and taking a pre-determined sample pixel coordinate corresponding to the sample image and a sample fault type as a sample label.

[0067] Alternatively, the fault recognition model can also output a fault type, and correspondingly, step 120, fault recognition is performed on the inspection image, and in a case where it is determined that the current image has a fault, first pixel coordinates of the fault in the current image are obtained, including: inputting the inspection image into a fault recognition model, and in a case where it is determined that the current image has a fault, obtaining first pixel coordinates of the fault in the current image and the fault type output by the fault recognition model, the fault recognition model being trained by taking a sample image as a sample, and taking a pre-determined sample pixel coordinate corresponding to the sample image and a sample fault type as a sample label.

[0068] The fault recognition model can be a yolov5 model or the like.

[0069] Step 130, based on the positioning coordinates of the current image, a plurality of adjacent images are determined in the plurality of inspection images.

[0070] The unmanned aerial vehicle is equipped with a positioning module, including but not limited to a GPS module or a Beidou module, etc., and when the camera carried by the unmanned aerial vehicle shoots the inspection image, the inspection image has the positioning coordinates of the unmanned aerial vehicle at the time of shooting.

[0071] The positioning coordinates of the current image can be analyzed to obtain the positioning coordinates of the inspection image. Other inspection images also have positioning coordinates.

[0072] By matching the positioning coordinates of the current image with those of other inspection images, a plurality of adjacent images adjacent to the current image can be determined.

[0073] In actual execution, the distance between the positioning coordinates (x, y, z) of the current image and the positioning coordinates (x, y, z) of other inspection images can be calculated, and several inspection images closest to the current image among the other inspection images are taken as adjacent images.

[0074] In actual execution, the distance between the positioning coordinates (x, y, z) of the current image and the positioning coordinates (x, y, z) of other inspection images can be calculated, and several inspection images closest to the current image among the other inspection images are taken as adjacent images.

[0075] The plurality of adjacent images can include 4 images, and the 4 adjacent images include 4 adjacent images located in front, behind, left and right of the current image; or the plurality of adjacent images can include 8 images, and the 8 visible light images include 4 adjacent images located in front, behind, left and right of the current image, and 4 adjacent images located in front left, back left, front right and back right of the current image.

[0076] Other combinations of adjacent images are not listed here.

[0077] 0It should be noted that, due to the overlap rate between adjacent inspection images, the adjacent images actually also contain

[0078] The fault point corresponding to the current image.

[0079] Step 140, determining the mapping relationship of pixel coordinates in the current image and adjacent images;

[0080] In this step, the mapping relationship of pixel coordinates between the two images is established, so that the pixel coordinates of a point in one image can be predicted in the other image.

[0081] 5The mapping relationship can be determined in various ways:

[0082] First, the mapping matrix is calculated by various data of the two images.

[0083] In this embodiment, the mapping matrix can be determined in the following way:

[0084] Obtain the positioning coordinates, attitude parameters and flight height information of the cameras of the two images;

[0085] Determine the camera attitude rotation matrix based on the camera attitude parameters of the two images;

[0086] 0Determine the mapping relationship of pixel coordinates in the two images based on the camera intrinsic matrix, camera attitude rotation matrix, camera positioning coordinates and flight height

[0087] information corresponding to the two images.

[0088] Second, a pixel position prediction model is trained to reflect the mapping relationship of the two images.

[0089] In this embodiment, feature points can be extracted from the two images, and paired feature points can be obtained through feature matching, and the pixel position prediction model is trained through the paired feature points.

[0090] In some examples, only the pixel coordinates of the paired feature points are used to complete the training, and each pixel position prediction model can only be used for pixel point mapping between specific two images.

[0091] In some examples, the mapping relationship between the pixel coordinates in the current image and the neighboring image is determined based on the pixel coordinates of the matched feature points and the camera pose.

[0092] In step 150, the second pixel coordinates in the plurality of neighboring images are determined based on the first pixel coordinates and the mapping relationship.

[0093] It can be understood that after the mapping relationship between the pixel coordinates in the current image and the neighboring image is obtained in step 140, the second pixel coordinates of the fault point determined from the current image in the neighboring image can be predicted.

[0094] There are a plurality of neighboring images, and correspondingly, the second pixel coordinates in the plurality of neighboring images can be obtained.

[0095] In the related art, fault identification needs to be performed on each image. Due to the shooting angle, some fault points cannot be accurately identified on some images, and due to the repetition of the photovoltaic scene texture, it is difficult to pair the fault points identified in the plurality of images.

[0096] In the present application, for the same fault point on the neighboring images, the pixel prediction method is adopted, which can be adapted to the photovoltaic scene.

[0097] In step 160, the three-dimensional coordinates of the fault point are determined based on the first pixel coordinates, the second pixel coordinates in the plurality of neighboring images, and the camera poses of the current image and the plurality of neighboring images.

[0098] Through the preceding steps 110-150, the pixel coordinates of the fault point in the plurality of images are obtained, and the camera poses of each image are known. In this way, through the multi-line common point method, the three-dimensional coordinates of the fault point can be obtained.

[0099] According to the fault positioning and tracking method of the photovoltaic power station provided in the embodiments of the present application, the pixel coordinates of the same fault point in the plurality of images are determined through the pixel position prediction method, which can be adapted to the photovoltaic scene, eliminates the difficulty of pairing the fault points, and accurately realizes the positioning of the fault point. At the same time, the tracking of the fault point is also completed, so as to eliminate the repeated statistics.

[0100] In some embodiments, in step 140, the mapping relationship between the pixel coordinates in the current image and the neighboring image is determined, including:

[0101] For the plurality of neighboring images, the mapping relationship between the pixel coordinates of each neighboring image and the current image is independently determined.

[0102] In other words, in this embodiment, each adjacent image has an independent mapping relationship with the current image, so that the prediction of the second pixel coordinates of the adjacent image is more accurate, and the amount of data required for constructing the independent mapping relationship is less, and the training process is simpler.

[0103] Taking the pixel position prediction model with the above mapping relationship as an example, the mapping relationship between each adjacent image and the current image is independently determined for a plurality of adjacent images, including:

[0104] Feature points in the current image and the adjacent image are extracted to obtain the feature points in the current image and the adjacent image.

[0105] The pixel position prediction model is trained by taking the pixel coordinates of the feature points in the current image as samples and taking the pixel coordinates of the homonymic points in the adjacent image as sample labels, and the trained pixel position prediction model is used as the mapping relationship.

[0106] Based on the first pixel coordinates and the mapping relationship, the second pixel coordinates in the plurality of adjacent images are determined, including: inputting the first pixel coordinates into each pixel position prediction model to obtain the second pixel coordinates in the plurality of adjacent images output by each pixel position prediction model.

[0107] In other words, when the second pixel coordinates of the fault point in the first adjacent image need to be predicted, the feature points in the current image and the first adjacent image are extracted to obtain the feature points in the current image and the first adjacent image.

[0108] The feature points in the current image and the feature points in the first adjacent image are matched to obtain the homonymic points of the feature points in the current image, and then the pixel position prediction model is trained by using the pixel coordinates of the feature points in the current image and the pixel coordinates of the corresponding homonymic points. The trained pixel position prediction model can predict the second pixel coordinates of the fault point in the first adjacent image.

[0109] The above operations are repeated for other adjacent images.

[0110] In other words, in this method, the corresponding pixel position prediction model is independently trained for each adjacent image; based on the first pixel coordinates and the mapping relationship, the second pixel coordinates in the plurality of adjacent images are determined, including: inputting the first pixel coordinates into each pixel position prediction model to obtain the second pixel coordinates output by each pixel position prediction model.

[0111] In this way, when training the pixel position prediction model corresponding to each adjacent image, a relatively small amount of samples can be used to train a relatively accurate model, and the accuracy of the finally predicted second pixel coordinates is also higher.

[0112] In some embodiments, after the plurality of inspection images of the photovoltaic power station are acquired by the UAV in step 110, and before the three-dimensional coordinates of the fault point are determined in step 160, the fault locating and tracking method can further include:

[0113] performing three-dimensional reconstruction on the plurality of inspection images to obtain the corrected camera pose of each inspection image.

[0114] It should be noted that each inspection image carries camera pose data, but the initial camera pose obtained by analyzing the image may not be accurate. By three-dimensional reconstruction, the initial camera pose can be corrected, so that the corrected camera pose can be used in step 160 to more accurately locate the three-dimensional coordinates.

[0115] The adjacent inspection images have an overlap rate, so that by processing the plurality of inspection images of the photovoltaic power station through three-dimensional reconstruction technology, the corrected camera pose of each inspection image can be obtained. Of course, when performing three-dimensional reconstruction, a sparse three-dimensional point cloud and pixel coordinates can also be obtained.

[0116] In some examples, performing three-dimensional reconstruction on the plurality of inspection images to obtain the corrected camera pose of each inspection image includes: performing three-dimensional reconstruction on the plurality of inspection images to obtain three-dimensional point cloud information, the three-dimensional point cloud information including pixel coordinates and three-dimensional coordinates of a plurality of feature points; and correcting the camera pose of the inspection image based on the pixel coordinates and three-dimensional coordinates of the feature points in the inspection image. In actual execution, a pose estimation algorithm such as EPNP algorithm can be used.

[0117] In some embodiments, step 160 includes determining the three-dimensional coordinates of the fault point based on the first pixel coordinates, the second pixel coordinates in the plurality of adjacent images, and the camera poses of the current image and the plurality of adjacent images, including:

[0118] constructing a plurality of collinear equations based on the first pixel coordinates, the second pixel coordinates in the plurality of adjacent images, and the camera poses;

[0119] taking the intersection coordinates of the plurality of collinear equations as the three-dimensional coordinates of the fault point.

[0120] It can be understood that the first pixel coordinates and the second pixel coordinates in the plurality of adjacent images both point to the same fault point. For this fault point, the current image and the plurality of adjacent images can be regarded as multiple views. The plurality of collinear equations constructed based on the first pixel coordinates, the second pixel coordinates in the plurality of adjacent images, and the camera poses can be used to solve the three-dimensional coordinates (x, y, z) of the fault photovoltaic module.

[0121] The fault locating and tracking method for the photovoltaic power station includes:

[0122] acquiring a plurality of inspection images of the photovoltaic power station by a UAV;

[0123] correcting the camera poses of the plurality of inspection images through three-dimensional reconstruction;

[0124] fault identification is performed on the inspection images to obtain a current image with a fault and first pixel coordinates of the fault in the current image;

[0125] based on the camera pose of the current image, a neighboring image is determined from the plurality of inspection images;

[0126] feature point extraction is performed on the current image and the neighboring image to obtain a corresponding feature point pair;

[0127] a pixel position prediction model is trained based on the feature point pair;

[0128] the first pixel coordinates of the fault in the current image are input into the trained pixel position prediction model to obtain second pixel coordinates of the fault in the neighboring image;

[0129] based on the pixel coordinates of the fault point in each image and the camera poses of each image, a multi-view Figure Three angular method is used to determine the three-dimensional coordinates of the fault point.

[0130] An embodiment of the fault locating method of the photovoltaic power station will be described below with reference to the accompanying drawings. Figure 2 The fault locating method takes a drone carrying an infrared camera as an example.

[0131] The fault locating and tracking method of the photovoltaic power station includes steps 201-203 and steps 210-260.

[0132] Step 201: Obtain a plurality of inspection images.

[0133] The photovoltaic power station is inspected and photographed by a drone to obtain inspection images, and adjacent inspection images have an overlap rate, such as a heading overlap degree of 60% and a lateral overlap degree of 40%.

[0134] Step 202: Three-dimensional reconstruction.

[0135] The inspection images are processed by three-dimensional reconstruction technology to establish three-dimensional point cloud information of the photovoltaic power station.

[0136] Step 203: Correct the camera pose.

[0137] The camera poses of each inspection image are corrected based on the three-dimensional point cloud information of the photovoltaic power station.

[0138] Step 210: Obtain each inspection image.

[0139] Step 220: Fault identification

[0140] In this step, fault identification is performed on the inspection image to obtain the pixel coordinates of the fault.

[0141] Step 230: Image feature extraction;

[0142] Feature extraction was performed on the inspection image to obtain multiple feature points.

[0143] Step 240: Pixel position prediction;

[0144] The pixel coordinates of the predicted fault are the pixel coordinates in the adjacent inspection images.

[0145] Step 250, Multi-view Figure Three Keratinization;

[0146] The first pixel coordinate and the second pixel coordinate in multiple adjacent images both point to the same fault point. For this fault point, the current image and multiple adjacent images can be regarded as multiple views. Multiple collinear equations are constructed by the first pixel coordinate, the second pixel coordinate in multiple adjacent images and the camera pose. The coordinates of the intersection of multiple collinear equations are used as the three-dimensional coordinates of the fault point.

[0147] Step 260: Tracking faulty photovoltaic modules.

[0148] The fault location and tracking method for photovoltaic power plants provided in the embodiments of this application determines the pixel coordinates of the same fault point in multiple images by predicting the pixel position. This method can be adapted to photovoltaic scenarios, eliminate the difficulty of fault point pairing, and accurately locate the fault point. At the same time, it also completes the tracking of the fault point to eliminate duplicate statistics.

[0149] The fault location and tracking method for photovoltaic power plants provided in this application can be executed by a fault location and tracking device for photovoltaic power plants. This application uses the execution of the fault location and tracking method by a fault location and tracking device for photovoltaic power plants as an example to illustrate the fault location and tracking device for photovoltaic power plants provided in this application.

[0150] This application also provides a fault location and tracking device for a photovoltaic power station.

[0151] like Figure 3 As shown, the fault location and tracking device of the photovoltaic power station includes: a first receiving module 310, a first processing module 320, a second processing module 330, a third processing module 340, a fourth processing module 350 and a fifth processing module 360.

[0152] The first receiving module 310 is used to acquire multiple inspection images of the photovoltaic power station via a drone;

[0153] The first processing module 320 is configured to perform fault identification on the inspection image, and obtain first pixel coordinates of the fault in the current image when it is determined that the current image has a fault.

[0154] The second processing module 330 is configured to determine a plurality of adjacent images in the plurality of inspection images based on the positioning coordinates of the current image.

[0155] The third processing module 340 is configured to determine a mapping relationship between pixel coordinates in the current image and the adjacent images.

[0156] The fourth processing module 350 is configured to determine second pixel coordinates of the plurality of adjacent images based on the first pixel coordinates and the mapping relationship.

[0157] The fifth processing module 360 is configured to determine three-dimensional coordinates of the fault point based on the first pixel coordinates, the second pixel coordinates of the plurality of adjacent images, and camera poses of the current image and the plurality of adjacent images.

[0158] According to the fault positioning and tracking device of the photovoltaic power station provided in the embodiments of the present application, the pixel coordinates of the same fault point in a plurality of images are determined by the pixel position prediction method, which can adapt to the photovoltaic scene, eliminate the difficulty of fault point pairing, accurately realize the positioning of the fault point, and also complete the tracking of the fault point to eliminate repeated statistics.

[0159] In some embodiments, the third processing module 340 is further configured to independently determine, for the plurality of adjacent images, a mapping relationship between pixel coordinates in each adjacent image and the current image.

[0160] In some embodiments, the third processing module 340 is further configured to perform feature point extraction on the current image and the adjacent image to obtain feature points in the current image and the adjacent image.

[0161] The pixel position prediction model is trained by taking the pixel coordinates of the feature points in the current image as samples and taking the pixel coordinates of the same-named points in the adjacent image as sample labels, and the trained pixel position prediction model is used as the mapping relationship.

[0162] Based on the first pixel coordinates and the mapping relationship, the second pixel coordinates of the plurality of adjacent images are determined, including: inputting the first pixel coordinates into each pixel position prediction model to obtain the second pixel coordinates of the plurality of adjacent images output by each pixel position prediction model.

[0163] In some embodiments, the fault positioning and tracking device can further include a sixth processing module configured to perform three-dimensional reconstruction on the plurality of inspection images to obtain corrected camera poses of each inspection image before determining the three-dimensional coordinates of the fault point.

[0164] In some embodiments, the sixth processing module is further configured to perform three-dimensional reconstruction on the plurality of inspection images to obtain three-dimensional point cloud information, the three-dimensional point cloud information comprising pixel coordinates and three-dimensional coordinates of a plurality of points; and correct the camera pose of the inspection image based on the pixel coordinates and the three-dimensional coordinates of the feature points in the inspection image.

[0165] In some embodiments, the fifth processing module 360 is further configured to construct a plurality of collinear equations based on the first pixel coordinates, second pixel coordinates in the plurality of adjacent images, and the camera pose.

[0166] The intersection coordinates of the plurality of collinear equations are taken as the three-dimensional coordinates of the fault point.

[0167] The photovoltaic power station fault positioning and tracking device in the embodiments of the present applicationapplicationbe an electronic device or a component in an electronic device, such as an integrated circuit or a chip. The electronic deviceapplicationbe a terminal or other devices other than a terminal. For example, the electronic deviceapplicationbe a mobile phone, a tablet computer, a notebook computer, a palm computer, a vehicle-mounted electronic device, a mobile Internet device (MID), an augmented reality (AR) / virtual reality (VR) device, a robot, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), andapplicationbe a server, a network attached storage (NAS), a personal computer (PC), a television (TV), a teller machine, or a self-service machine, and the like, and the embodiments of the present application do not make specific limitations.

[0168] The photovoltaic power station fault positioning and tracking device in the embodiments of the present applicationapplicationbe a device with an operating system. The operating systemapplicationbe a Microsoft (Windows) operating system, an Android operating system, an IOS operating system, or other possible operating systems, and the embodiments of the present application do not make specific limitations.

[0169] The photovoltaic power station fault positioning and tracking device provided in the embodiments of the present applicationapplicationbe able to implement the method embodiments Figures 1-2 The processes implemented by the method embodiments are not repeated here to avoid repetition.

[0170] In some embodiments, as Figure 4As shown, the electronic device 400 according to the embodiment of the present application further comprises a processor 401, a memory 402 and a computer program stored in the memory 402 and executable on the processor 401. The computer program is executed by the processor 401 to implement each process of the method for locating and tracking fault of photovoltaic power station according to the above embodiment, and achieve the same technical effects. To avoid repetition, details are not described herein.

[0171] It should be noted that the electronic device in the embodiment of the present application includes the mobile electronic device and the non-mobile electronic device.

[0172] The embodiment of the present application further provides a non-transitory computer readable storage medium, which stores a computer program. The computer program is executed by a processor to implement each process of the method for locating and tracking fault of photovoltaic power station according to the above embodiment, and achieve the same technical effects. To avoid repetition, details are not described herein.

[0173] The processor is the processor in the electronic device according to the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer readable only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0174] The embodiment of the present application further provides a computer program product, which includes a computer program. The computer program is executed by a processor to implement the method for locating and tracking fault of photovoltaic power station according to the above embodiment.

[0175] The processor is the processor in the electronic device according to the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer readable only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0176] The embodiment of the present application further provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is configured to run a program or an instruction to implement each process of the method for locating and tracking fault of photovoltaic power station according to the above embodiment, and achieve the same technical effects. To avoid repetition, details are not described herein.

[0177] It should be understood that the chip mentioned in the embodiment of the present application can also be referred to as a system level chip, a system chip, a chip system or a system on chip, etc.

[0178] It should be noted that, in the present document, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises a", "comprising", or "comprises" does not, without more constraints, preclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element. Additionally, it should be noted that the terms "one embodiment", "some embodiments", "certain embodiments", "certain examples", or "some examples" as used in the present document are intended to refer to one or more embodiments or examples that do not necessarily have to cover all embodiments or examples of the present application. In other words, use of the above terms does not necessarily refer to the same embodiment or example. Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0179] From the above description of the embodiments, it is clear that the above-described method of the embodiments can be realized by means of software and a general-purpose hardware platform as required, and of course, can also be realized by hardware, but in many cases, the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a computer software product, which is stored in a storage medium (such as a ROM / RAM, a magnetic disk, an optical disk), and includes a plurality of instructions for causing a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the method described in the embodiments of the present application.

[0180] The embodiments of the present application are described above in combination with the drawings, but the present application is not limited to the above-described specific embodiments, and the above-described specific embodiments are merely illustrative, rather than limiting, and those of ordinary skill in the art can make many forms without departing from the purpose of the present application and the scope protected by the claims under the inspiration of the present application, which are all within the protection of the present application.

[0181] In the description of the present specification, the description of the terms "one embodiment", "some embodiments", "certain embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Furthermore, the described specific features, structures, materials, or characteristics can be combined in any suitable manner in any one or more embodiments or examples.

[0182] While the embodiments of the application have been shown and described, it is to be understood that the embodiments can be varied, modified, substituted and changed by those skilled in the art without departing from the principles and spirit of the application, the scope of which is defined by the claims and their equivalents.

Claims

1. A fault location and tracing method for a photovoltaic power station, characterized in that, include: Multiple inspection images of photovoltaic power plants were obtained using drones; Fault identification is performed on the inspection image, and if a fault is determined in the current image, the first pixel coordinates of the fault in the current image are obtained; Based on the positioning coordinates of the current image, multiple adjacent images are determined among the multiple inspection images; Determine the mapping relationship between the pixel coordinates of the current image and the neighboring images; Based on the first pixel coordinates and the mapping relationship, the second pixel coordinates in multiple adjacent images are determined; Based on the first pixel coordinates, the second pixel coordinates in the multiple adjacent images, and the camera pose of the current image and the multiple adjacent images, the three-dimensional coordinates of the fault point are determined. Determining the mapping relationship between the current image and the pixel coordinates in the neighboring images includes: For multiple adjacent images, independently determine the mapping relationship between the mid-pixel coordinates of each adjacent image and the current image; Determining the mapping relationship between the mid-pixel coordinates of each of the multiple adjacent images and the current image independently includes: Feature points are extracted from the current image and the adjacent images to obtain the feature points in the current image and the adjacent images; Using the pixel coordinates of feature points in the current image as samples and the pixel coordinates of corresponding points in adjacent images as sample labels, a pixel position prediction model is trained, and the trained pixel position prediction model is used as the mapping relationship. The step of determining the second pixel coordinates in multiple adjacent images based on the first pixel coordinates and the mapping relationship includes: inputting the first pixel coordinates into each of the pixel position prediction models to obtain the second pixel coordinates in multiple adjacent images output by each of the pixel position prediction models.

2. The fault location and tracking method for photovoltaic power plants according to claim 1, characterized in that, Before determining the three-dimensional coordinates of the fault point, the method further includes: The multiple inspection images are reconstructed in three dimensions to obtain the corrected camera pose for each inspection image.

3. The fault location and tracking method for photovoltaic power plants according to claim 2, characterized in that, The step of performing 3D reconstruction on the multiple inspection images to obtain the corrected camera pose for each inspection image includes: The multiple inspection images are reconstructed in three dimensions to obtain three-dimensional point cloud information, which includes the pixel coordinates and three-dimensional coordinates of multiple points. The camera pose of the inspection image is corrected based on the pixel coordinates and three-dimensional coordinates of the feature points in the inspection image.

4. The fault location and tracing method for a photovoltaic power station according to any one of claims 1-3, characterized in that, Determining the three-dimensional coordinates of the fault point based on the first pixel coordinates, the second pixel coordinates in the plurality of adjacent images, and the camera pose of the current image and the plurality of adjacent images includes: Multiple collinearity equations are constructed using the first pixel coordinates, the second pixel coordinates in the multiple adjacent images, and the camera pose. The coordinates of the intersection point of the multiple collinear equations are used as the three-dimensional coordinates of the fault point.

5. A fault location and tracking device for a photovoltaic power station, characterized in that, The fault location and tracking device is used to implement the fault location and tracking method for a photovoltaic power station as described in any one of claims 1-4, and the fault location and tracking device includes: The first receiving module is used to acquire multiple inspection images of the photovoltaic power station via drone; The first processing module is used to identify faults in the inspection image and, if it is determined that there is a fault in the current image, obtain the first pixel coordinates of the fault in the current image. The second processing module is used to determine multiple adjacent images among the multiple inspection images based on the positioning coordinates of the current image; The third processing module is used to determine the mapping relationship between the pixel coordinates of the current image and the neighboring images; The fourth processing module is used to determine the coordinates of the second pixel in multiple adjacent images based on the first pixel coordinates and the mapping relationship; The fifth processing module is used to determine the three-dimensional coordinates of the fault point based on the first pixel coordinates, the second pixel coordinates in the multiple adjacent images, and the camera pose of the current image and the multiple adjacent images.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the fault location and tracking method for photovoltaic power plants as described in any one of claims 1-4.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the fault location and tracking method for a photovoltaic power plant as described in any one of claims 1-4.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the fault location and tracking method for a photovoltaic power station as described in any one of claims 1-4.

Images