Fault component positioning method and device and storage medium

Abstract

Embodiments of the present application disclose a fault component positioning method, device and storage medium. The fault component positioning method comprises: acquiring an infrared image and a panoramic image of a photovoltaic string; determining a first logical position of a fault component in the string and a fault positioning of the fault component in the infrared image according to the infrared image; determining a second logical position of the fault component in the panoramic image according to the fault positioning and the panoramic image; and correcting the second logical position of the fault component in the panoramic image according to the first logical position and the second logical position, to obtain an accurate logical positioning of the fault component in the panoramic image. The present scheme can obtain accurate positioning of the fault component in the panoramic image.

Description

Technical Field

[0001] The present invention relates to the field of photovoltaic technology, and in particular to a method, apparatus and storage medium for locating faulty components. Background Technology

[0002] In intelligent photovoltaic (PV) power plant operation and maintenance scenarios, drones equipped with infrared cameras capture two-dimensional infrared images of the PV array. Fault detection algorithms then pinpoint the location of faulty components within these images. To facilitate faster and more convenient location of faulty components in the actual environment, the location information of these components needs to be correlated with a panoramic map of the PV array. Any deviation in this correlation will directly impact the troubleshooting effectiveness at the power plant, thereby reducing the overall stability of the PV power plant system.

[0003] In the prior art, the invention patent application CN115100296A, entitled "A Method, Apparatus, Device, and Storage Medium for Fault Location of Photovoltaic Modules," proposes to establish a mapping relationship between two-dimensional image pixel coordinates and reconstructed three-dimensional point clouds using UAV photogrammetry. It then leverages the coplanarity of the three-dimensional point clouds on the surface of the photovoltaic string and the collinearity between the captured fault point and the image pixels to solve for the three-dimensional coordinates corresponding to the two-dimensional pixel fault point. However, this method relies on the quality and quantity of the three-dimensional point cloud reconstructed from infrared images, which may lead to inaccurate module positioning in panoramic images during practical use. Summary of the Invention

[0004] This invention provides a method, apparatus, and storage medium for locating faulty components, so as to obtain the accurate location of faulty components in panoramic images.

[0005] In a first aspect, embodiments of the present invention provide a method for locating a faulty component, the method comprising:

[0006] Acquire infrared and panoramic images of the photovoltaic string;

[0007] Based on the infrared image, determine the first logical position of the faulty component in the string and the fault location of the faulty component in the infrared image;

[0008] Based on the fault location and the panoramic image, determine the second logical position of the faulty component in the panoramic image;

[0009] Based on the first logical position and the second logical position, the second logical position of the faulty component in the panoramic image is corrected to obtain the precise logical location of the faulty component in the panoramic image.

[0010] Optionally, determining the first logical location of the faulty component in the string based on the infrared image includes:

[0011] The infrared image is subjected to distortion correction to obtain a corrected image;

[0012] The corrected image is segmented to obtain the component results of each group string separation in the corrected image;

[0013] Fault detection is performed on the infrared image to determine the pixel coordinates of the faulty component;

[0014] Based on the pixel coordinates of the faulty component and the component results separated from each string in the corrected image, the first logical position of the faulty component in the string is determined.

[0015] Optionally, the step of obtaining the corrected image includes:

[0016] The infrared image is reconstructed in three dimensions to obtain the camera that captured the infrared image, intrinsic parameters, and the reconstructed distortion parameter coefficients;

[0017] The infrared image is corrected based on the distortion parameter coefficients, the intrinsic parameters of the infrared image camera, and the infrared image to obtain the corrected image.

[0018] Optionally, the step of obtaining the component results of separating each group string in the corrected image includes:

[0019] The corrected image is segmented into its constituent strings to obtain the first image data of each string.

[0020] Self-correction is performed on the first image data of each group to obtain corrected group image data;

[0021] Each of the correction string image data is segmented and filled into components to obtain the component results of each correction string image data separation.

[0022] Optionally, the component result includes the number of each component relative to its string;

[0023] For each of the corrected string image data, component segmentation and padding are performed to obtain the component results of each corrected string image data separation, including:

[0024] Component segmentation is performed on each of the corrected string image data;

[0025] The unseparated components are populated to obtain populated component data;

[0026] Based on the filling component data and the first image data of each string, the beginning and end of each string are defined;

[0027] Each component is numbered according to the beginning and end of each string, thus obtaining the number of each component relative to its string.

[0028] Optionally, the step of determining the fault location of the faulty component in the infrared image based on the infrared image includes:

[0029] The infrared image is reconstructed in three dimensions to obtain the point cloud data corresponding to the infrared image.

[0030] The fault location of the faulty component is determined from each of the point cloud data.

[0031] Optionally, the step of determining the second logical location of the faulty component in the panoramic image includes:

[0032] The panoramic image is processed to obtain a string segmentation mask;

[0033] The second image data of the sequence in the panoramic image is marked according to the sequence segmentation mask;

[0034] The second image data is segmented to obtain the component segmentation results for each string;

[0035] Based on the fault location and the component segmentation results of each group, the second logical position of the faulty component in the panoramic image is determined.

[0036] Optionally, the step of obtaining the precise logical location of the faulty component in the panoramic image includes:

[0037] Determine the position of the second logical position relative to the string to obtain the third logical position;

[0038] Determine whether the third logical position is the same as the first logical position;

[0039] If they are the same, then the second logical position is the precise logical location of the faulty component in the panoramic image;

[0040] If they are different, the third logical position is adjusted according to the first logical position, and the precise logical location of the faulty component in the panoramic image is determined according to the third logical position.

[0041] Secondly, embodiments of the present invention also provide a fault component location device, the fault component location device comprising:

[0042] The acquisition module is used to acquire infrared and panoramic images of the photovoltaic strings;

[0043] The first positioning module is used to determine the first logical position of the faulty component in the string and the fault location of the faulty component in the infrared image based on the infrared image.

[0044] The second positioning module is used to determine the second logical position of the faulty component in the panoramic image based on the fault location and the panoramic image.

[0045] A precise logical positioning determination module is used to correct the second logical position of the faulty component in the panoramic image based on the first logical position and the second logical position, so as to obtain the precise logical position of the faulty component in the panoramic image.

[0046] Thirdly, embodiments of the present invention also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the fault component location method provided in any embodiment of the present invention.

[0047] This invention determines the precise first logical position of a faulty component within a string based on infrared images; it then determines the fault location of the faulty component within the infrared image, and finally determines the second logical position of the faulty component within the panoramic image based on the fault location and the panoramic image. Since the determination of the second logical position of the faulty component in the panoramic image is affected by the quality and quantity of the 3D point cloud reconstructed from the infrared image, there may be a positioning offset. This invention uses the precise first logical position to correct the second logical position of the faulty component in the panoramic image, thereby obtaining the precise logical location of the faulty component within the panoramic image. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0049] Figure 1 This is a flowchart illustrating a method for locating a faulty component according to an embodiment of the present invention.

[0050] Figure 2 This is a flowchart illustrating a step for determining the first logical position of a faulty component in a string based on an infrared image, as provided in an embodiment of the present invention.

[0051] Figure 3 A flowchart illustrating a step for obtaining a corrected image according to an embodiment of the present invention;

[0052] Figure 4 This is a flowchart illustrating the steps for obtaining the component results of separating each group string in a corrected image, as provided in an embodiment of the present invention.

[0053] Figure 5This is a flowchart illustrating a step for determining the fault location of a faulty component in an infrared image based on an infrared image, as provided in an embodiment of the present invention.

[0054] Figure 6 A flowchart illustrating a step for determining the second logical position of a faulty component in a panoramic image, provided by an embodiment of the present invention;

[0055] Figure 7 A flowchart illustrating the steps for obtaining the precise logical location of a faulty component in a panoramic image, as provided in an embodiment of the present invention.

[0056] Figure 8 This is a schematic diagram of a fault component location device provided in an embodiment of the present invention. Detailed Implementation

[0057] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0058] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0059] To illustrate the technical effects of the embodiments of the present invention, the following lists several abnormal scenarios in which inaccurate positioning occurs:

[0060] 1) Due to the overlap of infrared images, fault points can be detected in multiple images. Ideally (i.e., the 3D point cloud quality is the same in multiple detected infrared images), the fault points detected in multiple infrared images correspond to the same component. By fusing the four corner coordinates of the component segmentation to remove duplicates, a unique fault point can be obtained. However, in a non-ideal state (i.e., the quality of the reconstructed point cloud varies in different images of the same fault point), when multiple images corresponding to the same fault point are located on the faulty component, there will be an offset phenomenon, that is, the fault point will be repeatedly located. This will lead to the same fault cluster appearing repeatedly in the panoramic image when using existing technology.

[0061] 2) Especially in mountain power plants, there is a certain elevation difference between photovoltaic modules, which will cause the modules in the panoramic map to be seriously tilted. As a result, when the faulty module is merged into the module image in the panoramic map, the module will be offset by one grid to the adjacent module when using four-corner coordinate matching.

[0062] 3) If the quality and quantity of the point cloud are insufficient, when the two-dimensional pixels (marked with red dots) mapped from the 3D point cloud reconstructed from the infrared image are plotted onto the infrared image, there will be no reconstructed 3D points on the surface of the photovoltaic string where the faulty string is located, resulting in the inability to locate the fault. In addition, insufficient quality of the reconstructed point cloud will also cause location offset.

[0063] 4) In water-based photovoltaic power plants, there are no markers on the water surface, so it is impossible to perform feature matching between components in infrared images and components in panoramic images, resulting in a very low accuracy in associating faulty components in water-based photovoltaic power plants with panoramic images.

[0064] In addition to the aforementioned issues that may cause positioning deviations, factors such as varying skill levels among drone pilots and different camera models can also affect image quality, leading to shifts in component positions within the panoramic image.

[0065] Figure 1 This is a flowchart illustrating a fault component location method provided in an embodiment of the present invention. This embodiment is applicable to situations requiring precise location of faulty components. The method can be executed by a fault component location device, which can be implemented in hardware and / or software. The method specifically includes the following steps:

[0066] S110. Acquire infrared and panoramic images of the photovoltaic string.

[0067] Among these methods, drones can be used to inspect photovoltaic (PV) strings. Drones equipped with infrared cameras can capture infrared images of the PV strings in the power plant. Drones can also be equipped with visible light cameras to capture multiple visible light images of the PV strings. Furthermore, these multiple visible light images can be stitched together using DJI Terra software to create a panoramic image. This panoramic image can be a panoramic map, containing the latitude and longitude information of each pixel, which facilitates subsequent correlation with the location of faulty components.

[0068] S120. Based on the infrared image, determine the first logical position of the faulty component in the string and the fault location of the faulty component in the infrared image.

[0069] Image processing of infrared images allows for the marking of fault sequences and their constituent components within each sequence. For instance, in fault detection, fault detection boxes can be marked on the infrared image, and the coordinates of any pixel within the box can be used as the pixel position of the fault component in the infrared image. In summary, based on the marked fault components and their corresponding pixel positions within each fault sequence, the specific position of each fault component within its sequence can be determined, i.e., its first logical position (e.g., the fifth position in the first sequence, denoted as 1-5). Furthermore, infrared images can be reconstructed using 3D reconstruction algorithms to locate fault sequences, thereby obtaining the pixel positions of the fault components, which correspond to 3D coordinates.

[0070] S130. Based on the fault location and panoramic image, determine the second logical position of the faulty component in the panoramic image.

[0071] This process involves segmenting and numbering the strings in the panoramic image and the components included in each string. A mapping relationship is established between fault location and the components included in each string in the panoramic image, determining the second logical position of the faulty component in the panoramic image.

[0072] It is important to note that the fault location of faulty components in infrared images using 3D reconstruction algorithms depends on the quality and quantity of the reconstructed 3D point cloud. In practical applications, inaccurate location may occur in some abnormal scenarios, resulting in inaccurate second logical positions of faulty components in the panoramic image. Therefore, this project will subsequently correct the fault location of faulty components in the infrared images obtained in this step to improve the accuracy of faulty component location.

[0073] S140. Based on the first logical position and the second logical position, correct the second logical position of the faulty component in the panoramic image to obtain the precise logical location of the faulty component in the panoramic image.

[0074] Specifically, the second logical position represents the coordinates of the faulty component in the panoramic image, and its relative position within its respective string can be determined through this second logical position. The first logical position represents the relative position of the faulty component within its respective string. Therefore, the accuracy of the second logical position can be confirmed by comparing it with the first logical position, and if there is a deviation in the second logical position, it can be corrected based on the first logical position, thereby obtaining the precise logical location of the faulty component in the panoramic image.

[0075] This invention, in its embodiments, determines the precise first logical position of a faulty component within a string based on an infrared image; it then determines the fault location of the faulty component within the infrared image, and finally, based on the fault location and a panoramic image, determines the second logical position of the faulty component within the panoramic image. Since the determination of the second logical position of the faulty component in the panoramic image is affected by the quality and quantity of the 3D point cloud reconstructed from the infrared image, a positioning offset may occur. Therefore, the precise first logical position can be used to correct the second logical position of the faulty component in the panoramic image, thereby obtaining the precise logical location of the faulty component within the panoramic image.

[0076] For example, Figure 2 This is a flowchart illustrating a step for determining the first logical position of a faulty component in a string based on an infrared image, according to an embodiment of the present invention. Based on the above embodiment, the step for determining the first logical position of a faulty component in a string based on an infrared image is further explained in detail:

[0077] S210. Perform distortion correction on the infrared image to obtain a corrected image.

[0078] In this process, the infrared image can be reconstructed using a 3D reconstruction algorithm to obtain the intrinsic parameters of the infrared camera at the time of image capture and the reconstructed distortion parameter coefficients. The corrected image can be obtained by inputting the infrared image and the above two parameters into the distortion correction function.

[0079] S220. Perform segmentation processing on the corrected image to obtain the component results of each group string separation in the corrected image.

[0080] Image processing of the calibration image yields string segmentation masks and component segmentation masks. Using the string segmentation mask to segment the infrared image provides string-segmented images separating each string from the calibration image. The component segmentation mask is then used to segment these string-segmented images, resulting in the separated components. Each component is identified by its number relative to its corresponding string.

[0081] S230. Perform fault detection on the infrared image and determine the pixel coordinates of the faulty component.

[0082] Among them, a fault detection algorithm can be used to detect faults in the infrared image and obtain the fault detection result. The fault detection result can be a fault detection box marked in the infrared image. Any pixel coordinates within the fault detection box can be taken as the pixel position of the fault component in the infrared image.

[0083] S240. Based on the pixel coordinates of the faulty component and the component results of each string separation in the correction image, determine the first logical position of the faulty component in the string.

[0084] In this process, the pixel coordinates of the faulty component are associated with the component results separated from each group in the corrected image, thereby obtaining the first logical position of the faulty component in the group, that is, the relative position of the faulty component in its group.

[0085] In summary, the accurate position of the faulty component relative to the string can be obtained using the above method. Therefore, in subsequent steps, the accurate first logical position can be used to correct the second logical position of the faulty component in the panoramic image, thereby obtaining the accurate logical location of the faulty component in the panoramic image.

[0086] For example, Figure 3 This is a flowchart illustrating a step for obtaining a corrected image according to an embodiment of the present invention. Based on the above embodiment, the steps for obtaining the corrected image are further described in detail:

[0087] S310. Perform three-dimensional reconstruction on the infrared image to obtain the intrinsic parameters of the camera that captured the infrared image and the distortion parameter coefficients after reconstruction.

[0088] In practical applications, the distortion parameter coefficient is also called the distortion coefficient.

[0089] S320. Based on the distortion parameter coefficients, the intrinsic parameters of the infrared image camera, and the infrared image, the infrared image is corrected to obtain the corrected image.

[0090] The process involves calling a distortion correction function, inputting the distortion parameter coefficients, the intrinsic parameters of the infrared camera, and the infrared image into the distortion correction function to obtain the corrected image. For example, in OpenCV, the distortion correction function `cv2.undistort(img, cameraMatrix, distCoeffs)`, where `img` represents the input infrared image, `cameraMatrix` represents the intrinsic parameters of the infrared camera corresponding to the reconstructed image, and `distCoeffs` represents the reconstructed distortion parameter coefficients. The purpose of infrared image distortion correction is to avoid the string distortion caused by the captured image itself, which could affect the subsequent component positioning.

[0091] For example, Figure 4 This is a flowchart illustrating the steps for obtaining the component results of separating each group string in a corrected image according to an embodiment of the present invention. Based on the above embodiment, the steps for obtaining the component results of separating each group string in a corrected image are further described in detail:

[0092] S410. Perform string segmentation on each string in the corrected image to obtain the first image data of each string.

[0093] Specifically, image processing is performed on the corrected image (e.g., processing the corrected image using a pre-built neural network model) to obtain a string mask. Each string in the corrected image is labeled with the string mask to obtain the first image data for each string.

[0094] S420. Perform self-correction on the first image data of each group to obtain corrected group image data.

[0095] When the first image data of each string exhibits a tilt angle in the infrared image, self-correction is required to obtain corrected string image data. Specifically, this can be achieved by calculating the tilt angle of each string containing the detected faulty component and its corresponding mask, and then correcting the string to a horizontal state based on this tilt angle. In practical applications, edge fitting methods can be used to obtain the straight line of the string's edge, and the angle between the straight line and the horizontal line can be corrected to zero degrees.

[0096] S430. Perform component segmentation and padding on each correction group image data to obtain the component results of each correction group image data separation.

[0097] The component results include the component's number relative to its string. Component segmentation and padding are performed on the image data of each correction string to obtain the component results of each correction string image data separation, specifically:

[0098] Component segmentation is performed on each correction group of image data.

[0099] Specifically, each corrected image data set undergoes sequential processing including gradient calculation, erosion, Gaussian blurring, adaptive threshold segmentation, and edge detection.

[0100] Populate the components that are not separated to obtain the populated component data.

[0101] Specifically, for the separated components in a single string, the unseparated components are filled with the image data of the filled components.

[0102] Define the beginning and end of each string based on the data of the filling components and the first image data of each string.

[0103] The missing parts of the image in the sequence can be identified by the fill data. Therefore, the end of the first image data of each sequence that is furthest from the fill data can be defined as the beginning of the sequence.

[0104] In addition, the distance from the image boundary to the string containing the faulty component can be calculated based on the segmented component mask, thereby defining the beginning and end of the string number and assigning a number to the faulty component.

[0105] Each component is numbered according to the beginning and end of each string, thus obtaining the number of each component relative to its string.

[0106] For example, the component closest to the beginning of the string is component number 1, followed by component number 2, and so on.

[0107] Figure 5 This is a flowchart illustrating a step for determining the fault location of a faulty component in an infrared image based on an infrared image, according to an embodiment of the present invention. Based on the above embodiment, the step for determining the fault location of a faulty component in an infrared image is further described in detail:

[0108] S510. Perform three-dimensional reconstruction on the infrared image to obtain the point cloud data corresponding to the infrared image.

[0109] S520. Determine the fault location of the faulty component in each point cloud data.

[0110] Specifically, Structure-of-Motion (SMF) technology can be used to perform 3D reconstruction of infrared images, obtaining infrared image point cloud data, the pose and intrinsic parameters of the infrared camera. The point cloud data corresponding to the faulty component can then be identified from the point cloud data. This point cloud data can be 3D point cloud data corresponding to the pixels of the faulty component. Based on the intrinsic parameters, pose, pixel positions, and point cloud data of the infrared camera, the fault location of the faulty component can be determined.

[0111] Figure 6 This is a flowchart illustrating a step for determining the second logical position of a faulty component in a panoramic image, provided by an embodiment of the present invention. Based on the above embodiment, the step for determining the second logical position of the faulty component in the panoramic image is further explained as follows:

[0112] S610. Process the panoramic image to obtain a string segmentation mask.

[0113] For example, a string segmentation mask can be obtained by performing image processing on a panoramic image using a preset neural network model.

[0114] S620. The second image data of the sequence in the panoramic image is marked by the sequence segmentation mask.

[0115] S630. Perform segmentation processing on the second image data to obtain the component segmentation results for each group string.

[0116] For example, by segmenting the second image data using a preset neural network model, a component segmentation mask can be obtained. The component segmentation mask can be used to segment the second image data, thereby obtaining the component segmentation result of each string, that is, the number of each component in its string.

[0117] S640. Based on the fault location and the component segmentation results of each group, determine the second logical position of the faulty component in the panoramic image.

[0118] Specifically, the target pixel in the panoramic image is found to have the same coordinates as the fault location. The component where the target pixel is located is taken as the fault component in the panoramic image, and the position of the fault component in the panoramic image is taken as the second logical position.

[0119] Figure 7 This is a flowchart illustrating the steps for obtaining the precise logical location of a faulty component in a panoramic image, as provided in an embodiment of the present invention. Figure 7 As shown, the steps for obtaining the precise logical location of the faulty component in the panoramic image are further explained:

[0120] S710. Determine the position of the second logical position relative to the group string, and obtain the third logical position.

[0121] The third logical position refers to the relative position of the component marked in the panoramic image relative to its string, as determined by the second logical position.

[0122] S720. Determine if the third logical position is the same as the first logical position; if they are the same, execute S730; if they are not the same, execute S740.

[0123] If the third logical position is different from the first logical position, it indicates that the positioning of the second logical position has shifted; if the third logical position is the same as the first logical position, it indicates that the positioning of the second logical position is accurate.

[0124] S730, the second logical position is the precise logical location of the faulty component in the panoramic image;

[0125] S740. Adjust the third logical position according to the first logical position, and determine the precise logical location of the faulty component in the panoramic image according to the third logical position.

[0126] Figure 8 This is a schematic diagram of a fault component location device provided in an embodiment of the present invention, as shown below. Figure 8 As shown, the fault component location device includes:

[0127] The acquisition module 810 is used to acquire infrared images and panoramic images of the photovoltaic string;

[0128] The first positioning module 820 is used to determine the first logical position of the faulty component in the string and the fault location of the faulty component in the infrared image based on the infrared image.

[0129] The second positioning module 830 is used to determine the second logical position of the faulty component in the panoramic image based on the fault location and the panoramic image.

[0130] The precise logical positioning determination module 840 is used to correct the second logical position of the faulty component in the panoramic image based on the first logical position and the second logical position, so as to obtain the precise logical position of the faulty component in the panoramic image.

[0131] In this embodiment of the invention, the first positioning module 820 determines the precise first logical position of the faulty component in the string based on the infrared image, thus determining the fault location of the faulty component in the infrared image. The second positioning module 830 determines the second logical position of the faulty component in the panoramic image based on the fault location and the panoramic image. Since the determination of the second logical position of the faulty component in the panoramic image is affected by the quality and quantity of the 3D point cloud reconstructed from the infrared image, there may be a positioning offset. Therefore, the precise logical positioning determination module 840 can use the precise first logical position to correct the second logical position of the faulty component in the panoramic image, thereby obtaining the precise logical location of the faulty component in the panoramic image.

[0132] The first positioning module includes:

[0133] The correction unit is used to correct the distortion of the infrared image to obtain a corrected image;

[0134] The segmentation unit is used to segment the corrected image to obtain the component results of separating each group string in the corrected image;

[0135] The pixel determination unit is used to perform fault detection on infrared images and determine the pixel coordinates of faulty components.

[0136] The first logical position determination unit is used to determine the first logical position of the faulty component in the string based on the pixel coordinates of the faulty component and the component results of each string separation in the corrected image.

[0137] The correction unit is specifically used for:

[0138] Three-dimensional reconstruction of infrared images is performed to obtain the camera intrinsic parameters of the infrared image capture and the distortion parameter coefficients after reconstruction;

[0139] The corrected image is obtained by correcting the infrared image based on the distortion parameter coefficients, the intrinsic parameters of the infrared image camera, and the infrared image itself.

[0140] The segmentation unit includes: a first image data acquisition subunit, used to segment each group of strings in the corrected image to obtain the first image data of each group of strings;

[0141] The correction string image data acquisition subunit is used to perform self-correction on the first image data of each string to obtain the correction string image data.

[0142] The component result acquisition subunit is used to perform component segmentation and filling on each correction group image data to obtain the component results of each correction group image data separation.

[0143] Optionally, the component results include the number of each component relative to its string;

[0144] The component result acquisition subunit is specifically used for: performing component segmentation on each corrected string image data;

[0145] Populate the components that are not separated to obtain the populated component data;

[0146] Define the beginning and end of each string based on the data of the filling components and the first image data of each string;

[0147] Each component is numbered according to the beginning and end of each string, thus obtaining the number of each component relative to its string.

[0148] The first positioning module also includes:

[0149] The 3D modeling unit is used to reconstruct the infrared image in 3D and obtain the point cloud data corresponding to the infrared image.

[0150] The fault location unit is used to determine the fault location of the faulty component in each point cloud data.

[0151] The second positioning module is specifically used for:

[0152] The panoramic image is processed to obtain a string segmentation mask;

[0153] The second image data of the sequence in the panoramic image is marked by the sequence segmentation mask;

[0154] The second image data is segmented to obtain the component segmentation results for each string;

[0155] Based on the fault location and the component segmentation results of each group, the second logical position of the faulty component in the panoramic image is determined.

[0156] The precise logical positioning and determination module is specifically used to: determine the position of the second logical position relative to the string, and obtain the third logical position;

[0157] Determine whether the third logical position is the same as the first logical position;

[0158] If they are the same, then the second logical position is the precise logical location of the faulty component in the panoramic image;

[0159] If they are different, the third logical position is adjusted according to the first logical position, and the precise logical location of the faulty component in the panoramic image is determined according to the third logical position. This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method provided in any embodiment of this invention.

[0160] The computer storage medium of this invention can be any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. For example, a computer-readable storage medium can be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. Computer-readable storage media include (a non-exhaustive list): electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), electrically erasable, programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this invention, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0161] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, the data signals carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device.

[0162] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wire, optical fiber, radio frequency (RF), or any suitable combination thereof.

[0163] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination of programming languages, including object-oriented programming languages ​​such as Java, Smalltalk, C++, Ruby, and Go, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network (including a Local Area Network (LAN) or a Wide Area Network (WAN)), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0164] Those skilled in the art will understand that the term user terminal encompasses any suitable type of wireless user equipment, such as mobile phones, portable data processing devices, portable web browsers, or vehicle-mounted mobile stations.

[0165] Generally, various embodiments of the present invention can be implemented in hardware or dedicated circuitry, software, logic, or any combination thereof. For example, some aspects can be implemented in hardware, while others can be implemented in firmware or software that can be executed by a controller, microprocessor, or other computing device, although the invention is not limited thereto.

[0166] Embodiments of the present invention can be implemented by executing computer program instructions through the data processor of a mobile device, for example, in a processor entity, or through hardware, or through a combination of software and hardware. The computer program instructions can be assembly instructions, Instruction Set Architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages.

[0167] Any block diagram of logical flow in the accompanying drawings of this invention may represent program steps, or may represent interconnected logic circuits, modules, and functions, or may represent a combination of program steps and logic circuits, modules, and functions. The computer program may be stored in memory. The memory may be of any type suitable to the local technical environment and may be implemented using any suitable data storage technology, such as, but not limited to, read-only memory (ROM), random access memory (RAM), optical storage devices and systems (Digital Multifunction Discs, DVDs, or CDs), etc. Computer-readable media may include non-transitory storage media. The data processor may be of any type suitable to the local technical environment, such as, but not limited to, general-purpose computers, special-purpose computers, microprocessors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), and processors based on multi-core processor architectures.

[0168] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0169] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for locating a faulty component, characterized in that, include: Acquire infrared and panoramic images of the photovoltaic string; Based on the infrared image, the first logical position of the faulty component in the string and the fault location of the faulty component in the infrared image are determined; wherein, based on the faulty component marked in each faulty string in the infrared image and the pixel position marked by the faulty component in the infrared image, the first logical position of each faulty component in the string is obtained; the faulty string in the infrared image is located by a three-dimensional reconstruction algorithm to obtain the pixel position marked by the faulty component in the infrared image. Based on the fault location and the panoramic image, the second logical position of the faulty component in the panoramic image is determined; wherein, the strings in the panoramic image and the components included in each string are segmented and numbered, a mapping relationship between the fault location and the components included in each string in the panoramic image is established, and the second logical position of the faulty component in the panoramic image is determined. Based on the first logical position and the second logical position, the second logical position of the faulty component in the panoramic image is corrected to obtain the precise logical location of the faulty component in the panoramic image.

2. The fault component location method according to claim 1, characterized in that, Based on the infrared image, determining the first logical location of the faulty component in the string includes: The infrared image is subjected to distortion correction to obtain a corrected image; The corrected image is segmented to obtain the component results of each group string separation in the corrected image; Fault detection is performed on the infrared image to determine the pixel coordinates of the faulty component; Based on the pixel coordinates of the faulty component and the component results separated from each string in the corrected image, the first logical position of the faulty component in the string is determined.

3. The fault component location method according to claim 2, characterized in that, The steps to obtain the corrected image include: The infrared image is reconstructed in three dimensions to obtain the intrinsic parameters of the camera that captured the infrared image and the reconstructed distortion parameter coefficients; The infrared image is corrected based on the distortion parameter coefficients, the intrinsic parameters of the infrared image camera, and the infrared image to obtain the corrected image.

4. The fault component location method according to claim 2, characterized in that, The step of obtaining the component results of each group string separation in the corrected image includes: The corrected image is segmented into its constituent strings to obtain the first image data of each string. Self-correction is performed on the first image data of each group to obtain corrected group image data; Each of the correction string image data is segmented and filled into components to obtain the component results of each correction string image data separation.

5. The fault component location method according to claim 4, characterized in that, The component results include the number of each component relative to its string; For each of the corrected string image data, component segmentation and padding are performed to obtain the component results of each corrected string image data separation, including: Component segmentation is performed on each of the corrected string image data; The unseparated components are populated to obtain populated component data; Based on the filling component data and the first image data of each string, the beginning and end of each string are defined; Each component is numbered according to the beginning and end of each string, thus obtaining the number of each component relative to its string.

6. The fault component location method according to claim 1, characterized in that, The step of determining the fault location of the faulty component in the infrared image based on the infrared image includes: The infrared image is reconstructed in three dimensions to obtain the point cloud data corresponding to the infrared image. The fault location of the faulty component is determined from each of the point cloud data.

7. The fault component location method according to claim 1, characterized in that, The step of determining the second logical location of the faulty component in the panoramic image includes: The panoramic image is processed to obtain a string segmentation mask; The second image data of the sequence in the panoramic image is marked according to the sequence segmentation mask; The second image data is segmented to obtain the component segmentation results for each string; Based on the fault location and the component segmentation results of each group, the second logical position of the faulty component in the panoramic image is determined.

8. The fault component location method according to claim 1, characterized in that, The step of obtaining the precise logical location of the faulty component in the panoramic image includes: Determine the position of the second logical position relative to the string to obtain the third logical position; Determine whether the third logical position is the same as the first logical position; If they are the same, then the second logical position is the precise logical location of the faulty component in the panoramic image; If they are different, the third logical position is adjusted according to the first logical position, and the precise logical location of the faulty component in the panoramic image is determined according to the third logical position.

9. A fault component location device, characterized in that, include: The acquisition module is used to acquire infrared and panoramic images of the photovoltaic strings; The first positioning module is used to determine the first logical position of the faulty component in the string and the fault location of the faulty component in the infrared image based on the infrared image; wherein, the first logical position of each faulty component in the string is obtained based on the faulty component marked in each faulty string in the infrared image and the pixel position marked by the faulty component in the infrared image; and the faulty string in the infrared image is located by a three-dimensional reconstruction algorithm to obtain the pixel position marked by the faulty component in the infrared image. The second positioning module is used to determine the second logical position of the faulty component in the panoramic image based on the fault location and the panoramic image; wherein, the strings in the panoramic image and the components included in each string are segmented and numbered, a mapping relationship is established between the fault location and the components included in each string in the panoramic image, and the second logical position of the faulty component in the panoramic image is determined. A precise logical positioning determination module is used to correct the second logical position of the faulty component in the panoramic image based on the first logical position and the second logical position, so as to obtain the precise logical position of the faulty component in the panoramic image.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the fault component location method as described in any one of claims 1-8.

Images