Fault locating method, fault locating device and electronic equipment for photovoltaic power station
By combining image information and 3D point cloud data acquired by UAVs, the camera pose is corrected, and the 3D coordinates of photovoltaic power station faults are determined. This solves the problem of low positioning accuracy in existing technologies and achieves efficient and accurate fault detection and differentiation between new and old faults.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SUNGROW (SHANGHAI) CO LTD
- Filing Date
- 2022-12-13
- Publication Date
- 2026-06-12
AI Technical Summary
Existing technologies for fault detection and location in photovoltaic power plants have low accuracy, rely heavily on the flight stability of drones, and face significant challenges in image registration, thus failing to effectively improve detection efficiency and accuracy.
By acquiring images of photovoltaic power plants to be inspected using drones, and combining these images with camera pose, pixel coordinates of the current fault, target inspection images, and 3D point cloud information, the 3D coordinates of the fault are determined. Feature matching and 3D reconstruction techniques are then used to correct the camera pose and improve positioning accuracy.
It achieves high-efficiency and high-precision fault detection in photovoltaic power plants, enabling real-time detection and differentiation between new and old faults, reducing error accumulation, and improving the overall efficiency of drone inspections.
Smart Images
Figure CN116030127B_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the field of image processing technology, and in particular relates to a fault location method, fault location device and electronic equipment for a photovoltaic power station. Background Technology
[0002] With the development of drone technology, the operation and maintenance of photovoltaic power plants has gradually shifted from manual inspections to intelligent detection of photovoltaic module faults by using drones equipped with visible light or infrared cameras to collect photos. This method greatly improves the efficiency of photovoltaic power plant operation and maintenance and saves a lot of human resources.
[0003] In related technologies, image registration relies on visible light and infrared photographs, which is difficult to achieve in practice and results in low positioning accuracy. Alternatively, the positioning accuracy of fault detection is highly dependent on the flight stability of the UAV, which is difficult to achieve in practice. Or, relying solely on image algorithms does not take into account the high pixel repetition rate in photovoltaic scenarios, resulting in low positioning accuracy. Summary of the Invention
[0004] This application aims to address at least one of the technical problems existing in the prior art. To this end, this application proposes a fault location method, fault location device, and electronic equipment for photovoltaic power plants to improve the fault detection efficiency and ensure accuracy.
[0005] Firstly, this application provides a fault location method for a photovoltaic power plant, the method comprising:
[0006] Images of photovoltaic power plants to be inspected are obtained using drones;
[0007] Based on the positioning information of the image to be detected, a target inspection image is determined in the inspection image of the photovoltaic power station, and the inspection image is used to determine the three-dimensional point cloud information of the photovoltaic power station.
[0008] Fault identification is performed on the image to be detected to obtain the pixel coordinates of the current fault;
[0009] Based on the camera pose of the image to be detected, the pixel coordinates of the current fault, the target inspection image, and the three-dimensional point cloud information of the photovoltaic power station, the three-dimensional coordinates of the current fault are determined.
[0010] According to the fault location method for photovoltaic power plants in this application, the three-dimensional coordinates of the current fault are determined by combining the camera pose and pixel coordinate information of the current fault carried in the real-time image to be detected with the target inspection image in the database and the three-dimensional point cloud information of the photovoltaic power plant. This method can improve the fault detection efficiency of photovoltaic power plants and ensure accuracy.
[0011] According to one embodiment of this application, determining the three-dimensional coordinates of the current fault based on the camera pose of the image to be detected, the pixel coordinates of the current fault, the target inspection image, and the three-dimensional point cloud information of the photovoltaic power station includes:
[0012] The image to be detected is segmented based on the pixel coordinates of the current fault to obtain the string region where the current fault is located;
[0013] The three-dimensional coordinates of the current fault are determined based on the camera pose of the image to be detected, the string region, the pixel coordinates of the current fault, the target inspection image, and the three-dimensional point cloud information of the photovoltaic power station.
[0014] According to one embodiment of this application, determining the three-dimensional coordinates of the current fault based on the corrected camera pose, the string region, the pixel coordinates of the current fault, the target inspection image, and the three-dimensional point cloud information of the photovoltaic power station includes:
[0015] Feature extraction is performed on the string region to obtain multiple region feature points;
[0016] The feature points of the multiple regions are matched with the feature points of the target inspection image, and combined with the three-dimensional point cloud information of the photovoltaic power station to obtain the three-dimensional coordinates of the multiple region feature points.
[0017] Based on the three-dimensional coordinates of the multiple regional feature points, the plane equation of the string region is determined;
[0018] The collinearity equation is determined by the corrected camera pose and the pixel coordinates of the current fault.
[0019] Based on the collinearity equation and the plane equation, the three-dimensional coordinates of the current fault are determined.
[0020] According to one embodiment of this application, after determining the target inspection image from the inspection images of the photovoltaic power station, the method further includes:
[0021] Based on the target inspection image, the camera pose of the image to be detected is corrected.
[0022] According to one embodiment of this application, correcting the camera pose of the image to be detected based on the target inspection image includes:
[0023] Feature matching is performed between the image to be detected and the target inspection image, and the three-dimensional coordinates of the feature points in the image to be detected are determined by combining the three-dimensional point cloud information of the photovoltaic power station.
[0024] The camera pose of the image to be detected is corrected based on the pixel coordinates and three-dimensional coordinates of multiple feature points in the image to be detected.
[0025] According to one embodiment of this application, the step of performing feature matching between the image to be detected and the target inspection image, and combining the three-dimensional point cloud information of the photovoltaic power station to determine the three-dimensional coordinates of feature points in the image to be detected, includes:
[0026] Determine the points with the same name as the feature points in the image to be detected from the target inspection image;
[0027] Based on the three-dimensional point cloud information of the photovoltaic power station, determine the three-dimensional coordinates corresponding to the corresponding points;
[0028] The three-dimensional coordinates corresponding to the points of the same name are used as the three-dimensional coordinates of the feature points of the image to be detected.
[0029] According to one embodiment of this application, the inspection image is also used to determine the three-dimensional coordinates of the initial fault of the photovoltaic power station, the pixel coordinates of the initial fault, and the type information of the initial fault;
[0030] After determining the three-dimensional coordinates of the current fault, the method further includes: determining whether the current fault is a new fault based on the three-dimensional coordinates of the current fault and the three-dimensional coordinates of the initial fault.
[0031] By using the three-dimensional coordinates of the current fault, existing fault information in the power plant database can be extracted to determine whether the current fault is a new fault, and some non-permanent faults, such as dust accumulation, can also be ruled out.
[0032] According to one embodiment of this application, before acquiring the image of the photovoltaic power station to be inspected via a drone, the method further includes:
[0033] The inspection images of the photovoltaic power station are processed by three-dimensional reconstruction technology to establish three-dimensional point cloud information of the photovoltaic power station. The three-dimensional point cloud information includes: three-dimensional point cloud coordinates, pixel coordinates of each point in each inspection image, and descriptive information of the point.
[0034] The inspection images are used to identify faults, and the location information and type information of the initial fault are obtained.
[0035] Based on the three-dimensional point cloud information and the location information of the initial fault, the three-dimensional coordinates and pixel coordinates of the initial fault are obtained.
[0036] Secondly, this application provides a fault location device for a photovoltaic power station, the device comprising:
[0037] The first receiving module is used to acquire images of the photovoltaic power station to be inspected via a drone;
[0038] The first processing module is used to determine the target inspection image in the inspection image of the photovoltaic power station based on the positioning information of the image to be detected. The inspection image is used to determine the three-dimensional point cloud information of the photovoltaic power station.
[0039] The second processing module is used to perform fault identification on the image to be detected and obtain the pixel coordinates of the current fault;
[0040] The third processing module is used to determine the three-dimensional coordinates of the current fault based on the camera pose of the image to be detected, the pixel coordinates of the current fault, the target inspection image, and the three-dimensional point cloud information of the photovoltaic power station.
[0041] The fault location device for photovoltaic power plants according to this application determines the three-dimensional coordinates of the current fault by combining the camera pose and pixel coordinate information of the current fault carried in the real-time image to be detected with the target inspection image in the database and the three-dimensional point cloud information of the photovoltaic power plant. This can improve the fault detection efficiency of photovoltaic power plants and ensure accuracy.
[0042] Thirdly, this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the fault location method for a photovoltaic power station as described in the first aspect above.
[0043] Fourthly, this application provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the fault location method for a photovoltaic power station as described in the first aspect above.
[0044] Fifthly, this application provides a chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the fault location method for photovoltaic power plants as described in the first aspect.
[0045] Sixthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the fault location method for a photovoltaic power station as described in the first aspect above.
[0046] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description
[0047] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:
[0048] Figure 1 This is one of the flowcharts illustrating the fault location method for a photovoltaic power station provided in the embodiments of this application;
[0049] Figure 2 This is a second schematic flowchart of the fault location method for a photovoltaic power station provided in the embodiments of this application;
[0050] Figure 3 This is a schematic diagram of the structure of the fault location device for a photovoltaic power station provided in the embodiments of this application;
[0051] Figure 4 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation
[0052] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.
[0053] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.
[0054] The following description, in conjunction with the accompanying drawings, details the fault location method, fault location device, electronic equipment, and readable storage medium for photovoltaic power plants provided in this application, through specific embodiments and application scenarios.
[0055] Among them, the fault location method of photovoltaic power station can be applied to the terminal, which can be executed by the hardware or software in the terminal.
[0056] The fault location method for photovoltaic power plants provided in this application embodiment can be executed by an electronic device or a functional module or functional entity in an electronic device that can implement the fault location method for photovoltaic power plants. The electronic devices mentioned in this application embodiment include, but are not limited to, mobile phones, tablets, computers, cameras and wearable devices. The fault location method for photovoltaic power plants provided in this application embodiment will be described below using an electronic device as the execution subject as an example.
[0057] A photovoltaic power station consists of multiple photovoltaic panels, each with multiple photovoltaic modules (one cell of a photovoltaic panel is one photovoltaic module), and multiple photovoltaic modules are connected to form a photovoltaic string.
[0058] The fault location method for photovoltaic power plants can detect the three-dimensional coordinates of faulty photovoltaic modules to facilitate subsequent maintenance.
[0059] like Figure 1 As shown, the fault location method for this photovoltaic power station includes steps 110, 120, 130 and 140.
[0060] Step 110: Obtain the image of the photovoltaic power station to be inspected using a drone;
[0061] Drones can be equipped with cameras, and the type of image to be detected varies depending on the type of camera. For example, when a drone is equipped with a visible light camera, the image to be detected is a visible light image; when a drone is equipped with an infrared camera, the image to be detected is an infrared image.
[0062] The image to be detected in this step can be an image captured in real time by the drone's camera. When the drone flies over the photovoltaic power station, images of the power station below can be captured by the camera on board the drone.
[0063] Drones are equipped with positioning modules, including but not limited to GPS modules and Beidou modules. When the camera on the drone takes a picture of the image to be detected, the image contains the drone's positioning information at the time of the picture.
[0064] Step 120: Based on the positioning information of the image to be detected, determine the target inspection image in the inspection image of the photovoltaic power station. The inspection image is used to determine the three-dimensional point cloud information of the photovoltaic power station.
[0065] After acquiring the image to be detected, the image is parsed to obtain the positioning information of the image to be detected, which includes the coordinates (x, y, z) in the image to be detected.
[0066] The inspection images are existing images pre-stored in the database. Similarly, these inspection images also contain positioning information. By matching the positioning information of the image to be detected with the positioning information of the inspection images in the database, the target inspection image can be found. The target inspection image is the image that is closest to the current image to be detected among multiple inspection images.
[0067] In practice, the closest inspection image can be selected as the target inspection image by calculating the distance between the coordinates (x, y, z) of the image to be inspected and the coordinates (x, y, z) of the inspection images in the database.
[0068] The following describes the inspection images and the 3D point cloud information of the photovoltaic power station.
[0069] Understandably, before implementing fault location methods for photovoltaic power plants, existing inspection images can be used to construct 3D point cloud information of the photovoltaic power plant. Existing inspection images can be obtained as follows: A drone is used to inspect and photograph the photovoltaic power plant, acquiring inspection images. Adjacent inspection images have an overlap rate. This overlap rate refers to the degree of image overlap maintained between adjacent images or adjacent flight paths when the drone takes photos along its flight path. The former is called forward overlap, and the latter is called lateral overlap. It is expressed as a percentage of the ratio of the length of the overlapping portion to the image frame length, such as a forward overlap of 60% and a lateral overlap of 40%.
[0070] In this way, inspection images covering the entire photovoltaic power station can be obtained. Through technologies such as 3D reconstruction, 3D point cloud information of the photovoltaic power station can be obtained. The 3D point cloud information includes: 3D point cloud coordinates (x, y, z), pixel coordinates (u, v) of each feature point in each inspection image, and descriptive information of the feature points.
[0071] It should be noted that each 3D point cloud coordinate can correspond to multiple pixel coordinates, each of which corresponds to its own inspection image, and each pixel coordinate has corresponding descriptive information.
[0072] Step 130: Perform fault identification on the image to be detected to obtain the pixel coordinates of the current fault;
[0073] After obtaining the image to be detected in step 110, fault identification can be performed on the image to be detected using image recognition technology to obtain the pixel coordinates of the current fault.
[0074] In practice, the above-mentioned fault identification can be accomplished through a fault identification model.
[0075] Step 130: Perform fault identification on the image to be detected and obtain the pixel coordinates of the current fault. This includes: inputting the image to be detected into the fault identification model and obtaining the pixel coordinates of the current fault output by the fault identification model. The fault identification model is trained using sample images as samples and pre-determined sample fault pixel coordinates corresponding to the sample images as sample labels.
[0076] Alternatively, the fault identification model can also output the fault type. Correspondingly, step 130, performing fault identification on the image to be detected and obtaining the pixel coordinates of the current fault, includes: inputting the image to be detected into the fault identification model to obtain the pixel coordinates and fault type of the current fault output by the fault identification model. The fault identification model is trained using sample images as samples and pre-determined sample fault pixel coordinates and fault types corresponding to the sample images as sample labels.
[0077] Step 140: Determine the three-dimensional coordinates of the current fault based on the camera pose of the image to be detected, the pixel coordinates of the current fault, the target inspection image, and the three-dimensional point cloud information of the photovoltaic power station.
[0078] In step 130, after determining the pixel coordinates of the current fault, it is also necessary to obtain the three-dimensional coordinates of the current fault.
[0079] In step 140, the camera pose of the image to be detected, the pixel coordinates of the current fault, the target inspection image, and the 3D point cloud information of the photovoltaic power station are combined. The camera pose of the image to be detected and the pixel coordinates of the current fault are information that can be easily determined in real time and quickly. The target inspection image and the 3D point cloud information of the photovoltaic power station are existing data in the database, which have high accuracy.
[0080] This method is equivalent to matching real-time data with existing data in the database to perform online fault detection and location, balancing real-time performance and accuracy.
[0081] According to the fault location method for photovoltaic power plants provided in the embodiments of this application, the three-dimensional coordinates of the current fault are determined by combining the camera pose and pixel coordinate information of the current fault carried in the real-time image to be detected with the target inspection image in the database and the three-dimensional point cloud information of the photovoltaic power plant. This can improve the fault detection efficiency of photovoltaic power plants and ensure accuracy.
[0082] In some embodiments, before step 110, acquiring the image of the photovoltaic power station to be inspected via a drone, the method may further include:
[0083] The inspection images of photovoltaic power stations are processed by 3D reconstruction technology to establish 3D point cloud information of the photovoltaic power stations. The 3D point cloud information includes: 3D point cloud coordinates, pixel coordinates of each feature point in each inspection image, and descriptive information of the feature points.
[0084] Fault identification is performed on the inspection images to obtain the location information and type information of the initial fault;
[0085] Based on the 3D point cloud information and the location information of the initial fault, the 3D coordinates and pixel coordinates of the initial fault are obtained.
[0086] It should be noted that there is overlap between the inspection images, such as 60% overlap in the forward direction and 40% overlap in the lateral direction. Feature points are extracted from the inspection images to obtain point cloud data.
[0087] The obtained 3D point cloud information includes: 3D point cloud coordinates (x, y, z), pixel coordinates (u, v) of each feature point in each inspection image, and descriptive information of the feature points.
[0088] It should be noted that, due to the overlap between inspection images, each 3D point cloud coordinate can correspond to multiple inspection images, and thus multiple pixel coordinates, with each feature point corresponding to descriptive information.
[0089] In the above method, it is equivalent to using existing inspection photos and obtaining the three-dimensional point cloud coordinates (spatial coordinates (x, y, z) of feature points), pixel coordinates (u, v) of feature points, and descriptive information of feature points for each photo through three-dimensional reconstruction technology (SFM, Structure From Motion) before formally conducting fault detection.
[0090] Next, fault identification is performed on the inspection images. In actual execution, the inspection images can be input into the fault identification model to obtain the initial fault location information and initial fault type information output by the fault identification model. The fault identification model is trained using sample images as samples and pre-determined sample fault location information and fault type corresponding to the sample images as sample labels.
[0091] The fault location information includes the initial pixel coordinates of the fault and the size of the rectangle. The initial pixel coordinates are the center coordinates of the rectangle of the faulty photovoltaic panel (small squares of the photovoltaic panel). The rectangle includes width and height.
[0092] This fault identification model can be used to identify faults in inspection images as well as in images to be inspected.
[0093] The same fault identification model is used for fault identification in inspection images and fault identification in images to be inspected. This results in a lower error rate when using the 3D point cloud coordinates obtained from the inspection images to determine the 3D coordinates of the fault points in the images to be inspected.
[0094] Based on the 3D point cloud information and the location information of the initial fault, the 3D coordinates (x, y, z) and pixel coordinates (u, v) of the initial fault are obtained.
[0095] The above results are stored in the database for later use.
[0096] In some embodiments, after determining the target inspection image from the inspection images of the photovoltaic power station in step 120, the method further includes: correcting the camera pose of the image to be detected based on the target inspection image.
[0097] It is understandable that the initial camera pose data of the image to be detected contains errors. By using the target inspection image to correct the camera pose, the corrected camera pose will be more accurate, and the coordinates of the fault point obtained by locating it using the corrected camera pose will be more precise. Specifically, the EPNP algorithm can be used to achieve this.
[0098] The camera pose used in step 140 can be a calibrated camera pose.
[0099] In some examples, the above steps, based on the target inspection image, correct the camera pose of the image to be detected, including:
[0100] Feature matching is performed between the image to be detected and the target inspection image, and the three-dimensional coordinates (x, y, z) of the feature points in the image to be detected are determined by combining the three-dimensional point cloud information of the photovoltaic power station.
[0101] The camera pose of the image to be detected is corrected based on the pixel coordinates (u, v) and the three-dimensional coordinates (x, y, z) of multiple feature points in the image to be detected.
[0102] When correcting the camera pose, the three-dimensional coordinates in the three-dimensional point cloud information are used. These three-dimensional coordinates are highly accurate, and when combined with the positioning method in step 140, they can reduce the accumulation of different types of errors.
[0103] In some examples, the above steps involve feature matching between the image to be detected and the target inspection image, and combining this with the 3D point cloud information of the photovoltaic power station to determine the 3D coordinates of feature points in the image to be detected, including:
[0104] Identify points in the target inspection image that correspond to feature points in the image to be inspected;
[0105] Based on the three-dimensional point cloud information of the photovoltaic power station, determine the three-dimensional coordinates of the corresponding points;
[0106] The three-dimensional coordinates corresponding to the same point are used as the three-dimensional coordinates of the feature points in the image to be detected.
[0107] The above steps involve feature matching between the image to be detected and the target inspection image, and combining this with the 3D point cloud information of the photovoltaic power station to determine the 3D coordinates of the first feature point in the image to be detected, including:
[0108] Feature extraction is performed on the image to be detected to obtain feature points and corresponding descriptive information;
[0109] Based on the above feature points and corresponding descriptive information, the corresponding points are determined in the target inspection image;
[0110] Based on the three-dimensional point cloud information of the photovoltaic power station, the three-dimensional coordinates corresponding to the above-mentioned points are determined.
[0111] The three-dimensional coordinates corresponding to the same point are used as the three-dimensional coordinates of the feature points in the image to be detected.
[0112] In other words, by using the positioning information of the image to be detected, and finding the image with the closest positioning information among the inspection images—the target inspection image—feature extraction is performed on the image to be detected to obtain descriptive information. By matching this information with the target inspection image, the three-dimensional coordinates of the feature points of the image to be detected can be obtained, thereby correcting the camera pose.
[0113] In some embodiments, step 140, determining the three-dimensional coordinates of the current fault based on the camera pose of the image to be detected, the pixel coordinates of the current fault, the target inspection image, and the three-dimensional point cloud information of the photovoltaic power station, includes:
[0114] The image to be detected is segmented based on the pixel coordinates of the current fault to obtain the string region where the current fault is located;
[0115] Based on the camera pose of the image to be detected, the string region mentioned above, the pixel coordinates of the current fault, the target inspection image, and the 3D point cloud information of the photovoltaic power station, the 3D coordinates of the current fault are determined.
[0116] It is understandable that a photovoltaic power station consists of multiple photovoltaic panels, multiple photovoltaic modules on each photovoltaic panel (one cell of a photovoltaic panel is one photovoltaic module), and multiple photovoltaic modules connected together to form a photovoltaic string.
[0117] The pixel coordinates of the current fault determined by image recognition can point to a certain component within the string. By using image segmentation, the string region can be segmented from the image to be detected, and the string region points to a specific photovoltaic panel.
[0118] By segmenting the string region and combining the camera pose and pixel coordinates of the current fault carried in the image to be detected, along with the target inspection image in the database and the 3D point cloud information of the photovoltaic power station, the 3D coordinates of the current faulty photovoltaic module can be jointly calculated. This can improve the fault detection efficiency of the photovoltaic power station and ensure accuracy.
[0119] In some embodiments, the above steps, based on the corrected camera pose, string region, pixel coordinates of the current fault, target inspection image, and 3D point cloud information of the photovoltaic power station, determine the 3D coordinates of the current fault, including:
[0120] Feature extraction is performed on the string region to obtain multiple region feature points;
[0121] By performing feature matching between multiple regional feature points and feature points in the target inspection image, and combining this with the 3D point cloud information of the photovoltaic power station, the 3D coordinates of multiple regional feature points are obtained.
[0122] Based on the three-dimensional coordinates of multiple regional feature points, the planar equation of the string region is determined;
[0123] The collinearity equation is determined by the corrected camera pose and the pixel coordinates of the current fault.
[0124] Based on collinearity equations and plane equations, the three-dimensional coordinates of the current fault are determined.
[0125] In practice, the location coordinates (x, y, z) of the image to be detected are matched with existing data in the database to find the closest target inspection image. A segmentation model is used to segment the image to be detected into clusters, obtaining all cluster regions (boundaries containing the clusters). When a fault is found in the image to be detected, features are extracted from the cluster region containing the current fault, obtaining the pixel coordinates and descriptive information of the region's feature points. Similarity matching is performed between multiple region feature points and feature points in the target inspection image to obtain multiple feature points with similar descriptive information in the target inspection image. Based on the three-dimensional coordinates (x, y, z) of these feature points, the plane equation of the cluster region is obtained by least squares fitting. Collinearity equations are determined based on the corrected camera pose and the pixel coordinates of the current fault. Solving the line and plane equations yields the three-dimensional coordinates of the current fault.
[0126] The above processing method can accurately transform the pixel coordinates of the current fault into three-dimensional coordinates, and the fault location method has high detection accuracy.
[0127] In some embodiments, the inspection images are also used to determine the three-dimensional coordinates of the initial fault of the photovoltaic power plant, the pixel coordinates of the initial fault, and the type information of the initial fault.
[0128] In actual execution, the inspection image can be input into the fault identification model to obtain the pixel coordinates of the initial fault and the type information of the initial fault output by the fault identification model. The fault identification model is trained by using the sample image as the sample and the pre-determined sample fault pixel coordinates and fault type corresponding to the sample image as the sample label.
[0129] After determining the three-dimensional coordinates of the current fault in step 140, the method further includes: determining whether the current fault is a new fault based on the three-dimensional coordinates of the current fault and the three-dimensional coordinates of the initial fault.
[0130] In other words, after identifying the three-dimensional coordinates of the current fault in the image to be detected, it is also necessary to determine whether the fault is an existing fault or a new fault. If it is an existing fault, there is no need to output a new maintenance request. If it is a new fault, a new maintenance request needs to be output for engineers or maintenance robots to inspect and repair.
[0131] By using the three-dimensional coordinates of the current fault, existing fault information in the power plant database can be extracted to determine whether the located fault is a new fault, and to rule out some non-permanent faults, such as dust accumulation.
[0132] In actual execution, the distance between the three-dimensional coordinates of the current fault and the three-dimensional coordinates of the initial fault can be calculated. If the distance is not greater than a preset threshold, it indicates that the fault is an old fault; if it exceeds the threshold, it is a new fault. The preset threshold can be calibrated based on the recognition accuracy.
[0133] The following is combined Figure 2 This paper describes one embodiment of the fault location method for photovoltaic power plants according to this application. The fault location method is illustrated using an unmanned aerial vehicle (UAV) equipped with an infrared camera as an example.
[0134] The fault location method for this photovoltaic power station includes steps 201-203 and steps 210-260.
[0135] Step 201: Acquire existing inspection infrared images;
[0136] The photovoltaic power station is inspected by drones, which take pictures to obtain inspection infrared images. There is an overlap between adjacent inspection infrared images, such as 60% overlap in the flight direction and 40% overlap in the side direction.
[0137] Step 202: 3D reconstruction and fault detection;
[0138] The inspection infrared images are processed using 3D reconstruction technology to establish 3D point cloud information of the photovoltaic power station;
[0139] Fault identification is performed on the inspection infrared images to obtain information about the initial fault.
[0140] Step 203: 3D point cloud and initial fault storage;
[0141] Store the previously acquired 3D point cloud information and initial fault information.
[0142] Step 210: Acquire real-time infrared images;
[0143] This real-time infrared image is the image to be detected, captured by a drone.
[0144] Step 220: Camera pose correction;
[0145] In this step, the camera pose of the image to be inspected can be corrected based on the target inspection image.
[0146] In actual execution, the target inspection image is determined from the inspection infrared images based on the positioning information of the real-time infrared images. In actual execution, the positioning coordinates (x, y, z) of the real-time infrared images are matched with the existing data in the database to find the closest inspection infrared image. When the inspection infrared image is faulty, the pixel coordinates (u, v) of the fault point in the target inspection image are output.
[0147] Feature matching is performed between real-time infrared images and target inspection images, and the three-dimensional coordinates of feature points in the image to be detected are determined by combining the three-dimensional point cloud information of the photovoltaic power station.
[0148] The camera pose of the image to be detected is corrected based on the pixel coordinates of the aforementioned feature points and the 3D coordinates of the corresponding points of the aforementioned feature points.
[0149] Step 230: Fault detection and string area segmentation;
[0150] In this step, fault identification is performed on the real-time infrared image to obtain the pixel coordinates of the current fault;
[0151] The real-time infrared image is segmented to obtain all the string regions.
[0152] Step 240: Fault pixel coordinates and plane equations;
[0153] In this step, the string region where the current fault is located is obtained based on the pixel coordinates of the current fault;
[0154] Feature extraction is performed on the string region to obtain multiple region feature points;
[0155] By performing feature matching between multiple regional feature points and feature points in the target inspection image, and combining this with the 3D point cloud information of the photovoltaic power station, the 3D coordinates of multiple regional feature points are obtained.
[0156] Based on the three-dimensional coordinates of multiple regional feature points, the planar equation of the string region is determined;
[0157] Step 250: Joint positioning;
[0158] The collinearity equation is determined by the corrected camera pose and the pixel coordinates of the current fault.
[0159] Based on collinearity equations and plane equations, the three-dimensional coordinates of the current fault are determined.
[0160] Step 260, Fault Verification.
[0161] In this step, based on the three-dimensional coordinates of the current fault and the three-dimensional coordinates of the initial fault, it is determined whether the current fault is a new fault.
[0162] The fault location method for photovoltaic power plants proposed in this application can efficiently complete the task of drone inspection by constructing a database of photovoltaic power plants and real-time detection and location of faults. It can achieve real-time fault detection and fault verification. Compared with traditional drone inspection, which involves data collection, data copying, fault diagnosis and location, it greatly improves the efficiency of drone inspection.
[0163] The fault location method for a photovoltaic power station provided in this application can be executed by a fault location device for the photovoltaic power station. This application uses the example of a fault location device for the photovoltaic power station executing the fault location method to illustrate the fault location device for the photovoltaic power station provided in this application.
[0164] This application also provides a fault location device for a photovoltaic power station.
[0165] like Figure 3 As shown, the fault location device of the photovoltaic power station includes: a first receiving module 310, a first processing module 320, a second processing module 330 and a third processing module 340.
[0166] The first receiving module 310 is used to acquire images of a photovoltaic power station to be inspected via a drone;
[0167] The first processing module 320 is used to determine the target inspection image in the inspection image of the photovoltaic power station based on the positioning information of the image to be detected. The inspection image is used to determine the three-dimensional point cloud information of the photovoltaic power station.
[0168] The second processing module 330 is used to identify faults in the image to be detected and obtain the pixel coordinates of the current fault.
[0169] The third processing module 340 is used to determine the three-dimensional coordinates of the current fault based on the camera pose, string region, pixel coordinates of the current fault, target inspection image, and three-dimensional point cloud information of the photovoltaic power station in the image to be detected.
[0170] According to the fault location device for photovoltaic power plants provided in the embodiments of this application, the three-dimensional coordinates of the current fault are determined by combining the camera pose and pixel coordinate information of the current fault carried in the real-time image to be detected with the target inspection image in the database and the three-dimensional point cloud information of the photovoltaic power plant. This can improve the fault detection efficiency of the photovoltaic power plant and ensure accuracy.
[0171] In some embodiments, the third processing module 340 is further configured to segment the image to be detected based on the pixel coordinates of the current fault to obtain the string region where the current fault is located; and to determine the three-dimensional coordinates of the current fault based on the camera pose of the image to be detected, the string region, the pixel coordinates of the current fault, the target inspection image and the three-dimensional point cloud information of the photovoltaic power station.
[0172] In some embodiments, the third processing module 340 is further configured to extract features from the string region to obtain multiple region feature points; perform feature matching between the multiple region feature points and feature points of the target inspection image and combine them with the three-dimensional point cloud information of the photovoltaic power station to obtain the three-dimensional coordinates of the multiple region feature points; determine the plane equation of the string region based on the three-dimensional coordinates of the multiple region feature points; determine the collinearity equation by using the corrected camera pose and the pixel coordinates of the current fault; and determine the three-dimensional coordinates of the current fault based on the collinearity equation and the plane equation.
[0173] In some embodiments, the fault location device may further include: a first correction module, used to correct the camera pose of the image to be detected based on the target inspection image after determining the target inspection image in the inspection images of the photovoltaic power station.
[0174] In some embodiments, the first correction module is further configured to perform feature matching between the image to be detected and the target inspection image, and combine the three-dimensional point cloud information of the photovoltaic power station to determine the three-dimensional coordinates of the feature points in the image to be detected; and correct the camera pose of the image to be detected based on the pixel coordinates and three-dimensional coordinates of multiple feature points in the image to be detected.
[0175] In some embodiments, the first correction module is further configured to determine corresponding points from the target inspection image and feature points in the image to be detected; determine the three-dimensional coordinates corresponding to the corresponding points based on the three-dimensional point cloud information of the photovoltaic power station; and use the three-dimensional coordinates corresponding to the corresponding points as the three-dimensional coordinates of the feature points in the image to be detected.
[0176] In some embodiments, the inspection images are also used to determine the three-dimensional coordinates of the initial fault of the photovoltaic power plant, the pixel coordinates of the initial fault, and the type information of the initial fault.
[0177] The fault location device may further include: a fourth processing module, used to determine whether the current fault is a new fault based on the three-dimensional coordinates of the current fault and the three-dimensional coordinates of the initial fault after determining the three-dimensional coordinates of the current fault.
[0178] By using the three-dimensional coordinates of the current fault, existing fault information in the power plant database can be extracted to determine whether the current fault is a new fault, and some non-permanent faults, such as dust accumulation, can also be ruled out.
[0179] The fault location device for the photovoltaic power station in this application embodiment can be an electronic device or a component of an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the specific type of device.
[0180] The fault location device for the photovoltaic power station in this application embodiment can be a device with an operating system. This operating system can be a Microsoft (Windows) operating system, an Android operating system, an iOS operating system, or other possible operating systems; this application embodiment does not specifically limit it.
[0181] The fault location device for photovoltaic power plants provided in this application embodiment can achieve Figures 1 to 2 The various processes implemented in the method implementation examples will not be described again here to avoid repetition.
[0182] In some embodiments, such as Figure 4 As shown, this application embodiment also provides an electronic device 400, including a processor 401, a memory 402, and a computer program stored in the memory 402 and executable on the processor 401. When the program is executed by the processor 401, it implements the various processes of the above-described photovoltaic power station fault location method embodiment and can achieve the same technical effect. To avoid repetition, it will not be described again here.
[0183] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.
[0184] This application also provides a non-transitory computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the various processes of the above-described photovoltaic power station fault location method embodiment and achieves the same technical effect. To avoid repetition, it will not be described again here.
[0185] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.
[0186] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described fault location method for a photovoltaic power station.
[0187] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.
[0188] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface and the processor are coupled. The processor is used to run programs or instructions to implement the various processes of the above-described photovoltaic power station fault location method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be described again here.
[0189] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.
[0190] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.
[0191] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.
[0192] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.
[0193] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.
[0194] Although embodiments of this application have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the claims and their equivalents.
Claims
1. A fault location method for a photovoltaic power station, characterized in that, include: Images of photovoltaic power plants to be inspected are obtained using drones; Based on the positioning information of the image to be detected, a target inspection image is determined in the inspection image of the photovoltaic power station, and the inspection image is used to determine the three-dimensional point cloud information of the photovoltaic power station. Feature matching is performed between the image to be detected and the target inspection image, and the three-dimensional coordinates of the feature points in the image to be detected are determined by combining the three-dimensional point cloud information of the photovoltaic power station. The camera pose of the image to be detected is corrected based on the pixel coordinates and three-dimensional coordinates of multiple feature points in the image to be detected. Fault identification is performed on the image to be detected to obtain the pixel coordinates of the current fault; The image to be detected is segmented based on the pixel coordinates of the current fault to obtain the string region where the current fault is located; Based on the camera pose of the image to be detected, the string region, the pixel coordinates of the current fault, the target inspection image, and the three-dimensional point cloud information of the photovoltaic power station, the three-dimensional coordinates of the current fault are determined. The determination of the three-dimensional coordinates of the current fault based on the corrected camera pose, the string region, the pixel coordinates of the current fault, the target inspection image, and the three-dimensional point cloud information of the photovoltaic power station includes: Feature extraction is performed on the string region to obtain multiple region feature points; The feature points of the multiple regions are matched with the feature points of the target inspection image, and combined with the three-dimensional point cloud information of the photovoltaic power station to obtain the three-dimensional coordinates of the multiple region feature points. Based on the three-dimensional coordinates of the multiple regional feature points, the plane equation of the string region is determined; The collinearity equation is determined by the corrected camera pose and the pixel coordinates of the current fault. Based on the collinearity equation and the plane equation, the three-dimensional coordinates of the current fault are determined.
2. The fault location method for a photovoltaic power station according to claim 1, characterized in that, The step of performing feature matching between the image to be detected and the target inspection image, and combining this with the three-dimensional point cloud information of the photovoltaic power station to determine the three-dimensional coordinates of the feature points in the image to be detected, includes: Determine the points with the same name as the feature points in the image to be detected from the target inspection image; Based on the three-dimensional point cloud information of the photovoltaic power station, determine the three-dimensional coordinates corresponding to the same point; The three-dimensional coordinates corresponding to the points of the same name are used as the three-dimensional coordinates of the feature points of the image to be detected.
3. The fault location method for a photovoltaic power station according to claim 1, characterized in that, The inspection images are also used to determine the three-dimensional coordinates of the initial fault, the pixel coordinates of the initial fault, and the type information of the initial fault of the photovoltaic power station. After determining the three-dimensional coordinates of the current fault, the method further includes: determining whether the current fault is a new fault based on the three-dimensional coordinates of the current fault and the three-dimensional coordinates of the initial fault.
4. The fault location method for a photovoltaic power station according to any one of claims 1-3, characterized in that, Prior to acquiring the image of the photovoltaic power station to be inspected via a drone, the method further includes: The inspection images of the photovoltaic power station are processed by three-dimensional reconstruction technology to establish three-dimensional point cloud information of the photovoltaic power station. The three-dimensional point cloud information includes: three-dimensional point cloud coordinates, pixel coordinates of each point in each inspection image, and descriptive information of the point. The inspection images are used to identify faults, and the location information and type information of the initial faults are obtained. Based on the three-dimensional point cloud information and the location information of the initial fault, the three-dimensional coordinates and pixel coordinates of the initial fault are obtained.
5. A fault location device for a photovoltaic power station, characterized in that, include: The first receiving module is used to acquire images of the photovoltaic power station to be inspected via a drone; The first processing module is used to determine the target inspection image in the inspection image of the photovoltaic power station based on the positioning information of the image to be detected. The inspection image is used to determine the three-dimensional point cloud information of the photovoltaic power station. The first correction module is used to perform feature matching between the image to be detected and the target inspection image, and, in conjunction with the three-dimensional point cloud information of the photovoltaic power station, determine the three-dimensional coordinates of feature points in the image to be detected; and correct the camera pose of the image to be detected based on the pixel coordinates and three-dimensional coordinates of multiple feature points in the image to be detected. The second processing module is used to perform fault identification on the image to be detected and obtain the pixel coordinates of the current fault; The third processing module is used to segment the image to be detected based on the pixel coordinates of the current fault to obtain the string region where the current fault is located. Based on the camera pose of the image to be detected, the string region, the pixel coordinates of the current fault, the target inspection image, and the three-dimensional point cloud information of the photovoltaic power station, the three-dimensional coordinates of the current fault are determined. The determination of the three-dimensional coordinates of the current fault based on the corrected camera pose, the string region, the pixel coordinates of the current fault, the target inspection image, and the three-dimensional point cloud information of the photovoltaic power station includes: Feature extraction is performed on the string region to obtain multiple region feature points; The feature points of the multiple regions are matched with the feature points of the target inspection image, and combined with the three-dimensional point cloud information of the photovoltaic power station to obtain the three-dimensional coordinates of the multiple region feature points. Based on the three-dimensional coordinates of the multiple regional feature points, the plane equation of the string region is determined; The collinearity equation is determined by the corrected camera pose and the pixel coordinates of the current fault. Based on the collinearity equation and the plane equation, the three-dimensional coordinates of the current fault are determined.
6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the fault location method for a photovoltaic power station as described in any one of claims 1-4.
7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the fault location method for a photovoltaic power station as described in any one of claims 1-4.