Fault location method, device and electronic equipment
By acquiring and processing multiple picture samples collected by the drone, the positioning coordinates of the fault points of the photovoltaic power station equipment are calculated, which solves the problem of insufficient positioning accuracy in the existing technology and achieves high-precision fault point positioning.
Patent Information
- Application Number
- CN202110896467.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-08-05
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2041-08-05
Smart Images

Figure CN113624133B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of photovoltaic power station fault positioning, and more specifically, to a fault positioning method, device and electronic equipment. Background Art
[0002] With the development of drone technology, the operation and maintenance of photovoltaic power stations has gradually changed from manual inspections to collecting images of equipment through drones equipped with visible light cameras and infrared cameras, thereby realizing intelligent detection of equipment (such as photovoltaic modules) failures. This method can greatly improve the operation and maintenance efficiency of photovoltaic power stations and save human resources.
[0003] After determining the image including the fault point through the collected images, how to accurately locate the geographical coordinates of the fault point of the actual equipment based on the fault point in the image including the fault point is a technical problem that technical personnel in this field urgently need to solve. Summary of the invention
[0004] In view of this, the present invention provides a fault location method, device and electronic device to solve the problem of accurately locating the geographical coordinates of the fault point of the actual device according to the fault point in a picture including the fault point.
[0005] In order to solve the above technical problems, the present invention adopts the following technical solutions:
[0006] A fault location method, comprising:
[0007] Acquire a plurality of collected image samples within a preset range of the device under test, and determine actual elevation information within the preset range of the device under test based on the plurality of image samples;
[0008] Obtaining a picture including a fault point corresponding to the device under test, and determining camera parameter information corresponding to the picture;
[0009] Based on the actual elevation information and the camera parameter information, determining the positioning coordinates of the device under test corresponding to the fault point;
[0010] From the positioning coordinates, positioning coordinates that meet preset coordinate rules are screened out and used as actual fault point coordinates of the device under test corresponding to the fault point.
[0011] Optionally, determining actual elevation information within a preset range of the device under test based on the multiple image samples includes:
[0012] Extracting a camera parameter information sample from the image sample;
[0013] Correcting the camera parameter information sample in the image sample to obtain reference camera parameter information corresponding to the image sample;
[0014] Performing global dense matching on reference camera parameter information corresponding to the image samples to obtain a dense three-dimensional point cloud;
[0015] An inverse distance weighted interpolation calculation is performed on the dense three-dimensional point cloud to obtain actual elevation information within a preset range of the device under test.
[0016] Optionally, determining camera parameter information corresponding to the image includes:
[0017] Performing image matching between the image and the multiple image samples, determining an image sample corresponding to the image and using it as a target image sample;
[0018] The reference camera parameter information corresponding to the target image sample is obtained and used as the camera parameter information corresponding to the image.
[0019] Optionally, determining the location coordinates of the device under test corresponding to the fault point based on the actual elevation information and the camera parameter information includes:
[0020] Acquire a collinear equation, wherein the collinear equation includes an association relationship between the camera parameter information and the positioning coordinates of the device under test corresponding to the fault point;
[0021] Determine iteration parameters; the iteration parameters include maximum elevation, minimum elevation and elevation step length;
[0022] Based on the iteration parameters and the collinearity equation, an iterative calculation is performed to obtain the location coordinates of the device under test corresponding to the fault point.
[0023] Optionally, obtain collinear equations, including:
[0024] Obtaining a basic collinearity equation; the basic collinearity equation includes: the image coordinates of the fault point in the picture including the fault point, the camera parameter information, the positioning coordinates of the device under test corresponding to the fault point, and the correlation relationship between the specified coefficients;
[0025] Calculating the value of the specified coefficient according to the camera posture information in the camera parameter information;
[0026] Substituting the numerical value of the designated coefficient into the basic collinear equation, a collinear equation is obtained.
[0027] Optionally, determine iteration parameters, including:
[0028] Determine a location point corresponding to the camera parameter information in the actual elevation information;
[0029] Filtering out the maximum elevation and the minimum elevation within a specified range with the location point as a reference point from the actual elevation information;
[0030] Gets the preset elevation step size.
[0031] Optionally, performing iterative calculation based on the iteration parameter and the collinearity equation to obtain the location coordinates of the device under test corresponding to the fault point includes:
[0032] setting the reference elevation to the minimum elevation;
[0033] According to the collinear equation, the horizontal and vertical coordinate points of the positioning coordinates corresponding to the reference elevation are calculated, and the target elevation corresponding to the horizontal and vertical coordinate points in the actual elevation information is determined;
[0034] Determining whether the elevation difference between the target elevation and the reference elevation is less than a preset elevation difference;
[0035] If it is less than, the preset elevation difference is updated to the elevation difference, and the position of the device to be tested corresponding to the target elevation and the horizontal and vertical coordinate points is used as the positioning coordinates;
[0036] The sum of the reference elevation and the elevation step is used as a new reference elevation, and the process returns to the step of calculating the horizontal and vertical coordinate points of the positioning coordinates corresponding to the reference elevation according to the collinear equation, until the reference elevation is greater than the maximum elevation.
[0037] Optionally, determining a target elevation corresponding to the horizontal and vertical coordinate points in the actual elevation information includes:
[0038] Determine whether the horizontal and vertical coordinate points are located on the grid points in the actual elevation information;
[0039] If not, a bilinear interpolation method is used to determine the target elevation corresponding to the horizontal and vertical coordinate points in the actual elevation information.
[0040] Optionally, when it is determined that the elevation difference between the target elevation and the reference elevation is not less than a preset elevation difference, the method further includes:
[0041] The sum of the reference elevation and the elevation step is used as a new reference elevation, and the process returns to the step of calculating the horizontal and vertical coordinate points of the positioning coordinates corresponding to the reference elevation according to the collinear equation, until the reference elevation is greater than the maximum elevation.
[0042] Optionally, selecting positioning coordinates satisfying preset coordinate rules from the positioning coordinates and using them as actual fault point coordinates of the device under test corresponding to the fault point includes:
[0043] From the positioning coordinates, the positioning coordinates whose elevation difference between the target elevation and the reference elevation corresponding to the target elevation is the smallest or whose elevation ratio is closest to a specified value are screened out.
[0044] A fault locating device, comprising:
[0045] An elevation determination module, used to obtain a plurality of collected image samples within a preset range of the device under test, and determine actual elevation information within the preset range of the device under test based on the plurality of image samples;
[0046] An information determination module, used to obtain a picture including a fault point corresponding to the device under test, and determine camera parameter information corresponding to the picture;
[0047] A fault point determination module, used to determine the positioning coordinates of the device under test corresponding to the fault point based on the actual elevation information and the camera parameter information;
[0048] The fault point screening module is used to screen out the positioning coordinates that meet the preset coordinate rules from the positioning coordinates and use them as the actual fault point coordinates of the device under test corresponding to the fault point.
[0049] A storage medium includes a stored program, wherein when the program is run, the device where the storage medium is located is controlled to execute the above-mentioned fault locating method.
[0050] An electronic device comprising: a memory and a processor;
[0051] Wherein, the memory is used to store programs;
[0052] The processor calls the program and is used to execute the above fault location method.
[0053] Compared with the prior art, the present invention has the following beneficial effects:
[0054] The present invention provides a fault location method, device and electronic device, which obtains a plurality of collected image samples within a preset range of a device to be tested, and determines the actual elevation information within the preset range of the device to be tested based on the plurality of image samples, obtains an image including a fault point corresponding to the device to be tested, and determines the camera parameter information corresponding to the image, and determines the positioning coordinates of the device to be tested corresponding to the fault point based on the actual elevation information and the camera parameter information, and then selects the positioning coordinates that meet the preset coordinate rules from the positioning coordinates, and uses them as the actual fault point coordinates of the device to be tested corresponding to the fault point. Through the present invention, the actual coordinates of the fault point of the device can be located according to the fault point in the image including the fault point. Furthermore, when determining the actual fault point coordinates, the positioning coordinates that meet the preset coordinate rules are also selected from the positioning coordinates, and used as the actual fault point coordinates of the device to be tested corresponding to the fault point, thereby improving the accuracy of determining the actual fault point coordinates. BRIEF DESCRIPTION OF THE DRAWINGS
[0055] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.
[0056] Figure 1 A method flow chart of a fault location method provided by an embodiment of the present invention;
[0057] Figure 2 A method flow chart of another fault location method provided by an embodiment of the present invention;
[0058] Figure 3 A method flow chart of another fault location method provided by an embodiment of the present invention;
[0059] Figure 4 A method flow chart of another fault locating method provided by an embodiment of the present invention;
[0060] Figure 5 A method flow chart of a fifth fault locating method provided by an embodiment of the present invention;
[0061] Figure 6 A schematic structural diagram of a fault location device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0062] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0063] With the development of drone technology, the operation and maintenance of photovoltaic power stations has gradually changed from manual inspections in the past to the use of drones equipped with visible light and infrared cameras to collect photos for equipment, such as intelligent detection of photovoltaic module failures. This method has greatly improved the efficiency of photovoltaic power station operation and maintenance and saved a lot of human resources.
[0064] After the fault detection of the PV module is completed, it is necessary to accurately locate the PV module fault, mark the positioning result on the global electronic map, and guide the operation and maintenance personnel to perform operation and maintenance.
[0065] At present, the following three methods can be used to accurately locate photovoltaic module faults:
[0066] 1. Use visible light photos and infrared photos taken simultaneously during drone inspections to perform image registration, determine the pixel coordinate conversion relationship between visible light photos and infrared photos, convert the pixel coordinates of the fault detected in the infrared photo into the pixel coordinates of the visible light photo, and then align the visible light photo with the electronic map of the power station, and finally complete the calibration of the photovoltaic module fault on the electronic map of the power station.
[0067] The disadvantage of this solution is that due to the different imaging methods of visible light photos and infrared photos, the resolution and shooting range of visible light photos and infrared photos are very different, and the texture difference between strings is small. Therefore, it is difficult or impossible to accurately align visible light and infrared images in many drone photovoltaic inspection scenarios.
[0068] 2. When taking photos with a drone, the drone system provides the GPS coordinates of the camera photography center and the camera's attitude parameters (attitude parameters are mainly yaw angle, pitch angle, roll angle) and flight altitude data. The distance to the central pixel coordinates of the infrared photo is calculated based on the pixel coordinates of the fault detected on the infrared photo to locate the fault point. The problem with this type of solution is that the attitude parameters provided by the drone system are not accurate, the central pixel coordinates of the infrared photo are offset, and the drone's flight altitude data cannot remain constant in places with large elevation differences, which will lead to a large offset in the fault point location solution.
[0069] 3. Use SFM (Structure from Motion) to reconstruct the camera GPS (Global Positioning System) coordinates and camera posture of all photos taken by the drone, and establish the correspondence between three-dimensional coordinates and two-dimensional pixels based on the sparse point cloud data generated by SFM, and finally interpolate the three-dimensional coordinates corresponding to the pixel coordinates of the fault point. The biggest problem with this type of solution is that when there are not enough sparse point clouds or no sparse point clouds within a certain range around the pixel coordinates of the fault point, the interpolation will be unstable and the error will be uncontrollable, and this situation is very common in photovoltaic scenarios. In addition, the point cloud data is required to be the point cloud data of photovoltaic modules, but in practice, the point cloud data may be the point cloud data of trees around the photovoltaic modules, and thus the fault location cannot be achieved.
[0070] It can be seen from the above three methods that it is currently impossible to accurately locate the actual fault point coordinates of the equipment. Therefore, a method that can accurately locate the fault point is urgently needed.
[0071] To this end, the inventor has found through research that it is possible to obtain multiple image samples collected within a preset range of the device to be tested, and determine the actual elevation information within the preset range of the device to be tested based on the multiple image samples, obtain a picture including a fault point corresponding to the device to be tested, and determine the camera parameter information corresponding to the picture, and determine the positioning coordinates of the device to be tested corresponding to the fault point based on the actual elevation information and the camera parameter information, and then filter out the positioning coordinates that meet the preset coordinate rules from the positioning coordinates, and use them as the actual fault point coordinates of the device to be tested corresponding to the fault point. Through the present invention, the actual coordinates of the fault point of the device can be located according to the fault point in the picture including the fault point. Furthermore, when determining the actual fault point coordinates, the positioning coordinates that meet the preset coordinate rules will also be filtered out from the positioning coordinates, and used as the actual fault point coordinates of the device to be tested corresponding to the fault point, thereby improving the accuracy of determining the actual fault point coordinates.
[0072] Based on the above content, an embodiment of the present invention provides a fault location method, referring to Figure 1 , which may include:
[0073] S11. Acquire a plurality of image samples collected within a preset range of the device under test, and determine actual elevation information within the preset range of the device under test based on the plurality of image samples.
[0074] In this embodiment, the device to be tested can be a device in a photovoltaic power station, such as a photovoltaic module. Specifically, taking the device to be tested as a photovoltaic module as an example, in a photovoltaic inspection scenario, a drone equipped with an infrared camera can be used to collect images of the photovoltaic module to obtain drone infrared photos. When collecting specific images, a high heading and lateral overlap rate needs to be set between each picture. In addition, it is necessary to ensure that all pictures can be stitched together to cover all photovoltaic modules, that is, it is necessary to collect multiple photos of photovoltaic modules to cover all photovoltaic modules. In this embodiment, the collected pictures are called picture samples.
[0075] In addition to collecting infrared images, drones can also be used to collect visible light images.
[0076] Then, the actual elevation information within a preset range of the device under test can be determined based on the multiple image samples, such as obtaining the actual elevation information within 100 meters of the device under test. The actual elevation information can be digital surface model DSM information, which stores the actual elevation information of the ground, number, equipment (such as photovoltaic components), buildings, etc. within 100 meters of the device under test.
[0077] S12: Obtain a picture including a fault point corresponding to the device under test, and determine camera parameter information corresponding to the picture.
[0078] In this embodiment, a drone may be used to collect images of the device under test, which may be referred to as drone images in this embodiment. The drone images may be the above-mentioned infrared images or visible light images.
[0079] Then, the pictures including the fault point are screened out from the collected pictures, and then the camera parameter information corresponding to the pictures is determined. The camera parameter information in this embodiment may be the camera GPS coordinates, camera posture information, and camera internal parameters.
[0080] In this embodiment, the camera GPS coordinates are represented by (Xa, Ya, Za), and the camera posture information is represented by three angles To express, it means rotation around Y axis Angle, then rotate around the X axis by angle ω, and finally rotate around the Z axis by angle k.
[0081] The camera intrinsic parameters are represented by (f, cx, cy), where f represents the focal length of the camera, and cx and cy represent the pixel coordinates of the camera's photographic center.
[0082] The pixel coordinates of the fault point detected in the above picture are (u, v), and the actual three-dimensional coordinates of the fault point are represented by (X, Y, Z), where X, Y, and Z are the data to be solved in this embodiment.
[0083] S13: Based on the actual elevation information and the camera parameter information, determine the location coordinates of the device under test corresponding to the fault point.
[0084] In this embodiment, multiple positioning coordinates can be determined based on the actual elevation information and the camera parameter information. The positioning coordinates in this embodiment are possible locations where the device under test has a fault, and the final fault point location needs to be screened out from these locations.
[0085] S14. Filter out positioning coordinates that meet preset coordinate rules from the positioning coordinates, and use them as actual fault point coordinates of the device under test corresponding to the fault point.
[0086] Specifically, the positioning coordinates whose target elevation is closest to the reference elevation can be screened out from the positioning coordinates. Specifically, the two can satisfy a preset elevation difference rule or a preset elevation ratio rule, and serve as the actual fault point coordinates of the device under test corresponding to the fault point; the reference elevation is determined based on the camera parameter information and corresponds to the target elevation.
[0087] In this embodiment, the positioning coordinates having the smallest elevation difference between the target elevation and the reference elevation corresponding to the target elevation or having an elevation ratio closest to 1 (ie, a specified value) can be screened out from the positioning coordinates.
[0088] Among them, if the elevation difference between the target elevation and the reference elevation corresponding to the target elevation is the smallest or the elevation ratio is closest to 1, it means that the determined target elevation is closer to the reference elevation, which improves the accuracy of determining the elevation of the actual fault point coordinates. Then, XY, that is, the values of the horizontal and vertical coordinates, are calculated based on the elevation, and then the actual three-dimensional coordinates of the fault point can be obtained. In this embodiment, since the accuracy of the determined elevation Z is improved, the XY coordinates are calculated based on the elevation Z, thereby improving the accuracy of determining the actual position of the actual fault point coordinates.
[0089] It should be noted that the positioning coordinates whose target elevation is closest to the reference elevation can be screened out through elevation difference or elevation ratio. In addition, the positioning coordinates whose target elevation is closest to the reference elevation can be determined through deviation, similarity, etc.
[0090] Based on the above, refer to Figure 2 , obtain the actual elevation information within the preset range of the device under test, which can include:
[0091] S21. Extracting a camera parameter information sample from the image sample.
[0092] Specifically, the device to be tested is a photovoltaic module. In the photovoltaic inspection scenario, a drone equipped with an infrared camera can be used to collect images of the photovoltaic module, and the drone infrared photos can be taken as an example. The pictures collected by the infrared camera will record the camera parameter information samples, including: camera GPS coordinates, camera attitude information, and camera internal parameters.
[0093] S22: Correct the camera parameter information sample in the image sample to obtain reference camera parameter information corresponding to the image sample.
[0094] The camera parameter information samples carried in the above-mentioned image samples will be affected by the flight stability of the drone and have low accuracy. Therefore, the camera parameter information samples need to be corrected. Specifically, the SFM three-dimensional reconstruction technology is used to restore the precise camera GPS coordinates, camera attitude information and camera intrinsic parameters of each photo. In this embodiment, the corrected camera parameter information samples are called reference camera parameter information.
[0095] S23. Perform global dense matching on the reference camera parameter information corresponding to the image sample to obtain a dense three-dimensional point cloud.
[0096] Specifically, the above-mentioned precise camera GPS coordinates, camera posture information and camera internal parameters are used to perform global dense matching to obtain a dense three-dimensional point cloud, which is referred to as a dense three-dimensional point cloud in this embodiment.
[0097] S24, performing inverse distance weighted interpolation calculation on the dense three-dimensional point cloud to obtain actual elevation information within a preset range of the device under test.
[0098] Specifically, the dense three-dimensional point cloud is generated by using the inverse distance weighted interpolation method based on the photovoltaic module information. Specifically, when using the inverse distance weighted interpolation method, it is necessary to consider the spatial range and weight method involved in the inverse distance weighted interpolation.
[0099] In this embodiment, the obtained DSM model is the actual elevation information.
[0100] In this embodiment, when a DSM model is generated based on a three-dimensional point cloud, global dense matching is performed, thereby making the three-dimensional point cloud more dense, and thus the generated DSM more accurate.
[0101] Based on the above embodiment of determining the DSM model, determining the camera parameter information corresponding to the image may include:
[0102] S31. Perform image matching on the image and the multiple image samples to determine an image sample corresponding to the image and use it as a target image sample.
[0103] Specifically, the above-mentioned multiple picture samples are all pictures taken of photovoltaic components, and the reference camera parameter information is determined, and then the picture samples that match the pictures taken this time including the fault points can be screened out from the multiple picture samples, so that the reference camera parameter information of the picture samples is used as the camera parameter information of the picture this time.
[0104] When performing image matching between the image and the plurality of image samples, a picture name matching method may be adopted, or the similarity of the images may be calculated, and the image sample with the greatest similarity may be used as the image sample that matches the image.
[0105] It should be noted that in order to improve the accuracy of image matching, when collecting images and image samples, they can be shot in the same shooting method, such as shooting at the same height.
[0106] S32: Obtain reference camera parameter information corresponding to the target image sample, and use the reference camera parameter information corresponding to the image.
[0107] In this embodiment, since the reference camera parameter information of multiple image samples is determined in advance, and the reference camera parameter information is corrected by the camera parameter information samples, the accuracy is relatively high. Then, the image sample matching the image can be directly determined, so that the reference camera parameter information of the image sample is used as the camera parameter information of the current image. Compared with the method of directly extracting the camera parameter information of the image from the image and then correcting it, the efficiency of determining the camera parameter information can be improved.
[0108] In another implementation of the present invention, a specific implementation process of "determining the positioning coordinates of the device under test corresponding to the fault point based on the actual elevation information and the camera parameter information" is given, referring to Figure 4 , which may include:
[0109] S41. Obtain collinear equations.
[0110] The collinear equation includes an association relationship between the camera parameter information and the positioning coordinates of the device under test corresponding to the fault point.
[0111] In practical applications, the collinearity equation also needs to be obtained by calculating the values of the specified coefficients in the basic collinearity equation.
[0112] Therefore, obtaining the collinear equations may include:
[0113] 1) Obtain the basic collinear equation.
[0114] The basic collinear equation includes: the image coordinates of the fault point in the picture including the fault point, the camera parameter information, the positioning coordinates of the device under test corresponding to the fault point, and the correlation relationship between the specified coefficients.
[0115] Specifically, the basic collinear equation is:
[0116]
[0117]
[0118] Wherein, (u, v) is the image coordinate of the fault point in the picture including the fault point, also called pixel coordinate.
[0119] The GPS coordinates of the camera are represented by (Xa, Ya, Za), and the actual three-dimensional coordinates of the fault point are represented by (X, Y, Z), where X, Y, and Z are the data to be solved in this embodiment.
[0120] The camera intrinsic parameters are represented by (f, cx, cy), where f represents the focal length of the camera, and cx and cy represent the pixel coordinates of the camera's photographic center.
[0121] Camera pose information using three angles To express, it means rotation around Y axis Angle, then rotate around the X axis by angle ω, and finally rotate around the Z axis by angle k.
[0122] a11, a12, a13, a21, a22, a23, a31, a32, and a33 are designated coefficients in this embodiment and can be calculated using camera posture information.
[0123] 2) Calculate the value of the specified coefficient based on the camera posture information in the camera parameter information.
[0124] The calculation method of a11,a12,a13,a21,a22,a23,a31,a32,a33 is as follows:
[0125] a11,a12,a13,a21,a22,a23,a31,a32,a33 and camera attitude information satisfy the following relationship:
[0126]
[0127] Through this relationship, the values of a11, a12, a13, a21, a22, a23, a31, a32, and a33 can be calculated.
[0128] 3) Substitute the numerical value of the specified coefficient into the basic collinearity equation to obtain the collinearity equation.
[0129] Specifically, by substituting the above-mentioned a11, a12, a13, a21, a22, a23, a31, a32, and a33 into the basic collinearity equation, we can obtain the collinearity equation. In this equation, u and v are known, so there are only three unknowns, XYZ.
[0130] S42. Determine iteration parameters.
[0131] In this embodiment, it can be seen from the collinear equation that the unknowns to be solved are (X, Y, Z), that is, three unknowns, but the collinear equation only satisfies two equations and cannot directly solve the three unknowns. Therefore, it is necessary to use DSM data, that is, the actual elevation information to provide the Z value for auxiliary iterative solution.
[0132] In this embodiment, the iteration parameters include maximum elevation, minimum elevation and elevation step.
[0133] Specifically, the process of determining the iteration parameters is as follows:
[0134] 1) Determine the location point corresponding to the camera parameter information in the actual elevation information.
[0135] Specifically, after determining the camera parameter information corresponding to the image, a corresponding position can be determined in the actual elevation information based on the information, which is referred to as a position point in this embodiment.
[0136] 2) Filter out the maximum elevation and the minimum elevation within a specified range with the location point as a reference point in the actual elevation information.
[0137] Specifically, an area may be selected with the location point as the center and 10 meters as the radius, and is called area Z. Then, the maximum elevation Zmax and the minimum elevation Zmin in area Z are obtained.
[0138] 3) Obtain the preset elevation step length.
[0139] In this embodiment, the elevation step length step can be manually set, such as 0.1 meter.
[0140] S43. Perform iterative calculation based on the iterative parameters and the collinearity equation to obtain the location coordinates of the device under test corresponding to the fault point.
[0141] In practical applications, refer to Figure 5 , step S43 may include:
[0142] S51. Setting the reference elevation as the minimum elevation.
[0143] In this embodiment, since the above-mentioned area Z has been defined, it is necessary to select a suitable elevation from the area Z as the actual elevation of the positioning coordinates of the device under test. Therefore, in this embodiment, the actual elevation of the positioning coordinates of the device under test ranges between the minimum elevation and the maximum elevation, and the most suitable elevation needs to be determined through iteration.
[0144] First, the elevation of the positioning coordinates is set to the minimum elevation, and then the elevation value is continuously adjusted until the maximum elevation is reached. The entire process is iteratively calculated, and the appropriate elevation value is selected from the maximum elevation and the minimum elevation, and used as the actual elevation of the positioning coordinates of the device under test.
[0145] In this embodiment, the reference elevation may be set to Zi, that is, in this embodiment, Zi=Zmin.
[0146] S52. Calculate the horizontal and vertical coordinate points of the positioning coordinates corresponding to the reference elevation according to the collinear equation, and determine the target elevation corresponding to the horizontal and vertical coordinate points in the actual elevation information.
[0147] Specifically, assuming that the elevation of the current iteration is the Zi value, at this time, there are only two unknowns XY in the collinear equation, and the values of XY can be calculated, which are Xi and Yi respectively.
[0148] According to the current DSM data, that is, the actual elevation information mentioned above, use Xi, Yi to calculate the grid point position in the DSM data to determine whether the horizontal and vertical coordinate points are located on the grid points in the actual elevation information. If so, directly read the elevation value of the grid point and use it as the target elevation Zt.
[0149] If not, a bilinear interpolation method is used to determine the target elevation corresponding to the horizontal and vertical coordinate points in the actual elevation information, that is, the actual elevation value corresponding to Xi, Yi, that is, the target elevation Zt, is determined by a bilinear interpolation method.
[0150] S53, determining whether the elevation difference between the target elevation and the reference elevation is less than a preset elevation difference; if so, executing step S54; if not, executing step S55.
[0151] In this embodiment, the elevation difference Zdiff=abs(Zi-Zt) is calculated, and it is determined whether Zdiff is less than a preset elevation difference DIFFZ. In this embodiment, DIFFZ may be initially a relatively large value, such as 999.
[0152] When the elevation difference between the target elevation and the reference elevation is not less than the preset elevation difference, it means that the elevation difference is too large, that is, the difference between the calculated target elevation and the selected reference elevation is too large and does not meet the requirements, then Zi=Zi+step is updated and the next iteration is performed.
[0153] S54: Update the preset elevation difference to the elevation difference, and use the target elevation and the position of the device to be tested corresponding to the horizontal and vertical coordinate points as positioning coordinates.
[0154] If Zdiff is less than the preset elevation difference DIFFZ, then DIFFZ=Zdiff, and the current Xi, Yi, Zt values are recorded as X=Xi, Y=Yi, Z=Zt, and the position point of the device under test corresponding to (Xi, Yi, Zt) is used as the positioning coordinate.
[0155] S55, judging whether the reference elevation is greater than the maximum elevation; if so, the process ends; if not, executing step S56.
[0156] In this embodiment, when the reference elevation is greater than the maximum elevation Zmax, the loop is exited and the iteration stops.
[0157] S56. Taking the sum of the reference elevation and the elevation step as a new reference elevation.
[0158] Specifically, update Zi=Zi+step, and return to execute step S52. If Zi>Zmax, then jump out of the loop.
[0159] In this embodiment, Zi=Zi+step is continuously updated, and then it is determined whether the current Zi has a corresponding positioning coordinate. When Zi>Zmax, multiple positioning coordinates are obtained through continuous iteration.
[0160] After obtaining multiple positioning coordinates, it is necessary to select a suitable positioning coordinate from them. In this embodiment, the positioning coordinates with the smallest elevation difference between the target elevation and the reference elevation corresponding to the target elevation or the elevation ratio closest to 1 are screened out from the positioning coordinates, and then the coordinates of the determined positioning coordinates, that is, the XYZ values, are output. Specifically, they can be output to a display interface or to a preset terminal, such as a patrol personnel's mobile phone or other terminal.
[0161] The accuracy of the drone photovoltaic inspection data that has been collected so far has been verified, and its positioning accuracy error is less than 50cm, which can meet the level of accurate positioning of the component. (The photovoltaic component is 1.5m long and 1m wide, and the maximum height of the fault positioning for the existing drone photovoltaic inspection is less than 1m).
[0162] In this embodiment, the precise camera GPS coordinates, camera attitude information and camera intrinsic parameters are reconstructed through the SFM process, and a dense point cloud is generated through dense matching to obtain the DSM. Based on the SFM results, the precise camera GPS coordinates, camera attitude information, camera intrinsic parameters and high-resolution DSM data are used to iteratively solve the collinear equation to obtain the precise three-dimensional coordinates of the fault point of the photovoltaic module, thereby improving the accuracy of fault point positioning.
[0163] Optionally, based on the above-mentioned embodiment of the fault location method, another embodiment of the present invention provides a fault location device, including:
[0164] The elevation determination module 11 is used to obtain a plurality of image samples collected within a preset range of the device under test, and determine the actual elevation information within the preset range of the device under test based on the plurality of image samples;
[0165] An information determination module 12 is used to obtain a picture including a fault point corresponding to the device under test, and determine camera parameter information corresponding to the picture;
[0166] A fault point determination module 13 is used to determine the positioning coordinates of the device under test corresponding to the fault point based on the actual elevation information and the camera parameter information;
[0167] The fault point screening module 14 is used to screen out the positioning coordinates that meet the preset coordinate rules from the positioning coordinates, and use them as the actual fault point coordinates of the device under test corresponding to the fault point.
[0168] Further, the elevation determination module 11 includes:
[0169] A sample processing submodule, used to extract a camera parameter information sample from the image sample;
[0170] A correction submodule, used to correct the camera parameter information sample in the image sample to obtain reference camera parameter information corresponding to the image sample;
[0171] A dense matching submodule, used for performing global dense matching on the reference camera parameter information corresponding to the image sample to obtain a dense three-dimensional point cloud;
[0172] The information calculation submodule is used to perform inverse distance weighted interpolation calculation on the dense three-dimensional point cloud to obtain actual elevation information within a preset range of the device under test.
[0173] Further, the information determination module 12 is specifically used for:
[0174] The image is matched with the multiple image samples to determine the image sample corresponding to the image and use it as the target image sample; reference camera parameter information corresponding to the target image sample is obtained and used as the camera parameter information corresponding to the image.
[0175] Further, the fault point determination module 13 includes:
[0176] An equation acquisition submodule, used to acquire a collinear equation, wherein the collinear equation includes an association relationship between the camera parameter information and the positioning coordinates of the device under test corresponding to the fault point;
[0177] The parameter determination submodule is used to determine the iteration parameters; the iteration parameters include the maximum elevation, the minimum elevation and the elevation step length;
[0178] The fault point determination submodule is used to perform iterative calculation based on the iterative parameters and the collinear equation to obtain the positioning coordinates of the device under test corresponding to the fault point.
[0179] Furthermore, the equation acquisition submodule includes:
[0180] An equation acquisition unit, used to acquire a basic collinear equation; the basic collinear equation includes: the image coordinates of the fault point in the picture including the fault point, the camera parameter information, the positioning coordinates of the device under test corresponding to the fault point, and the correlation relationship between the specified coefficients;
[0181] A numerical calculation unit, used to calculate the numerical value of the specified coefficient according to the camera posture information in the camera parameter information;
[0182] The equation determination unit is used to substitute the numerical value of the specified coefficient into the basic collinear equation to obtain the collinear equation.
[0183] Furthermore, the parameter determination submodule includes:
[0184] A position point determination unit, used to determine the position point corresponding to the camera parameter information in the actual elevation information;
[0185] An elevation screening unit, used to screen out the maximum elevation and the minimum elevation within a specified range with the location point as a reference point in the actual elevation information;
[0186] The step length acquisition unit is used to obtain a preset elevation step length.
[0187] Furthermore, the fault point determination submodule includes:
[0188] an elevation setting unit, used for setting the reference elevation to the minimum elevation;
[0189] An elevation determination unit, used to calculate the horizontal and vertical coordinate points of the positioning coordinates corresponding to the reference elevation according to the collinear equation, and determine the target elevation corresponding to the horizontal and vertical coordinate points in the actual elevation information;
[0190] A first judging unit, used to judge whether the elevation difference between the target elevation and the reference elevation is less than a preset elevation difference;
[0191] A fault point determination unit, configured to update the preset elevation difference to the elevation difference if it is less than , and use the target elevation and the position of the device to be tested corresponding to the horizontal and vertical coordinate points as positioning coordinates;
[0192] A second judging unit, used for judging whether the reference elevation is greater than the maximum elevation;
[0193] The elevation setting unit is also used to use the sum of the reference elevation and the elevation step as a new reference elevation when the reference elevation is not greater than the maximum elevation.
[0194] Further, when the elevation determination unit is used to determine the target elevation corresponding to the horizontal and vertical coordinate points in the actual elevation information, it is specifically used to:
[0195] Determine whether the horizontal and vertical coordinate points are located on the grid points in the actual elevation information; if not, use bilinear interpolation to determine the target elevation corresponding to the horizontal and vertical coordinate points in the actual elevation information.
[0196] Furthermore, the elevation setting unit is also used for:
[0197] When it is determined that the elevation difference between the target elevation and the reference elevation is not less than the preset elevation difference and the reference elevation is not greater than the maximum elevation, the sum of the reference elevation and the elevation step length is used as a new reference elevation.
[0198] Further, the fault point screening module 14 is specifically used for:
[0199] From the positioning coordinates, the positioning coordinates whose elevation difference between the target elevation and the reference elevation corresponding to the target elevation is the smallest or whose elevation ratio is closest to a specified value are screened out.
[0200] In this embodiment, the actual elevation information within the preset range of the device under test is determined based on the collected multiple image samples, the image including the fault point corresponding to the device under test is obtained, and the camera parameter information corresponding to the image is determined. Based on the actual elevation information and the camera parameter information, the positioning coordinates of the device under test corresponding to the fault point are determined, and then the positioning coordinates that meet the preset coordinate rules are screened out from the positioning coordinates, and used as the actual fault point coordinates of the device under test corresponding to the fault point. Through the present invention, the actual coordinates of the fault point of the device can be located according to the fault point in the image including the fault point. Furthermore, when determining the actual fault point coordinates, the positioning coordinates that meet the preset coordinate rules are also screened out from the positioning coordinates, and used as the actual fault point coordinates of the device under test corresponding to the fault point, thereby improving the accuracy of determining the actual fault point coordinates.
[0201] It should be noted that, for the working process of each module, sub-module and unit in this embodiment, please refer to the corresponding description in the above embodiment, which will not be repeated here.
[0202] Optionally, based on the embodiments of the above-mentioned fault location method and device, another embodiment of the present invention provides a storage medium, wherein the storage medium includes a stored program, wherein when the program is running, the device where the storage medium is located is controlled to execute the above-mentioned fault location method.
[0203] Optionally, based on the above-mentioned embodiments of the fault location method and device, another embodiment of the present invention provides an electronic device, including: a memory and a processor;
[0204] Wherein, the memory is used to store programs;
[0205] The processor calls the program and is used to execute the above fault location method.
[0206] In this embodiment, the actual elevation information within the preset range of the device under test is determined based on the collected multiple image samples, the image including the fault point corresponding to the device under test is obtained, and the camera parameter information corresponding to the image is determined. Based on the actual elevation information and the camera parameter information, the positioning coordinates of the device under test corresponding to the fault point are determined, and then the positioning coordinates that meet the preset coordinate rules are screened out from the positioning coordinates, and used as the actual fault point coordinates of the device under test corresponding to the fault point. Through the present invention, the actual coordinates of the fault point of the device can be located according to the fault point in the image including the fault point. Furthermore, when determining the actual fault point coordinates, the positioning coordinates that meet the preset coordinate rules are also screened out from the positioning coordinates, and used as the actual fault point coordinates of the device under test corresponding to the fault point, thereby improving the accuracy of determining the actual fault point coordinates.
[0207] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A fault location method, It is characterized in that include: Acquire a plurality of collected image samples within a preset range of the device under test, and determine actual elevation information within the preset range of the device under test based on the plurality of image samples; The actual elevation information is digital surface model DSM information; Obtaining a picture including a fault point corresponding to the device under test, and determining camera parameter information corresponding to the picture; Based on the actual elevation information and the camera parameter information, determining the positioning coordinates of the device under test corresponding to the fault point; From the positioning coordinates, positioning coordinates that meet preset coordinate rules are screened out and used as actual fault point coordinates of the device under test corresponding to the fault point.
2. The fault location method according to claim 1, It is characterized in that Determining actual elevation information within a preset range of the device under test based on the multiple image samples includes: Extracting a camera parameter information sample from the image sample; Correcting the camera parameter information sample in the image sample to obtain reference camera parameter information corresponding to the image sample; Performing global dense matching on reference camera parameter information corresponding to the image samples to obtain a dense three-dimensional point cloud; An inverse distance weighted interpolation calculation is performed on the dense three-dimensional point cloud to obtain actual elevation information within a preset range of the device under test.
3. The fault location method according to claim 1, It is characterized in that Determining the camera parameter information corresponding to the image includes: Performing image matching between the image and the multiple image samples, determining an image sample corresponding to the image and using it as a target image sample; The reference camera parameter information corresponding to the target image sample is obtained and used as the camera parameter information corresponding to the image.
4. The fault location method according to claim 1, It is characterized in that Determining the location coordinates of the device under test corresponding to the fault point based on the actual elevation information and the camera parameter information includes: Acquire a collinear equation, wherein the collinear equation includes an association relationship between the camera parameter information and the positioning coordinates of the device under test corresponding to the fault point; Determine iteration parameters; the iteration parameters include maximum elevation, minimum elevation and elevation step length; Based on the iteration parameters and the collinearity equation, an iterative calculation is performed to obtain the location coordinates of the device under test corresponding to the fault point.
5. The fault location method according to claim 4, It is characterized in that Obtain collinear equations, including: Obtaining a basic collinearity equation; the basic collinearity equation includes: the image coordinates of the fault point in the picture including the fault point, the camera parameter information, the positioning coordinates of the device under test corresponding to the fault point, and the correlation relationship between the specified coefficients; Calculating the value of the specified coefficient according to the camera posture information in the camera parameter information; Substituting the numerical value of the designated coefficient into the basic collinear equation, a collinear equation is obtained.
6. The fault location method according to claim 4, It is characterized in that Determine the iteration parameters, including: Determine a location point corresponding to the camera parameter information in the actual elevation information; Filtering out the maximum elevation and the minimum elevation within a specified range with the location point as a reference point from the actual elevation information; Gets the preset elevation step size.
7. The fault location method according to claim 4, It is characterized in that Based on the iteration parameter and the collinearity equation, iterative calculation is performed to obtain the location coordinates of the device under test corresponding to the fault point, including: setting the reference elevation to the minimum elevation; According to the collinear equation, the horizontal and vertical coordinate points of the positioning coordinates corresponding to the reference elevation are calculated, and the target elevation corresponding to the horizontal and vertical coordinate points in the actual elevation information is determined; Determining whether the elevation difference between the target elevation and the reference elevation is less than a preset elevation difference; If it is less than, the preset elevation difference is updated to the elevation difference, and the position of the device to be tested corresponding to the target elevation and the horizontal and vertical coordinate points is used as the positioning coordinates; The sum of the reference elevation and the elevation step is used as a new reference elevation, and the process returns to the step of calculating the horizontal and vertical coordinate points of the positioning coordinates corresponding to the reference elevation according to the collinear equation, until the reference elevation is greater than the maximum elevation.
8. The fault location method according to claim 7, It is characterized in that Determining the target elevation corresponding to the horizontal and vertical coordinate points in the actual elevation information includes: Determine whether the horizontal and vertical coordinate points are located on the grid points in the actual elevation information; If not, a bilinear interpolation method is used to determine the target elevation corresponding to the horizontal and vertical coordinate points in the actual elevation information.
9. The fault location method according to claim 7, It is characterized in that In the case where it is determined that the elevation difference between the target elevation and the reference elevation is not less than a preset elevation difference, the method further includes: The sum of the reference elevation and the elevation step is used as a new reference elevation, and the process returns to the step of calculating the horizontal and vertical coordinate points of the positioning coordinates corresponding to the reference elevation according to the collinear equation, until the reference elevation is greater than the maximum elevation.
10. The fault location method according to claim 7, It is characterized in that The method further comprises: selecting positioning coordinates that meet preset coordinate rules from the positioning coordinates and using the positioning coordinates as the actual fault point coordinates of the device under test corresponding to the fault point. From the positioning coordinates, the positioning coordinates whose elevation difference between the target elevation and the reference elevation corresponding to the target elevation is the smallest or whose elevation ratio is closest to a specified value are screened out.
11. A fault location device, It is characterized in that include: An elevation determination module, used to obtain a plurality of collected image samples within a preset range of the device under test, and determine actual elevation information within the preset range of the device under test based on the plurality of image samples; The actual elevation information is digital surface model DSM information; An information determination module, used to obtain a picture including a fault point corresponding to the device under test, and determine camera parameter information corresponding to the picture; A fault point determination module, used to determine the positioning coordinates of the device under test corresponding to the fault point based on the actual elevation information and the camera parameter information; The fault point screening module is used to screen out the positioning coordinates that meet the preset coordinate rules from the positioning coordinates and use them as the actual fault point coordinates of the device under test corresponding to the fault point.
12. A storage medium, It is characterized in that The storage medium includes a stored program, wherein when the program is running, the device where the storage medium is located is controlled to execute the fault location method according to any one of claims 1 to 10.
13. An electronic device, It is characterized in that include: Memory and processor; Wherein, the memory is used to store programs; The processor calls the program and is used to execute the fault location method according to any one of claims 1 to 10.
