A dual-light pairing positioning method and system for inspecting defects of a photovoltaic module

Abstract

The application relates to the technical field of photovoltaic inspection, and discloses a dual-light pairing positioning method and system for photovoltaic module inspection defects, the method comprising the following steps: generating a dual-light associated image group by pairing a time group string with a hardware exposure time stamp, correcting dual-light coordinates in a Cartesian three-dimensional rectangular coordinate system and converting a defect three-dimensional space coordinate point by using a space displacement compensation vector, positioning the defect to a target component physical identifier based on a surrounding number topology structure and issuing an operation and maintenance work order; the application eliminates associated misplacement through double physical recording, compensates shutter time difference dispersion by using a three-dimensional rectangular coordinate system and avoids plane projection edge drift, and realizes a complete closed loop from exposure compensation to physical device positioning.

Description

Technical Field

[0001] This invention relates to the field of photovoltaic inspection technology, and in particular to a dual-light pairing positioning method and system for inspecting defects in photovoltaic modules. Background Technology

[0002] With the widespread application of UAV dual-light inspection technology in large-scale photovoltaic power plants, joint inspection using visible light and infrared images has become the primary means of identifying defects such as surface damage and hot spots on photovoltaic modules. This technology uses image recognition algorithms to detect defect targets from both visible light and infrared images, and then maps the defect coordinates to a polygon of the photovoltaic module in a geographic information system based on UAV positioning data. This enables physical anchoring of defects and issuance of maintenance work orders. Conventional mapping processes typically employ point-to-surface geometric comparison methods in a planar coordinate system, converting image pixel coordinates to geographic coordinates and then comparing them with the module's bounding box to determine inclusion relationships, thus automatically identifying the defect's location.

[0003] In practical applications, the geographical coordinates of the same physical defect calculated from visible light and infrared images can diverge spatially, causing the system to incorrectly anchor it to adjacent photovoltaic modules. For example, when a drone photographs the boundary area of ​​a photovoltaic array, the infrared image identifies a hot spot defect on a module, while the paired visible light image identifies a surface damage defect at the same location. When the system maps these defect coordinates to ground modules, the latitude and longitude of the hot spot in the infrared image are assigned to the quadrilateral area of ​​module ZJ01, while the latitude and longitude of the surface damage in the visible light image are assigned to the area of ​​the adjacent module ZJ02. This phenomenon means that when maintenance personnel go to ZJ01 to check for surface damage defects based on work orders, they cannot find the corresponding defect target on that module, causing the logic of dual-light joint confirmation of the same module's anomaly to fail. Ultimately, the issued maintenance work order has an incorrect physical location, resulting in wasted time for on-site inspection personnel and maintenance resources. Therefore, how to reduce the cross-module anchoring deviation of the same defect during the coordinate mapping process of dual-light inspection is a problem that urgently needs to be solved in this field. Summary of the Invention

[0004] To address the shortcomings of existing technologies, the present invention aims to provide a dual-light pairing and positioning method for defects in photovoltaic modules. This method involves using a drone equipped with a dual-light camera to simultaneously acquire visible light and infrared images and generating associated image groups based on hardware exposure timestamps. By combining spatial displacement compensation and orbital number topology, the method can accurately locate defects, achieving the technical effect of automatically associating dual-light images, correcting coordinates, and outputting target module identification to issue maintenance work orders.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, the present invention provides a dual-light pairing method for locating defects in photovoltaic modules during inspection, comprising: A drone equipped with a dual-light camera simultaneously acquires visible light image file sequences and infrared image file sequences of a photovoltaic array, extracts the corresponding hardware exposure timestamps, and combines them in a time sequence to generate a dual-light associated image group. Pixel-level defect detection boxes and initial defect type codes are extracted from the dual-light correlated image group. Spatial displacement compensation vectors are calculated using the hardware exposure timestamps and converted into three-dimensional spatial coordinate points of the defects using a three-dimensional rectangular coordinate system. Based on the surrounding number topology, the geometric relationship between the three-dimensional spatial coordinates of the defect and the quadrilateral vertex sequence of the photovoltaic module is calculated, the physical identifier of the target module is output, and an operation and maintenance work order is issued in conjunction with the initial defect type code.

[0006] Furthermore, the step of extracting the corresponding hardware exposure timestamps and combining them in a time sequence to generate a dual-light correlated image group includes: Read the underlying hardware exposure timestamps of visible light image file sequences and infrared image file sequences, and extract the time grouping strings from the file names; The visible light image file sequence and the infrared image file sequence are paired and merged according to the time grouping string, and a dual-light associated image group is generated by combining the hardware exposure timestamp.

[0007] Furthermore, the step of reading the underlying hardware exposure timestamps of the visible light image file sequence and the infrared image file sequence, and extracting the time grouping string from the filename includes: Parse the exchangeable image file format metadata segment of each image file, read the exposure time field written by the camera imaging sensor at the moment the shutter opens and closes, and obtain the hardware exposure timestamp in milliseconds. The filenames are matched using regular expressions, the type identifier characters and file extensions at the end of the filenames are removed, and the continuous substrings obtained by matching are used as time group strings; The time grouping string, type identifier character, and hardware exposure timestamp are written into the visible light mapping table and infrared mapping table one by one according to the image file.

[0008] Further, the step of pairing and merging the visible light image file sequence and the infrared image file sequence according to the time grouping string, and generating a dual-light associated image group by combining the hardware exposure timestamp includes: Iterate through the deduplicated set of all time group strings, and for each currently iterated time group string, retrieve the corresponding visible light image file path and infrared image file path from the visible light mapping table and infrared mapping table, respectively. The absolute value of the difference between the hardware exposure timestamp corresponding to the extracted visible light image file path and the hardware exposure timestamp corresponding to the infrared image file path is calculated to obtain the exposure timestamp deviation. If the exposure timestamp deviation is less than the dual-light simultaneous shooting determination threshold, then the visible light image file and the infrared image file are determined to be dual-light photos taken at the same time and are paired and merged, and written into a dual-light associated image group named with the time grouping string of the currently traversed time.

[0009] Further, the step of extracting pixel-level defect detection boxes and initial defect type codes from the dual-light correlated image group, calculating spatial displacement compensation vectors using the hardware exposure timestamps, and converting them into three-dimensional spatial coordinate points of the defects using a three-dimensional Cartesian coordinate system includes: The dual-light correlated image group is input into the image feature extraction unit, which outputs the pixel-level defect detection box and the initial defect type code corresponding to the visible light image file sequence and the infrared image file sequence, respectively. Calculate the hardware exposure timestamp difference between the visible light image file sequence and the infrared image file sequence in the dual-light correlated image group, and multiply the hardware exposure timestamp difference by the instantaneous flight speed output by the UAV inertial navigation unit to obtain the spatial displacement compensation vector. The latitude and longitude sensor values ​​of the UAV gimbal are obtained, and the latitude and longitude sensor values ​​are converted into Cartesian three-dimensional rectangular coordinate values ​​through a spherical projection algorithm. The center pixel of the pixel-level defect detection box is mapped and superimposed onto the Cartesian three-dimensional rectangular coordinate values. The Cartesian three-dimensional rectangular coordinates are added to the spatial displacement compensation vector to correct the coordinates. The corrected Cartesian three-dimensional rectangular coordinates are then converted back to geographic latitude and longitude values ​​through reverse projection to obtain the three-dimensional spatial coordinates of the defect.

[0010] Further, the step of acquiring the latitude and longitude sensor values ​​of the UAV gimbal, converting the latitude and longitude sensor values ​​into Cartesian three-dimensional rectangular coordinate values ​​through a spherical projection algorithm, and mapping and superimposing the center pixel of the pixel-level defect detection box onto the Cartesian three-dimensional rectangular coordinate values ​​includes: The latitude and longitude sensor values, including the latitude and longitude values ​​of the gimbal lens's sub-satellite point at the shooting time, are obtained through the gimbal positioning module. The Cartesian three-dimensional rectangular coordinate values ​​are then calculated using a spherical projection algorithm with the Earth's average radius as the spherical radius. Calculate the x and y coordinates of the center point pixel of the pixel-level defect detection box, subtract the coordinates of the principal point of the image to obtain the pixel offset, and then perform yaw angle rotation correction. The pixel offset result after yaw angle rotation correction is multiplied by the pixel size and combined with the flight altitude and lens focal length to convert it into the eastward and northward components of ground offset. These components are then multiplied by their corresponding unit vectors and superimposed onto the Cartesian three-dimensional rectangular coordinate value.

[0011] Furthermore, the simultaneous acquisition of visible light image file sequences and infrared image file sequences of the photovoltaic array by a drone equipped with a dual-light camera includes: Read the preset waypoints from the flight mission management module. The preset waypoints are generated by the geographical coordinates of the four corners of the photovoltaic array and the combined points according to the preset heading overlap rate and lateral overlap rate. The dual-light camera on the control drone takes pictures of the photovoltaic array according to the preset waypoints. At each preset waypoint, the visible light lens and the infrared thermal imaging lens are triggered to take pictures, so as to obtain the visible light image file sequence and the infrared image file sequence with corresponding identification characters respectively.

[0012] Furthermore, the calculation of the geometric relationship between the three-dimensional spatial coordinates of the defect and the sequence of quadrilateral vertices of the photovoltaic module based on the wrap-around number topology, and the output of the target module physical identifier, includes: Read the sequence of quadrilateral vertices of the photovoltaic modules corresponding to the current inspection area from the storage module; Each physical boundary line segment of the quadrilateral vertex sequence of the photovoltaic module is extracted sequentially. The cross product of the three-dimensional spatial coordinates of the defect relative to each physical boundary line segment is obtained by calculating the three-point turning direction. The positive and negative values ​​of the cross product of all physical boundary line segments are accumulated to obtain the topology wrapping value. If the topology surround value is not equal to zero, it is determined that the three-dimensional spatial coordinate point of the defect is located inside the polygon of the current photovoltaic module, and the built-in device code corresponding to the current photovoltaic module is used as the physical identifier of the target module. If the topology surround value is equal to zero, calculate the shortest Euclidean distance from the three-dimensional spatial coordinate point of the defect to all physical boundary line segments, and select the built-in device code corresponding to the shortest Euclidean distance as the physical identifier of the target component.

[0013] Further, the step of sequentially extracting each physical boundary segment of the quadrilateral vertex sequence of the photovoltaic module, obtaining the vector cross product of the defect's three-dimensional spatial coordinates with respect to each physical boundary segment by calculating the three-point turning direction, and accumulating the positive and negative values ​​of the cross product of all physical boundary segments to obtain the topology wrapping value includes: For each currently traversed physical boundary line segment, extract the latitude and longitude difference components of the vector from the starting vertex to the ending vertex and the vector from the starting vertex to the three-dimensional spatial coordinate point of the defect to calculate the cross product of the vectors in the turning direction. When the physical boundary line segment crosses the latitude of the three-dimensional spatial coordinate point of the defect from bottom to top and the cross product of the vector is positive, the topology wrap value is incremented by one; when the physical boundary line segment crosses the latitude of the three-dimensional spatial coordinate point of the defect from top to bottom and the cross product of the vector is negative, the topology wrap value is decremented by one. The final topology wrap value is obtained by summing the contributions of all physical boundary segments.

[0014] Furthermore, calculating the shortest Euclidean distance from the three-dimensional spatial coordinate point of the defect to all physical boundary line segments includes: For each physical boundary segment of each photovoltaic module, the offset vector pointing from the starting vertex of the physical boundary segment to the three-dimensional spatial coordinate point of the defect is multiplied by the boundary vector pointing from the starting vertex to the ending vertex, and then divided by the square of the boundary vector magnitude to obtain the projection parameter. The projection parameters are restricted to the range of zero to one so that the projection point falls on the physical boundary line segment; Calculate the distance between the three-dimensional spatial coordinates of the defect and the projection point after limiting the projection parameters, and use it as the shortest Euclidean distance of the corresponding physical boundary line segment.

[0015] Furthermore, the issuance of maintenance work orders in conjunction with the initial defect type code includes: The initial defect type code is numerically replaced according to a preset digital mapping rule to convert it into a unified defect type code. All unified defect type codes corresponding to the visible light image file sequence and infrared image file sequence bound under the same target component physical identifier are extracted. For all unified defect type codes under the same target component physical identifier, the defect handling mark is judged one by one. If any unified defect type code indicates an abnormal state, a dual-light confirmation signal is output to confirm the abnormality. For the physical identifier of the target component that outputs the judgment signal, the physical identifier of the target component, the unified defect type code indicating the abnormal state, the defect level parameter, and the three-dimensional spatial coordinates of the defect are combined and packaged to generate an operation and maintenance work order and sent to the operation and maintenance terminal.

[0016] Secondly, the present invention provides a dual-light pairing positioning system for inspecting defects in photovoltaic modules, comprising: The dual-light correlated image group generation module is used to simultaneously acquire visible light image file sequences and infrared image file sequences of photovoltaic arrays by using a drone equipped with a dual-light camera, extract the corresponding hardware exposure timestamps and combine them in time sequence to generate dual-light correlated image groups; The defect three-dimensional spatial coordinate correction module is used to extract pixel-level defect detection boxes and initial defect type codes from the dual-light correlated image group, calculate spatial displacement compensation vectors using the hardware exposure timestamps, and convert them into defect three-dimensional spatial coordinate points using a three-dimensional rectangular coordinate system. The target component location and work order issuance module is used to calculate the geometric relationship between the three-dimensional spatial coordinates of the defect and the quadrilateral vertex sequence of the photovoltaic module based on the surrounding number topology, output the physical identifier of the target component, and issue an operation and maintenance work order in conjunction with the initial defect type code.

[0017] Compared with existing technologies, the beneficial effects of this invention are as follows: This invention generates a dual-light associated image group with consistent physical time by using time grouping strings and hardware exposure timestamps as dual physical records, cutting off the transmission path from the source of association misalignment caused by disordered reading and writing at the underlying level to subsequent coordinate calculation; it directly eliminates the dual-light coordinate discrepancy caused by shutter time difference in the Cartesian three-dimensional rectangular coordinate system by using spatial displacement compensation vectors, and avoids the edge drift error caused by planar projection in the longitude range by utilizing the characteristic that the three-dimensional rectangular coordinate system does not depend on the convergence rate of meridians, so that the three-dimensional spatial coordinate points of the same physical defect in the visible light and infrared dimensions remain spatially coincident. Based on the loop number topology and the shortest Euclidean distance fallback strategy, the system achieves accurate geometric inclusion determination of defect points in irregular component polygons and robust attribution in boundary detachment scenarios. By using a unified defect type encoding on the physical identification dimension of the target component, the joint determination logic is directly bound to the physical device object, ensuring that even when a single light image is missing, the system can still complete the anomaly determination and work order issuance based on the objective component dimension. This avoids determination deadlock caused by image breakage, thereby improving the overall accuracy and fault tolerance of photovoltaic power station image defect recognition and location. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart of a dual-light pairing method for locating defects in photovoltaic modules according to the present invention; Figure 2 This is a schematic diagram of the photovoltaic array coverage in an embodiment of the present invention; Figure 3 This is a schematic diagram of the visible light mapping table and infrared mapping representation in an embodiment of the present invention; Figure 4 This is a schematic diagram of yaw angle rotation correction in an embodiment of the present invention; Figure 5 This is a schematic diagram of the spatial displacement compensation vector in an embodiment of the present invention; Figure 6 This is a schematic diagram showing the arrangement of the vertices of the quadrilateral in an embodiment of the present invention; Figure 7 This is a schematic diagram of the vector cross product in an embodiment of the present invention; Figure 8 This is a schematic diagram of the shortest Euclidean distance in an embodiment of the present invention; Figure 9 This is a schematic diagram of the digital mapping rules in an embodiment of the present invention; Figure 10 This is a functional block diagram of a dual-light pairing positioning system for inspecting defects in photovoltaic modules according to the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] Example 1: Please see Figure 1 As shown, this embodiment provides a dual-light pairing method for locating defects in photovoltaic modules during inspection, including: This embodiment uses a dual-light camera from a drone equipped with a visible light lens and an infrared thermal imaging lens as the data acquisition front-end, deployed on a server, to describe the working condition of dual-light defect localization divergence in the boundary area of ​​a large photovoltaic power station array. The entire process is executed sequentially from step S1 to step S3.

[0022] S1: A drone equipped with a dual-light camera synchronously acquires visible light image file sequences and infrared image file sequences of the photovoltaic array, extracts the corresponding hardware exposure timestamps, combines them in time sequence, and generates a dual-light associated image group.

[0023] In the scenario of dual-light inspection of photovoltaic power plants using drones, the visible light lens and the infrared thermal imaging lens are mounted on the same gimbal, but the shutter triggering circuits of the two lenses are independent, resulting in millisecond-level differences in exposure timing. Simultaneously, the drone continuously shifts along its flight path, causing the same physical defect to be recorded as two independent data sets with different hardware exposure timestamps in the visible light and infrared image file sequences. If the visible light and infrared image file sequences are matched solely based on the filename characters in the file system, the disordered read / write operations of the camera's underlying storage circuit during high-frequency continuous shooting will result in two images with the same name not being taken at the same time, leading to the coordinate calculation in subsequent step S2 being based on incorrect image pairing. Therefore, step S1 must simultaneously extract two types of physical records: the hardware exposure timestamp and the time grouping string in the filename, as the basis for constructing the dual-light associated image group.

[0024] Further, step S1 includes: S11: Control the drone equipped with a dual-light camera to take pictures of the photovoltaic array according to the preset waypoints, and obtain visible light image file sequences and infrared image file sequences.

[0025] In the photovoltaic array coverage shooting mission, the photovoltaic modules are arranged in a regular array on the ground. The drone must hover or pass through the pre-planned route at equal intervals and trigger the shooting. This arrangement characteristic determines that shooting driven by waypoint sequence can ensure that the modules are completely covered and that there is a fixed overlapping area between adjacent photos.

[0026] Specifically, the dual-light camera on the drone consists of a visible light lens and an infrared thermal imaging lens. The two lenses are fixedly mounted on the same three-axis stabilization gimbal, which is locked in a downward-looking posture at the time of shooting.

[0027] The UAV flight control unit reads preset waypoints from the flight mission management module. The preset waypoints refer to a set of latitude, longitude and flight altitude points generated in the flight route management module based on the geographical coordinates of the four corners of the photovoltaic array and the preset forward overlap rate and lateral overlap rate. For example, the forward overlap rate is 0.8 and the lateral overlap rate is 0.7.

[0028] When the drone reaches a preset waypoint, the dual-light camera simultaneously triggers the visible light lens and the infrared thermal imaging lens to each take a picture. The visible light lens outputs an image file with a visible light type identifier and appends it to the visible light image file sequence, while the infrared thermal imaging lens outputs an image file with an infrared type identifier and appends it to the infrared image file sequence.

[0029] The filename of each image file in the visible light image file sequence and the infrared image file sequence is formed by concatenating the device prefix character, the 8-digit date character, the serial number character, and the type identifier character in sequence. The type identifier character for the visible light image file is V, and the type identifier character for the infrared image file is T. For example, the visible light image file name is "device prefix 202507020001V", and the infrared image file name is "device prefix 202507020001T".

[0030] S12: Read the underlying hardware exposure timestamps of the visible light image file sequence and infrared image file sequence, and extract the time grouping string from the file name.

[0031] The visible light image file sequence and infrared image file sequence output in step S11 only carry sequence characters at the file name level. The time identifier that can be used for cross-sequence comparison has not yet been extracted. Furthermore, the file name sequence characters may not be consistent with the actual shooting order due to the influence of the camera's continuous shooting storage mechanism. If the file name is directly passed to step S13, it will be impossible to confirm whether the two photos are taken at the same time in the physical time dimension. This step supplements the reading of the underlying hardware exposure timestamp and the extraction of time grouping strings.

[0032] Specifically, the hardware exposure timestamp is read as follows: parsing the exchangeable image file format metadata segment of each image file, reading the exposure time field written by the camera imaging sensor at the moment the shutter opens and closes, and obtaining the hardware exposure timestamp in milliseconds.

[0033] The extraction method for the time group string is as follows: the file name is matched with a regular expression. The regular expression matches a continuous substring composed of device prefix characters, 8-digit date characters, and serial number characters. The type identifier character and file extension at the end of the file name are removed, and the matched continuous substring is used as the time group string. For example, the time group string extracted from the visible light image file name "device prefix 202507020001V" and the infrared image file name "device prefix 202507020001T" is "device prefix 202507020001".

[0034] The method for extracting the type identifier character is as follows: First, read the second to last character after removing the extension from the file name. If the second to last character is V or T, it is used as the type identifier character. If the second to last character is not V or T, then search for the substring "V" or "T" in the file name. If the search is successful, take the V or T in the substring as the type identifier character. If neither method is successful, then mark the type identifier character as an unknown type.

[0035] Write the time group string, type identifier character and hardware exposure timestamp into the visible light map table and infrared map table one by one according to the image file. The visible light map table uses the time group string as the key and the visible light image file path and hardware exposure timestamp as the value. The infrared map table uses the time group string as the key and the infrared image file path and hardware exposure timestamp as the value. At the same time, output the deduplicated set of all time group strings.

[0036] S13: Based on the time grouping string, the visible light image file sequence and the infrared image file sequence are paired and merged, and combined with the hardware exposure timestamp to generate a dual-light associated image group.

[0037] During the inspection of large photovoltaic power plants, the camera storage medium is subject to the constraint of file write order disorder when writing continuously at high frequency. Simply matching the file name sequence characters will result in the mismatch between the sequence characters and the actual shooting time, leading to incorrect pairing of visible light photos and infrared photos from different times. Under this constraint, the common method of pairing by file name sorting will produce systematic correlation misalignment in the pairing and merging process. This step adds time proximity verification of hardware exposure timestamps on the basis of time group string matching.

[0038] Specifically, iterate through the deduplicated set of all time group strings output in step S12, and for each currently traversed time group string, retrieve the visible light image file path and infrared image file path with the currently traversed time group string as the key from the visible light mapping table and the infrared mapping table, respectively.

[0039] The absolute value of the difference between the hardware exposure timestamp corresponding to the path of the extracted visible light image file and the hardware exposure timestamp corresponding to the path of the infrared image file is calculated to obtain the exposure timestamp deviation. If the exposure timestamp deviation is less than the dual-light simultaneous shooting determination threshold, the visible light image file and the infrared image file are determined to be dual-light photos taken at the same shooting time and are paired and merged, and written into a dual-light associated image group named with the current traversed time grouping string. The method for determining the dual-light simultaneous shooting determination threshold is as follows: the maximum trigger delay of the shutter triggering circuit of the two lenses of the dual-light camera and the short buffer time of gimbal attitude stabilization when the UAV hovers and triggers shooting at a single waypoint are taken as the dual-light simultaneous shooting determination threshold. The short buffer time of attitude stabilization refers to the waiting time between the gimbal locking the downward attitude and the shutter being allowed to be triggered. For example, the dual-light simultaneous shooting determination threshold is 1 second.

[0040] If the current time group string exists only in the visible light map table but not in the infrared map table, or exists only in the infrared map table but not in the visible light map table, then construct a dual-light associated image group containing only a single photo using the existing single-sided image file path, and set the image file path field of the missing side to empty, to accommodate the case where one photo exists and the other does not in the dual-light photo group.

[0041] Each dual-light correlated image group is written in batches to the photo group storage module as a photo group entity. The data structure of the photo group entity is a record containing a photo group name field, a visible light photo identifier field, and an infrared photo identifier field. The photo group name field takes the time group string as its value.

[0042] In the boundary area of ​​the photovoltaic array, the same physical defect is located at the edge of the module polygon. Once a pairing misalignment occurs, it will directly cause step S2 to perform coordinate calculation on the incorrect image pair. The small edge deviation is amplified into a positioning jump across modules. This step uses both time grouping strings and hardware exposure timestamps as dual physical records to constrain pairing and merging: the time grouping strings ensure that the two photos participating in the pairing come from the same sequence number period, while the time proximity check of the hardware exposure timestamps further eliminates the situation where the same sequence number character corresponds to different shooting times due to camera disk loading disorder. The two types of physical records act on the file name character layer and the imaging sensor physical time layer, respectively. The disk loading disorder that cannot be detected by relying solely on file name character matching is truncated in the hardware exposure timestamp verification stage, making the dual-light associated image group entering step S2 truly consistent in terms of physical shooting time. This cuts off the transmission path from the associated misalignment caused by the underlying read / write disorder to the subsequent coordinate calculation from the source.

[0043] S2: Extract pixel-level defect detection boxes and initial defect type codes from the dual-light correlated image group, calculate the spatial displacement compensation vector using the hardware exposure timestamp, and convert it into the three-dimensional spatial coordinate points of the defect using the three-dimensional rectangular coordinate system.

[0044] While the dual-light correlated image set output in step S1 is consistent in terms of physical shooting time, there is still a millisecond-level difference in shutter triggering between the visible light lens and the infrared thermal imaging lens. The drone undergoes actual displacement along its flight direction between the two exposures, resulting in physical differences in the drone's spatial pose corresponding to the same physical defect in the visible light image and the infrared image. If a simple point-plane projection is performed on the center point of the detection box in each image using a conventional planar coordinate system, the same physical defect located at the edge of the component polygon will fall into adjacent different component polygons under the visible light and infrared projections, respectively. Furthermore, the edge error of the planar projection is nonlinearly amplified when shooting across a large longitude range. Therefore, pixel-level defect detection boxes and initial defect type codes are first extracted, then a spatial displacement compensation vector is constructed using the hardware exposure timestamp difference, and coordinate correction is performed in a three-dimensional Cartesian coordinate system.

[0045] Further, step S2 includes: S21: Input the dual-light correlated image group into the image feature extraction unit, and output the pixel-level defect detection box and initial defect type code corresponding to the visible light image file sequence and the infrared image file sequence, respectively.

[0046] Subsequent steps S22 and S23 require geometric transformation with the pixel position of the defect in the image as input. The dual-light associated image group output by step S1 is only the original image file and does not yet contain the position and category information of the defect in the image coordinate system. Therefore, pixel-level defect detection boxes and initial defect type codes must be extracted from the image first.

[0047] Specifically, the image feature extraction unit is a target detection network deployed on the defect recognition server. The target detection network is based on the YOLOv8 target detection model and trained on a photovoltaic defect image dataset. The training dataset covers infrared defect samples such as open circuit, diode, string non-tracking, hot spot, dust accumulation, and missing components, as well as visible light defect samples such as missing components and weeds. The output is a pixel-level defect detection box and an initial defect type code corresponding to each defect.

[0048] The input of the dual-light correlated image group to the image feature extraction unit supports two methods: For a large number of photos, the image files in the dual-light correlated image group are uploaded in batches to the file transfer protocol path of the defect recognition service via the file transfer protocol, and then the hypertext transfer protocol interface is called to notify the defect recognition service to start recognition. The interface parameters include the file transfer protocol path, model type, focal length, and pixel size; For real-time recognition of a single photo, the image file is encoded into a Base64 string and then uploaded via the hypertext transfer protocol interface.

[0049] The defect identification service adopts an asynchronous polling task status mechanism, with a maximum of 10 polls and a 2-second interval between each poll. During the polling period, the task status field is read, and the task status field indicates that the identification result is returned through a callback method after the identification is completed.

[0050] The data structure of the pixel-level defect detection box is a record containing four pixel values: the X coordinate of the top left corner of the detection box, the Y coordinate of the top left corner of the detection box, the width of the detection box, and the height of the detection box. The initial defect type code is either an infrared defect type code with a value range of 0 to 5 or a visible light defect type code with a value range of 0 to 1.

[0051] S22: Calculate the hardware exposure timestamp difference between the visible light image file sequence and the infrared image file sequence in the dual-light correlated image group, and multiply the hardware exposure timestamp difference by the instantaneous flight speed output by the UAV inertial navigation unit to obtain the spatial displacement compensation vector.

[0052] The visible light image file sequence and the infrared image file sequence output in step S21 correspond to their respective pixel-level defect detection boxes. Steps S22 to S24 must be executed independently once each with the visible light detection box and the infrared detection box as input, to obtain two sets of results: the three-dimensional spatial coordinate points of the visible light defect and the three-dimensional spatial coordinate points of the infrared defect. The spatial displacement compensation vector is only superimposed on the Cartesian three-dimensional rectangular coordinate value corresponding to the side with the later hardware exposure timestamp (i.e., the image file with the later exposure time), with the spatial pose of the side with the earlier exposure time as the reference, and the other side is not superimposed.

[0053] The independent physical characteristics of the shutter trigger circuits of the two lenses of the dual-light camera determine that the imaging moments of the visible light image file and the infrared image file in the same dual-light associated image group do not coincide. The UAV continuously moves at the inspection flight speed between the two imagings. This displacement is exactly equal to the product of the time difference and the flight speed. Therefore, the actual displacement of the UAV between the two imagings can be accurately reconstructed by the product of the hardware exposure timestamp difference and the instantaneous flight speed.

[0054] Specifically, the corresponding hardware exposure timestamps are read back from the visible light image identifier field and infrared image identifier field written in step S1 of the dual-light associated image group. The difference between the hardware exposure timestamp of the infrared image file and the hardware exposure timestamp of the visible light image file is calculated to obtain the hardware exposure timestamp difference, which is a signed millisecond value.

[0055] The instantaneous flight speed is obtained from the output interface of the UAV inertial navigation unit. The instantaneous flight speed is the horizontal velocity vector composed of the eastward velocity component and the northward velocity component output by the UAV inertial navigation unit at the time of shooting.

[0056] The spatial displacement compensation vector is calculated as follows: after converting the hardware exposure timestamp difference into a value in seconds, it is multiplied by the eastward velocity component and the northward velocity component of the instantaneous flight speed, respectively, to obtain the eastward displacement component and the northward displacement component. The eastward displacement component and the northward displacement component together form the spatial displacement compensation vector.

[0057] For example, assuming the hardware exposure timestamp difference is 0.2 seconds, the eastward velocity component of the instantaneous flight speed is 7 meters per second and the northward velocity component is 3 meters per second, then the eastward displacement component is 0.2 multiplied by 7 equals 1.4 meters, the northward displacement component is 0.2 multiplied by 3 equals 0.6 meters, and the spatial displacement compensation vector is 1.4 meters eastward and 0.6 meters northward.

[0058] S23: Obtain the latitude and longitude sensor values ​​of the UAV gimbal, convert the latitude and longitude sensor values ​​into Cartesian three-dimensional rectangular coordinate values ​​through a spherical projection algorithm, and map and superimpose the center point pixel of the pixel-level defect detection box onto the Cartesian three-dimensional rectangular coordinate values.

[0059] The constraints of photovoltaic power stations spanning a wide range of longitudes cause scale distortion in the planar coordinate system with the latitude and longitude of the shooting point as the origin at both ends of the array. The common planar rectangular coordinate superposition method causes deviations in the ground distance corresponding to the same pixel offset due to changes in the convergence rate of meridians when crossing a wide range of longitudes. Therefore, it is necessary to uniformly convert the latitude and longitude to the Cartesian three-dimensional rectangular coordinate system with the Earth's center as the origin, and complete the superposition of pixel offsets in the three-dimensional coordinate system.

[0060] Further, step S23 includes: S231: Obtain the latitude and longitude sensor values ​​of the UAV gimbal and convert them into Cartesian three-dimensional rectangular coordinate values ​​through a spherical projection algorithm.

[0061] Specifically, the latitude and longitude sensor values ​​of the drone gimbal are obtained from the gimbal positioning module, including the latitude and longitude values ​​of the gimbal lens's nadir point at the time of shooting.

[0062] The execution process of the spherical projection algorithm is as follows: using the average radius of the Earth as the radius of the sphere, and denoting the latitude value as... Longitude values ​​are denoted as Calculate the three components of the Cartesian three-dimensional rectangular coordinates using the following formula:

[0063] in The average radius of the Earth is taken as 6,371,393 meters. , , These are the three components of the Cartesian three-dimensional rectangular coordinates with the Earth's center as the origin.

[0064] For example, when the latitude value is 30.000050 degrees and the longitude value is 120.000060 degrees, Approximately 0.866025, Approximately 0.500001, Approximately , Approximately 0.866025, substituting into the equation yields... Approximately meters, approximately 4,779,850 meters Approximately 3,185,700 meters; the above values ​​are approximate estimates, and actual calculations should use the values ​​in the formula. , , Precise substitution is required.

[0065] S232: Map and overlay the center pixel of the pixel-level defect detection box onto the Cartesian three-dimensional rectangular coordinate value.

[0066] The Cartesian 3D rectangular coordinates output in step S231 correspond to the nadir position of the gimbal lens. The pixel-level defect detection box output in step S21 indicates that the defect is offset relative to the principal point of the image plane. If the nadir coordinates are directly used as the defect coordinates, defects at the image edge will have a positioning error proportional to the offset. Therefore, this step maps the center pixel of the pixel-level defect detection box to the ground offset and then superimposes it onto the Cartesian 3D rectangular coordinates. Furthermore, when the UAV performs shooting tasks along different sides of a bow-shaped flight path, the gimbal yaw angle changes with the flight direction, and the image coordinate axes are not always aligned with the geographic east and north directions. If no yaw angle rotation correction is applied to the pixel offset, directly equating the image's horizontal and vertical axes to the geographic east and north directions will produce directional errors in non-northward flight paths. Therefore, this step performs yaw angle rotation correction on the pixel offset before converting the ground offset.

[0067] Specifically, the center point pixel is calculated as follows: the x-coordinate of the center point pixel is the sum of the x-coordinate of the top left corner of the pixel-level defect detection box and half the width of the detection box; the y-coordinate of the center point pixel is the sum of the y-coordinate of the top left corner of the pixel-level defect detection box and half the height of the detection box.

[0068] The ground offset is calculated as follows: subtract the x and y coordinates of the image principal point from the x and y coordinates of the center point pixel, respectively, to obtain the pixel offset x and y components; obtain the yaw angle of the UAV gimbal at the time of shooting, where the yaw angle is the clockwise angle between the horizontal axis of the camera imaging coordinate system and the geographic true north direction, denoted as . Perform a two-dimensional rotation transformation on the horizontal and vertical components of the pixel offset using the following formula:

[0069] in , These are the horizontal and vertical components of the pixel offset. , The horizontal and vertical components of the rotated pixel offset are given. The horizontal and vertical components of the rotated pixel offset are multiplied by the pixel size and then by the flight altitude of the gimbal relative to the surface of the photovoltaic module. The result is divided by the lens focal length to obtain the eastward and northward components of the ground offset. The pixel size and lens focal length are inherent parameters of the dual-light camera and are read from the defect identification interface parameters. The flight altitude is obtained from the fusion output of the UAV barometric altimeter and the laser rangefinder module. The yaw angle is obtained from the attitude sensor output interface of the UAV gimbal.

[0070] The method for superimposing the ground offset to Cartesian three-dimensional rectangular coordinates is as follows: using the latitude value of the sub-satellite point... With longitude value Construct the components of the eastward and northward unit vectors in a Cartesian three-dimensional rectangular coordinate system. The eastward unit vector is... The northward unit vector is The eastward component of the ground offset is multiplied by the eastward unit vector, and the northward component of the ground offset is multiplied by the northward unit vector. These components are then added to the Cartesian three-dimensional rectangular coordinates output in step S231 to obtain the Cartesian three-dimensional rectangular coordinates of the superimposed center point pixel.

[0071] S24: Perform vector addition on the Cartesian three-dimensional rectangular coordinates and the spatial displacement compensation vector to complete the coordinate correction. Then, restore the corrected Cartesian three-dimensional rectangular coordinates to geographic latitude and longitude values ​​through reverse projection to obtain the three-dimensional spatial coordinates of the defect.

[0072] In the same dual-light image set, the visible light image and the infrared image correspond to different UAV spatial poses at the two imaging moments due to the physical time difference of the two lens shutters. If this pose difference is not eliminated, the latitude and longitude of the same physical defect will spatially diverge under the visible light and infrared calculations, causing defects located at the edge of components to be anchored to different adjacent component polygons, thus causing the dual-light joint judgment logic to fail. Therefore, the spatial displacement compensation vector is used to directly add and correct the Cartesian three-dimensional rectangular coordinate values ​​in the three-dimensional coordinate system.

[0073] Specifically, the vector addition operation is performed as follows: the eastward displacement component of the spatial displacement compensation vector output in step S22 is multiplied by the eastward unit vector, and the northward displacement component is multiplied by the northward unit vector. Then, the components are added to the Cartesian three-dimensional rectangular coordinate value after superimposing the center point pixel output in step S232 to obtain the corrected Cartesian three-dimensional rectangular coordinate value. The construction method of the eastward unit vector and the northward unit vector is the same as that in step S232.

[0074] The reverse projection is performed as follows: the three components of the corrected Cartesian three-dimensional rectangular coordinates are denoted as... , , Restore geographical latitude and longitude values ​​using the following formula:

[0075] in The restored latitude value. The restored longitude value. The arctangent function is used in the four quadrants, and the three-dimensional spatial coordinates of the defect are determined by the restored latitude values. With longitude value composition.

[0076] For example, after the Cartesian three-dimensional rectangular coordinates obtained from infrared image calculation are corrected by superimposing a spatial displacement compensation vector of 1.4 meters in the east and 0.6 meters in the north, the deviation between the corrected Cartesian three-dimensional rectangular coordinates obtained from visible light image calculation and the Cartesian three-dimensional rectangular coordinates obtained from visible light image calculation in the east and north directions is reduced from about 1.5 meters before correction to the order of 0.05 meters. After reverse projection, the defect three-dimensional spatial coordinates of the two fall into the same component polygon.

[0077] In the boundary region of the photovoltaic array, the same physical defect is located at the edge of the polygonal module. The absolute displacement of the UAV caused by the shutter time difference between the two lenses of the dual-light system results in a discrepancy between the visible light and infrared settlement coordinates. Furthermore, when shooting across a large longitude range, conventional planar projection will amplify this discrepancy at the edge, turning it into a drift across the physical boundary. This step performs the additive correction of the spatial displacement compensation vector in a Cartesian three-dimensional rectangular coordinate system with the Earth's center as the origin. On the one hand, it uses the difference in hardware exposure timestamps and the real displacement reconstructed from the instantaneous flight speed to directly offset the pose difference between the two images, eliminating the discrepancy of the dual-light coordinates at the source. On the other hand, the three-dimensional rectangular coordinate system does not depend on the convergence rate of meridians, and the same displacement maintains a consistent measurement scale at both ends of the large longitude, avoiding the scale distortion of the planar projection in the longitude direction. Thus, while eliminating the physical time difference deviation in the underlying coordinate system, it unexpectedly also eliminates the edge drift error amplified by the planar projection over a large longitude range, ensuring that the three-dimensional spatial coordinates of the same physical defect in the visible light and infrared dimensions remain spatially coincident. The suppression of edge drift is not the original design goal of the spatial displacement compensation vector. The original goal of the spatial displacement compensation vector is only to offset the shutter speed difference displacement. However, its structure, which participates in the calculation in the form of a vector in the three-dimensional rectangular coordinate system, makes the compensation amount and the coordinate amount in the same scale-free distortion measurement space. Thus, it naturally avoids the edge magnification defect of planar projection without the need to introduce an additional projection correction module.

[0078] S3: Based on the wrap-around number topology, calculate the geometric relationship between the three-dimensional spatial coordinates of the defect and the quadrilateral vertex sequence of the photovoltaic module, output the physical identifier of the target module, and issue an operation and maintenance work order in conjunction with the initial defect type code.

[0079] The 3D spatial coordinates of the defect output in step S2 are geographic latitude and longitude values, which have not yet been correlated with specific photovoltaic modules on the ground. Since photovoltaic modules are distributed in irregular quadrilateral polygons in geographic space, simple latitude and longitude interval comparisons cannot determine whether the defect point falls within a specific irregular quadrilateral. Therefore, this step determines the geometric inclusion relationship between the 3D spatial coordinates of the defect and the quadrilateral vertex sequence of the photovoltaic module based on the surrounding number topology, accurately locating the defect to the target module. It also completes the unification and joint determination of dual-light defect types at the target module level, and finally issues a maintenance work order.

[0080] Further, step S3 includes: S31: Read the sequence of quadrilateral vertices of the photovoltaic module corresponding to the current inspection area from the storage module.

[0081] The subsequent step S32 requires extracting physical boundary segments sequentially using the ordered sequence of quadrilateral vertices of the photovoltaic module as input. However, the original input order of the module vertices recorded in the storage module is inconsistent. If vertices with inconsistent order are directly passed to step S32, the connection relationship of the physical boundary segments will intersect, and the result of the sum of the wrapping number will be distorted. Therefore, the vertex order must be unified after reading the vertices.

[0082] Specifically, the system reads the equipment installation area bound to the current inspection flight mission from the flight mission management module, and reads all photovoltaic arrays under the equipment installation area and all photovoltaic modules under each photovoltaic array from the component management module. Each photovoltaic module stores the coordinates of the four corner points, the coordinates of the center point, and the built-in device code.

[0083] The coordinates of the four corner points of each photovoltaic module are sorted. The coordinate sorting tool arranges the four vertices in the order of top left, top right, bottom right, and bottom left: first, the vertex with the largest latitude value is selected as the top left vertex, and if the latitude values ​​are the same, the vertex with the smallest longitude value is selected as the top left vertex; then, the azimuth angles of the other three vertices relative to the top left vertex are calculated based on the top left vertex, and the top right vertex, bottom right vertex, and bottom left vertex are obtained in order according to the azimuth angles. The vertex arrangement direction can be clockwise or counterclockwise, which does not affect the internal and external consistency judgment of the subsequent encirclement number determination; the sorted four vertices form the quadrilateral vertex sequence of the photovoltaic module.

[0084] The built-in device code is a physical identifier for photovoltaic modules, named according to four levels: transformer substation, inverter, string, and module. The naming format is to concatenate the transformer substation number, inverter number, string number, and module number in sequence. For example, the built-in device code is "NB01-XB02-ZC03-ZJ01".

[0085] S32: Extract each physical boundary line segment of the quadrilateral vertex sequence of the photovoltaic module in sequence, obtain the vector cross product of the defect three-dimensional space coordinate point with respect to each physical boundary line segment by calculating the three-point turning direction, and accumulate the positive and negative values ​​of the cross product of all physical boundary line segments to obtain the topology wrap value.

[0086] Under strong wind loads and installation tolerances, photovoltaic modules exhibit irregular quadrilateral shapes that are not strictly rectangular. The commonly used latitude and longitude bounded rectangle inclusion judgment will misclassify defect points outside the quadrilateral as internal points at the boundary of the irregular quadrilateral. However, the wrap-around number topology determines the directed wrap-around number of defect points by statistically analyzing the boundary, and can give the correct internal and external relationship for both irregular quadrilaterals and concave polygons. Therefore, this step uses the wrap-around number topology for geometric inclusion judgment.

[0087] Further, step S32 includes: S321: Extract each physical boundary line segment of the quadrilateral vertex sequence of the photovoltaic module in sequence.

[0088] The photovoltaic module quadrilateral vertex sequence output in step S31 is an ordered set of vertices. Subsequent cross product calculations must use the directed edges connecting adjacent vertices as the object. Here, the vertex sequence is converted into a physical boundary line segment sequence.

[0089] Specifically, adjacent vertices in the quadrilateral vertex sequence of the photovoltaic module are connected sequentially, and the last vertex is connected to the first vertex to form a closed boundary, resulting in four physical boundary segments. Each physical boundary segment consists of the latitude and longitude values ​​of the starting vertex and the ending vertex.

[0090] S322: Obtain the vector cross product of the three-dimensional spatial coordinates of the defect relative to each physical boundary line segment by calculating the three-point turning direction.

[0091] The physical boundary line segments output in step S321 only give the geometric position of the boundary and do not reflect which side of each physical boundary line segment the three-dimensional spatial coordinate point of the defect is located on. Therefore, this step quantifies the orientation of the three-dimensional spatial coordinate point of the defect relative to each physical boundary line segment by the vector cross product of the three-point turning directions.

[0092] Specifically, for each currently traversed physical boundary segment, the turning direction is calculated using the starting vertex, ending vertex, and the defect's three-dimensional spatial coordinate point: the vector pointing from the starting vertex to the ending vertex is denoted as... The difference in longitude is denoted as The latitude difference component is denoted as The vector pointing from the starting vertex to the three-dimensional coordinate point of the defect is denoted as... The longitude difference component is denoted as and the latitude difference component is denoted as . ;according to Calculate the cross product of vectors .

[0093] When determining whether a defect's 3D spatial coordinate point lies on the currently traversed physical boundary segment, a floating-point comparison error tolerance is used for the determination, i.e., when the vector cross product... When the absolute value of the error is less than the floating-point comparison error tolerance and the latitude and longitude of the defect's three-dimensional spatial coordinates fall within the range bounded by the latitude and longitude of the starting and ending vertices, the defect's three-dimensional spatial coordinates are determined to be located on the currently traversed physical boundary line segment; the floating-point comparison error tolerance is a fixed value. The value is based on the minimum resolvable angle level required for component-level positioning when latitude and longitude are in degrees.

[0094] For example, if the starting vertex of the physical boundary line segment is at latitude 30.000100 degrees and longitude 120.000000 degrees, and the ending vertex is at latitude 30.000100 degrees and longitude 120.000120 degrees, and the three-dimensional spatial coordinates of the defect are at latitude 30.000050 degrees and longitude 120.000060 degrees, then... , , , cross product of vectors A negative cross product indicates that the three-dimensional coordinate point of the defect is located to the right of the physical boundary line segment.

[0095] S323: Accumulate the positive and negative cross product values ​​of all physical boundary segments to obtain the topology wrap value.

[0096] The vector cross product of each physical boundary line segment output in step S322 only reflects the orientation of the three-dimensional spatial coordinate point of the defect relative to a single line segment. The directional wrapping effect of the four physical boundary line segments needs to be summarized in order to determine the overall inclusion relationship.

[0097] Specifically, the four physical boundary segments are traversed. Based on the relative relationship between the latitude values ​​of the starting and ending vertices of each physical boundary segment and the latitude value of the defect's three-dimensional spatial coordinate point, it is determined whether the physical boundary segment is crossed by the horizontal ray of the defect's three-dimensional spatial coordinate point: when the physical boundary segment crosses the latitude of the defect's three-dimensional spatial coordinate point from bottom to top and the vector cross product is positive, the topology encirclement value is incremented by 1; when the physical boundary segment crosses the latitude of the defect's three-dimensional spatial coordinate point from top to bottom and the vector cross product is negative, the topology encirclement value is decremented by 1; the contributions of the four physical boundary segments are accumulated to obtain the topology encirclement value.

[0098] For example, when the defect's three-dimensional spatial coordinates are located inside the component quadrilateral, the sum of the contributions of the four physical boundary line segments results in a topology wrap value of 1; when the defect's three-dimensional spatial coordinates are located outside the component quadrilateral, the sum of the topology wrap values ​​is 0.

[0099] S33: If the topology surround value is not equal to zero, determine that the three-dimensional spatial coordinate point of the defect is located inside the polygon of the current photovoltaic module, and use the built-in device code corresponding to the current photovoltaic module as the physical identifier of the target module; if the topology surround value is equal to zero, calculate the shortest Euclidean distance from the three-dimensional spatial coordinate point of the defect to all physical boundary line segments, and select the built-in device code corresponding to the shortest Euclidean distance as the physical identifier of the target module.

[0100] Under strong wind loads, the photovoltaic module support will undergo elastic deformation, causing the coordinates of the four corner points of the module to shift relative to the recorded values. The three-dimensional spatial coordinates of defects located at the edge of the module may fall outside all the quadrilaterals of the module due to deformation. If only the non-zero topology wrap value is used as the only hit condition, there will be isolated loss of defect points that cannot be associated with any module. Therefore, it is necessary to set the shortest Euclidean distance as a fallback value for the case where the topology wrap value is equal to zero.

[0101] Specifically, all photovoltaic modules read in step S31 are traversed, and step S32 is performed for each photovoltaic module to obtain the topology wrap value; if the topology wrap value is not equal to zero, it is determined that the three-dimensional spatial coordinate point of the defect is located inside the polygon of the currently traversed photovoltaic module, and the built-in device code of the currently traversed photovoltaic module is used as the physical identifier of the target module and the traversal ends.

[0102] If the topology wraparound value is zero after traversing all photovoltaic modules, then for each physical boundary segment of each photovoltaic module, calculate the shortest Euclidean distance from the defect's 3D spatial coordinate point to the physical boundary segment: take the vector dot product of the offset vector from the starting vertex of the physical boundary segment to the defect's 3D spatial coordinate point and the boundary vector from the starting vertex to the ending vertex, and then divide by the square of the boundary vector's magnitude to obtain the projection parameter. Projection parameters Limited to Within the interval, to ensure that the projection point falls on the physical boundary line segment, the Euclidean distance from the defect's three-dimensional spatial coordinate point to the projected point after the constraint is calculated as the shortest Euclidean distance of the physical boundary line segment. Here, the Euclidean distance is calculated in the latitude and longitude numerical space. Since the scale of a single photovoltaic module is much smaller than the influence range of latitude and longitude distortion, the small proportion of distortion in the latitude and longitude numerical space does not change the ranking conclusion of the relative distance at the module level. The built-in device code of the photovoltaic module corresponding to the minimum value of the shortest Euclidean distance among all physical boundary line segments is taken as the physical identifier of the target module.

[0103] For example, projection parameters When the calculation result is 1.3, the projection parameters will be... With a limit of 1, the projection point is taken as the endpoint vertex of the physical boundary line segment, and then the Euclidean distance is calculated.

[0104] In the boundary area of ​​the photovoltaic array, wind-induced deformation of the modules causes edge defects to detach from all quadrilateral boundaries. Without a fallback strategy, the 3D spatial coordinates of the defects will be zero due to topological encirclement, making it impossible to write them into the physical identifier of any target module, resulting in the loss of physical location for the defects in subsequent work orders. This step uses the shortest Euclidean distance from a point to the physical boundary line segment as a metric to force the 3D spatial coordinates of the defects detached from the boundary to be assigned to the photovoltaic module with the closest geometric distance. The projection parameters are in... The interval constraint ensures that the distance measurement is always based on the real point on the physical boundary line segment rather than its extension, so that the physical identifier of the target component assigned by the catch-all assignment corresponds to the real adjacent component in the geographic space, thus maintaining the continuous associativity from the defect to the component even when the physical deformation of the component causes the boundary mismatch.

[0105] S34: Convert the initial defect type code into a unified defect type code by numerical replacement according to the preset digital mapping rules, and extract the unified defect type codes corresponding to the visible light image file sequence and infrared image file sequence bound under the physical identifier of the same target component.

[0106] In the initial defect type code output in step S21, the infrared defect type code has a value range of 0 to 5, and the visible light defect type code has a value range of 0 to 1. The numerical ranges of the two coding systems overlap and their meanings are inconsistent. If the initial defect type code is directly passed to step S35 for joint judgment, the same value will be mistakenly identified as the same defect under visible light and infrared sources. Therefore, the code is first uniformly encoded according to the preset digital mapping rules.

[0107] Specifically, the preset digital mapping rules are as follows: the initial defect type codes from the infrared image file sequence are kept unchanged, with 0 corresponding to open circuit, 1 to diode, 2 to non-tracking of the string, 3 to hot spot, 4 to dust accumulation, and 5 to missing component; for the initial defect type codes from the visible light image file sequence, the value 0 is replaced with 6 and corresponds to missing component, and the value 1 is replaced with 7 and corresponds to weeds; the replaced values ​​are used as the unified defect type codes, and the value range of the unified defect type codes is from 0 to 7.

[0108] The method for determining the source of the initial defect type encoding is as follows: read the type identifier character in the image file path recorded in step S1. When the type identifier character is V, it is determined that it comes from a visible light image file sequence; when the type identifier character is T, it is determined that it comes from an infrared image file sequence.

[0109] For each unified defect type code, query the associated information from the defect association module according to the unified defect type code, fill in the defect level, defect features, and processing suggestions, and save the confidence level, pixel-level defect detection box position, and defect three-dimensional spatial coordinate point; at the same time, according to the target component physical identifier output in step S33, extract all unified defect type codes corresponding to the visible light image file sequence and infrared image file sequence bound to the same target component physical identifier.

[0110] S35: If any unified defect type code under the same target component physical identifier indicates an abnormal state, output a dual-light confirmation signal to determine the abnormality, and package the target component physical identifier and the unified defect type code indicating the abnormal state to generate an operation and maintenance work order.

[0111] In field inspection environments, there are extreme cases where single-sided photos from visible light lenses or infrared thermal imaging lenses may be lost due to dust obstruction or transmission damage. If the joint judgment logic based on the direct binding of dual-light photo files is used, the judgment process will not be able to close the loop once a single-sided photo is missing, as the photo pair binding relationship will be broken. Therefore, the judgment input conditions need to be decoupled from the image files and bound to the objectively existing physical identifier dimension of the target component.

[0112] Specifically, the values ​​of the defect handling flag are defined as follows: 0 indicates conversion to defect, 1 indicates no conversion, and 2 indicates no defect. The unified defect type code indicates an abnormal state when the defect handling flag corresponding to the unified defect type code is 0 and the defect is converted to defect. The unified defect type code indicates a normal state when the defect handling flag corresponding to the unified defect type code is 2 and the defect is not a defect.

[0113] The joint judgment is executed as follows: For all unified defect type codes under the same target component physical identifier extracted in step S34, the defect handling mark is judged one by one. If any unified defect type code indicates an abnormal state, a judgment signal confirming the abnormality is output. Only when all unified defect type codes under the same target component physical identifier indicate a normal state is a judgment signal confirming the normality is output. If there is no unified defect type code indicating an abnormal state under the same target component physical identifier, but there is a unified defect type code with a defect handling mark of 1 and no conversion, a judgment signal confirming the normality is also output, and no maintenance work order is generated.

[0114] The maintenance work order generation method is as follows: for the physical identifier of the target component that is confirmed to be abnormal by outputting dual light, the physical identifier of the target component, the unified defect type code indicating the abnormal state, the defect level and handling suggestions filled by the defect association module, and the three-dimensional spatial coordinates of the defect are packaged to generate an maintenance work order and sent to the maintenance terminal.

[0115] Dual-light correlation reverse lookup is performed in the operation and maintenance work order display process: When the operation and maintenance terminal requests to view the associated photos of a certain image file, if the requested image file type identifier character is V, the infrared image file is retrieved from the infrared photo identifier field of the photo group entity; if the type identifier character is T, the visible light image file is retrieved from the visible light photo identifier field, realizing the comparison display of visible light defects and infrared defects.

[0116] The unified defect type code is aggregated in step S34 at the target component physical identifier dimension. The target component physical identifier is obtained in step S33 based on the wrap-around number topology and the shortest Euclidean distance. The two together constitute the input for the joint determination in this step. If there is only a unified defect type code but no aggregation at the target component physical identifier dimension, the determination must still fall back to using the two-light photo file pair as the binding object. When one side of the photo is missing, the photo pair will break, causing the determination to deadlock and making it impossible to complete defect screening under single-light conditions. If there is only the target component physical identifier but no unified defect type code, defects from visible light sources and infrared sources cannot be distinguished as different defects due to overlapping coding intervals. The anomaly count under the same target component physical identifier will be duplicated or confused, and the joint determination will lose a comparable category benchmark. The combination of these two approaches ensures that the judgment input is both locked onto the objective physical device object and based on a unified defect category system: when one side of the visible light or infrared image file is missing, the unified defect type code identified from the other image remains bound to the same target component physical identifier. The judgment process is executed in a closed loop based on the physical device object, without relying on the completeness of the image pair, thus enabling defect screening and work order issuance even under single-light conditions. The combined effect of these two approaches cannot be attributed to any single mechanism: without the unified defect type code, the convergence of physical device dimensions will be unable to distinguish the source of dual-light defects; without the target component physical identifier dimension, the unified defect type code has nowhere to attach when the image pair is broken. The two approaches support each other in the judgment logic; the absence of either one will cause the joint judgment to fail under single-light missing conditions in the field.

[0117] In summary, this embodiment generates a dual-light associated image group with consistent physical time through step S1 using a time grouping string and a hardware exposure timestamp as dual physical records; step S2 eliminates the dual-light coordinate discrepancy caused by shutter speed difference and avoids edge drift of large longitude plane projection by using a spatial displacement compensation vector in a Cartesian three-dimensional rectangular coordinate system; and step S3 accurately locates the three-dimensional spatial coordinate point of the defect to the physical identifier of the target component by using a wraparound number topology and the shortest Euclidean distance as a fallback, and completes the unification and joint determination of dual-light defect type in the dimension of the physical identifier of the target component, thus forming a complete closed-loop link from hardware exposure signal compensation to physical equipment positioning and work order issuance.

[0118] Example 2: This embodiment, based on Embodiment 1, provides a dual-light pairing positioning system for inspecting defects in photovoltaic modules, such as... Figure 10 As shown, it includes: The dual-light correlated image group generation module is used to simultaneously acquire visible light image file sequences and infrared image file sequences of photovoltaic arrays by using a drone equipped with a dual-light camera, extract the corresponding hardware exposure timestamps and combine them in time sequence to generate dual-light correlated image groups; The defect three-dimensional spatial coordinate correction module is used to extract pixel-level defect detection boxes and initial defect type codes from the dual-light correlated image group, calculate spatial displacement compensation vectors using the hardware exposure timestamps, and convert them into defect three-dimensional spatial coordinate points using a three-dimensional rectangular coordinate system. The target component location and work order issuance module is used to calculate the geometric relationship between the three-dimensional spatial coordinates of the defect and the quadrilateral vertex sequence of the photovoltaic module based on the surrounding number topology, output the physical identifier of the target component, and issue an operation and maintenance work order in conjunction with the initial defect type code.

Claims

1. A dual-light pairing method for locating defects in photovoltaic modules during inspection, characterized in that, include: A drone equipped with a dual-light camera simultaneously acquires visible light image file sequences and infrared image file sequences of a photovoltaic array, extracts the corresponding hardware exposure timestamps, and combines them in a time sequence to generate a dual-light associated image group. Pixel-level defect detection boxes and initial defect type codes are extracted from the dual-light correlated image group. Spatial displacement compensation vectors are calculated using the hardware exposure timestamp difference between the visible light image file sequence and the infrared image file sequence and the instantaneous flight speed output by the UAV inertial navigation unit. The vectors are then converted into three-dimensional spatial coordinates of the defect using a three-dimensional rectangular coordinate system. The topology wrapping value is obtained by accumulating the vector crosses of the defect's three-dimensional spatial coordinates relative to the physical boundary segments of the photovoltaic module quadrilateral. The geometric relationship between the defect's three-dimensional spatial coordinates and the vertex sequence of the photovoltaic module quadrilateral is determined based on the topology wrapping value. The physical identifier of the target module is output, and a maintenance work order is issued in conjunction with the initial defect type code.

2. The dual-light pairing method for locating defects in photovoltaic modules according to claim 1, characterized in that, The step of extracting the corresponding hardware exposure timestamps and combining them in a time sequence to generate a dual-light correlated image group includes: Read the underlying hardware exposure timestamps of visible light image file sequences and infrared image file sequences, and extract the time grouping strings from the file names; The visible light image file sequence and the infrared image file sequence are paired and merged according to the time grouping string, and a dual-light associated image group is generated by combining the hardware exposure timestamp.

3. The dual-light pairing method for locating defects in photovoltaic modules according to claim 2, characterized in that, The process of reading the underlying hardware exposure timestamps of the visible light image file sequence and the infrared image file sequence, and extracting the time grouping strings from the filenames, includes: Parse the exchangeable image file format metadata segment of each image file, read the exposure time field written by the camera imaging sensor at the moment the shutter opens and closes, and obtain the hardware exposure timestamp in milliseconds. The filenames are matched using regular expressions, the type identifier characters and file extensions at the end of the filenames are removed, and the continuous substrings obtained by matching are used as time group strings; The time grouping string, type identifier character, and hardware exposure timestamp are written into the visible light mapping table and infrared mapping table one by one according to the image file.

4. The dual-light pairing method for locating defects in photovoltaic modules according to claim 2, characterized in that, The step of pairing and merging the visible light image file sequence and the infrared image file sequence according to the time grouping string, and generating a dual-light associated image group by combining the hardware exposure timestamp includes: Iterate through the deduplicated set of all time group strings, and for each currently iterated time group string, retrieve the corresponding visible light image file path and infrared image file path from the visible light mapping table and infrared mapping table, respectively. The absolute value of the difference between the hardware exposure timestamp corresponding to the extracted visible light image file path and the hardware exposure timestamp corresponding to the infrared image file path is calculated to obtain the exposure timestamp deviation. If the exposure timestamp deviation is less than the dual-light simultaneous shooting determination threshold, then the visible light image file and the infrared image file are determined to be dual-light photos taken at the same time and are paired and merged, and written into a dual-light associated image group named with the time grouping string of the currently traversed time.

5. The dual-light pairing method for locating defects in photovoltaic modules according to claim 1, characterized in that, The step of extracting pixel-level defect detection boxes and initial defect type codes from the dual-light correlated image group, calculating spatial displacement compensation vectors, and converting them into three-dimensional spatial coordinate points of the defects using a three-dimensional rectangular coordinate system includes: The dual-light correlated image group is input into the image feature extraction unit, which outputs the pixel-level defect detection box and the initial defect type code corresponding to the visible light image file sequence and the infrared image file sequence, respectively. Multiply the hardware exposure timestamp difference by the instantaneous flight speed output by the UAV inertial navigation unit to obtain the spatial displacement compensation vector; The latitude and longitude sensor values ​​of the UAV gimbal are obtained, and the latitude and longitude sensor values ​​are converted into Cartesian three-dimensional rectangular coordinate values ​​through a spherical projection algorithm. The center pixel of the pixel-level defect detection box is mapped and superimposed onto the Cartesian three-dimensional rectangular coordinate values. The coordinates are corrected by performing vector addition with the spatial displacement compensation vector. The corrected Cartesian three-dimensional rectangular coordinates are then restored to geographic latitude and longitude values ​​through inverse projection to obtain the three-dimensional spatial coordinates of the defect.

6. The dual-light pairing method for locating defects in photovoltaic modules according to claim 5, characterized in that, The steps of acquiring the latitude and longitude sensor values ​​of the UAV gimbal, converting the latitude and longitude sensor values ​​into Cartesian three-dimensional rectangular coordinate values ​​through a spherical projection algorithm, and mapping and superimposing the center pixel of the pixel-level defect detection box onto the Cartesian three-dimensional rectangular coordinate values ​​include: The latitude and longitude sensor values, including the latitude and longitude values ​​of the gimbal lens's sub-satellite point at the shooting time, are obtained through the gimbal positioning module. The Cartesian three-dimensional rectangular coordinate values ​​are then calculated using a spherical projection algorithm with the Earth's average radius as the spherical radius. Calculate the x and y coordinates of the center point pixel of the pixel-level defect detection box, subtract the coordinates of the principal point of the image to obtain the pixel offset, and then perform yaw angle rotation correction. The pixel offset result after yaw angle rotation correction is multiplied by the pixel size and combined with the flight altitude and lens focal length to convert it into the eastward and northward components of ground offset. These components are then multiplied by their corresponding unit vectors and superimposed onto the Cartesian three-dimensional rectangular coordinate value.

7. The dual-light pairing method for locating defects in photovoltaic modules according to claim 1, characterized in that, The simultaneous acquisition of visible light image file sequences and infrared image file sequences of the photovoltaic array by a drone equipped with a dual-light camera includes: Read the preset waypoints from the flight mission management module. The preset waypoints are generated by the geographical coordinates of the four corners of the photovoltaic array and the combined points according to the preset heading overlap rate and lateral overlap rate. The dual-light camera on the control drone takes pictures of the photovoltaic array according to the preset waypoints. At each preset waypoint, the visible light lens and the infrared thermal imaging lens are triggered to take pictures, so as to obtain the visible light image file sequence and the infrared image file sequence with corresponding identification characters respectively.

8. The dual-light pairing method for locating defects in photovoltaic modules according to claim 1, characterized in that, The step of determining the geometric relationship between the three-dimensional spatial coordinates of the defect and the sequence of vertices of the quadrilateral of the photovoltaic module based on the topology surround value and outputting the physical identifier of the target module includes: Read the sequence of quadrilateral vertices of the photovoltaic modules corresponding to the current inspection area from the storage module; Each physical boundary line segment of the quadrilateral vertex sequence of the photovoltaic module is extracted sequentially, and the vector cross product of the three-dimensional spatial coordinate point of the defect with respect to each physical boundary line segment is obtained by calculating the three-point turning direction; If the topology surround value is not equal to zero, it is determined that the three-dimensional spatial coordinate point of the defect is located inside the polygon of the current photovoltaic module, and the built-in device code corresponding to the current photovoltaic module is used as the physical identifier of the target module. If the topology surround value is equal to zero, calculate the shortest Euclidean distance from the three-dimensional spatial coordinate point of the defect to all physical boundary line segments, and select the built-in device code corresponding to the shortest Euclidean distance as the physical identifier of the target component.

9. The dual-light pairing method for locating defects in photovoltaic modules according to claim 8, characterized in that, The calculation of the topology wrap-around value includes: For each currently traversed physical boundary line segment, extract the latitude and longitude difference components of the vector from the starting vertex to the ending vertex and the vector from the starting vertex to the three-dimensional spatial coordinate point of the defect to calculate the cross product of the vectors in the turning direction. When the physical boundary line segment crosses the latitude of the three-dimensional spatial coordinate point of the defect from bottom to top and the cross product of the vector is positive, the topology wrap value is incremented by one; when the physical boundary line segment crosses the latitude of the three-dimensional spatial coordinate point of the defect from top to bottom and the cross product of the vector is negative, the topology wrap value is decremented by one. The final topology wrap value is obtained by summing the contributions of all physical boundary segments.

10. The dual-light pairing method for locating defects in photovoltaic modules according to claim 8, characterized in that, The calculation of the shortest Euclidean distance from the three-dimensional spatial coordinate point of the defect to all physical boundary line segments includes: For each physical boundary segment of each photovoltaic module, the offset vector pointing from the starting vertex of the physical boundary segment to the three-dimensional spatial coordinate point of the defect is multiplied by the boundary vector pointing from the starting vertex to the ending vertex, and then divided by the square of the boundary vector magnitude to obtain the projection parameter. The projection parameters are restricted to the range of zero to one so that the projection point falls on the physical boundary line segment; Calculate the distance between the three-dimensional spatial coordinates of the defect and the projection point after limiting the projection parameters, and use it as the shortest Euclidean distance of the corresponding physical boundary line segment.

11. The dual-light pairing method for locating defects in photovoltaic modules according to claim 1, characterized in that, The joint issuance of maintenance work orders based on the initial defect type code includes: The initial defect type code is numerically replaced according to a preset digital mapping rule to convert it into a unified defect type code. All unified defect type codes corresponding to the visible light image file sequence and infrared image file sequence bound under the same target component physical identifier are extracted. For all unified defect type codes under the same target component physical identifier, the defect handling mark is judged one by one. If any unified defect type code indicates an abnormal state, a dual-light confirmation signal is output to confirm the abnormality. For the physical identifier of the target component that outputs the judgment signal, the physical identifier of the target component, the unified defect type code indicating the abnormal state, the defect level parameter, and the three-dimensional spatial coordinates of the defect are combined and packaged to generate an operation and maintenance work order and sent to the operation and maintenance terminal.

12. A dual-light pairing positioning system for photovoltaic module inspection defects, used to implement the dual-light pairing positioning method for photovoltaic module inspection defects as described in any one of claims 1 to 11, characterized in that, The system includes: The dual-light correlated image group generation module is used to simultaneously acquire visible light image file sequences and infrared image file sequences of photovoltaic arrays by using a drone equipped with a dual-light camera, extract the corresponding hardware exposure timestamps and combine them in time sequence to generate dual-light correlated image groups; The defect three-dimensional spatial coordinate correction module is used to extract pixel-level defect detection boxes and initial defect type codes from the dual-light correlated image group, calculate spatial displacement compensation vectors using the hardware exposure timestamp difference between the visible light image file sequence and the infrared image file sequence and the instantaneous flight speed output by the UAV inertial navigation unit, and convert them into defect three-dimensional spatial coordinate points using a three-dimensional rectangular coordinate system. The target component location and work order issuance module is used to obtain a topology wrap value based on the vector cross accumulation of the defect's three-dimensional spatial coordinates relative to each physical boundary line segment of the photovoltaic module quadrilateral, determine the geometric relationship between the defect's three-dimensional spatial coordinates and the vertex sequence of the photovoltaic module quadrilateral based on the topology wrap value, output the target component physical identifier, and issue an operation and maintenance work order in conjunction with the initial defect type code.

Images