Panoramic Image Correction Method, Apparatus and Electronic Equipment for Photovoltaic Modules
By identifying the boundaries of photovoltaic modules and constructing a multi-constraint optimization framework for adaptive geometric correction, the problems of boundary misalignment and distortion in panoramic image stitching of photovoltaic modules are solved, improving the accuracy and reliability of panoramic images and ensuring the overall alignment consistency of the photovoltaic module array.
Patent Information
- Application Number
- CN202611054956.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-07-15
- Publication Date
- 2026-11-10
AI Technical Summary
Existing technologies for panoramic image stitching of photovoltaic modules suffer from problems such as boundary misalignment, geometric distortion, and discontinuity between adjacent images, resulting in insufficient accuracy and reliability of panoramic images and difficulty in maintaining the overall arrangement consistency and boundary regularity of the photovoltaic module array.
By identifying the boundaries of photovoltaic modules, determining shape deviation, alignment deviation, and arrangement deviation, a multi-constraint optimization framework is constructed for adaptive geometric correction. Combining the standard rectangular geometric prior of photovoltaic modules and the array arrangement rules, an adaptive geometric correction transformation is generated.
It improves the geometric consistency and boundary regularity of panoramic images, enhancing the reliability and consistency of photovoltaic module inspection, positioning, and analysis tasks.
Smart Images

Figure CN123071201A_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the field of image vision, and in particular relates to a panoramic image correction method, device and electronic device for photovoltaic modules. Background Technology
[0002] With the development of automated inspection technology, photovoltaic robots are widely used in photovoltaic power plants to monitor and analyze the operating status of photovoltaic modules. This technology can acquire top-down information of photovoltaic panels through multi-field-of-view images, providing data support for inspection, positioning, and maintenance, thereby improving inspection efficiency and operational safety. Multi-field images typically require stitching or projection processing to generate panoramic views in order to obtain a complete view of the photovoltaic module array. Existing methods generally rely on image registration, projection transformation, or pixel-domain fusion for image stitching.
[0003] However, due to the complexity of image capture angles and on-site environments, stitched panoramic images are prone to boundary misalignment, geometric distortion, and discontinuities between adjacent images. Furthermore, when photovoltaic modules are densely arranged or have obvious repetitive structures, it is difficult to maintain overall consistency and boundary regularity. These issues result in insufficient accuracy and reliability of panoramic images during subsequent inspections, positioning, or analysis. Summary of the Invention
[0004] This application aims to address at least one of the technical problems existing in the prior art. To this end, this application proposes a panoramic image correction method, apparatus, and electronic device for photovoltaic modules to improve the reliability and consistency of panoramic images.
[0005] In a first aspect, this application provides a panoramic image correction method for photovoltaic modules, the method comprising: An initial panoramic image is obtained, which is obtained by stitching together top-view images of each photovoltaic module under multiple field-of-view areas; Identify the boundaries of each photovoltaic module in the initial panoramic image to obtain boundary identification information; Based on the boundary recognition information, the shape deviation between the photovoltaic module boundary and the expected boundary shape, the alignment deviation between photovoltaic module boundaries at the intersection of different field of view regions, and the arrangement deviation between photovoltaic modules in the photovoltaic module array are determined respectively. By combining the shape deviation, alignment deviation, and arrangement deviation, the initial panoramic image is corrected to obtain the target panoramic image.
[0006] Secondly, this application provides a panoramic image correction device for photovoltaic modules, the device comprising: A panoramic stitching module is used to acquire an initial panoramic image, which is obtained by stitching together top-view images of each photovoltaic module under multiple field-of-view areas; A boundary recognition module is used to identify the boundaries of each photovoltaic module in the initial panoramic image and obtain boundary recognition information; The constraint generation module is used to determine, based on the boundary recognition information, the shape deviation between the photovoltaic module boundary and the expected boundary shape, the alignment deviation between the photovoltaic module boundaries at the intersection of different field of view regions, and the arrangement deviation between each photovoltaic module in the photovoltaic module array. The panoramic correction module is used to combine the shape deviation, alignment deviation, and arrangement deviation to correct the initial panoramic image and obtain the target panoramic image.
[0007] Thirdly, this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the panoramic image correction method for photovoltaic modules as described in the first aspect above.
[0008] Fourthly, this application provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the panoramic image correction method for photovoltaic modules as described in the first aspect above.
[0009] Fifthly, this application provides a chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the panoramic image correction method for photovoltaic modules as described in the first aspect.
[0010] In a sixth aspect, this application provides a computer program product, including a computer program that, when executed by a processor, implements the panoramic image correction method for photovoltaic modules as described in the first aspect above.
[0011] The above-described one or more technical solutions in the embodiments of this application have at least one of the following technical effects: After acquiring an initial panoramic image formed by stitching together top-view images of photovoltaic modules from multiple fields of view, the boundaries of each photovoltaic module in the initial panoramic image are identified to obtain boundary recognition information that characterizes the position, outline, and spatial distribution of the modules. This allows the image correction process to start from the structural features of the photovoltaic modules themselves, rather than relying solely on pixel fusion or ordinary image registration. Based on the boundary recognition information, the shape deviation between the photovoltaic module boundary and the expected boundary shape, the alignment deviation between photovoltaic module boundaries at the intersection of different fields of view, and the arrangement deviation between photovoltaic modules in the photovoltaic module array are determined. This allows the correction process to simultaneously consider the morphological regularity of individual module boundaries, the boundary connection relationship across fields of view, and the overall array relationship between multiple photovoltaic modules. Furthermore, by jointly correcting the initial panoramic image based on shape deviation, alignment deviation, and arrangement deviation, the local optimization and error accumulation caused by relying solely on a single boundary feature or step-by-step correction can be avoided. This makes the photovoltaic module boundaries in the target panoramic image closer to the expected boundary shape, the boundaries at the intersection of different fields of view more continuous, and the arrangement of the photovoltaic module array more stable, thereby improving the reliability and usability of the target panoramic image in subsequent visual tasks such as inspection, positioning, recognition, and path planning.
[0012] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description
[0013] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which: Figure 1 This is a schematic flowchart of the panoramic image correction method for photovoltaic modules provided in the embodiments of this application; Figure 2 This is a schematic diagram illustrating the application scenario of panoramic image correction for photovoltaic modules provided in the embodiments of this application; Figure 3 This is a schematic diagram of the structure of the panoramic image correction device for photovoltaic modules provided in the embodiments of this application; Figure 4 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation
[0014] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments that can be directly obtained based on the embodiments of this application are within the scope of protection of this application.
[0015] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.
[0016] In practical photovoltaic power plant inspection scenarios, photovoltaic robots can be equipped with image acquisition devices in the front, back, left, and right directions. These image acquisition devices can be, for example, fisheye cameras. The acquired multi-field images are converted into a top-down view after inverse perspective transformation and further stitched together to form a 360° panoramic map, used for photovoltaic module boundary detection, defect analysis, and inspection path planning.
[0017] However, there are still some problems with using this approach in related technologies.
[0018] First, the inherent imaging characteristics of image acquisition devices lead to significant image distortion, especially noticeable at the image edges. For example, fisheye cameras can have a field of view of 180°–220°, resulting in severe distortion at the image edges. Related techniques typically employ polynomial distortion models (such as radial and tangential distortion) or unified spherical models for correction. However, these methods are essentially approximate fits and cannot fully describe the complex nonlinear characteristics of the optical system, further amplifying residual errors at the image edges. When the distortion-corrected image undergoes inverse perspective transformation, these residual errors are amplified, causing the boundaries of photovoltaic modules in the generated panoramic image to deviate from standard geometry.
[0019] Secondly, limited on-site calibration conditions make it difficult for static calibration parameters to adapt to dynamic changes in actual operations. Although camera calibration uses calibration boards / cloths, the on-site environment differs significantly from the laboratory environment, making it difficult to acquire sufficient calibration images from multiple angles and distances as in the laboratory, resulting in insufficient robustness of calibration parameters. In real-world scenarios, the posture of the image acquisition device relative to the ground changes when the photovoltaic robot moves on photovoltaic panels at different tilt angles; mechanical vibrations or slight offsets in installation position can also introduce systematic errors, causing offsets in the boundaries of the photovoltaic modules after inverse perspective conversion, and potential alignment failures during multi-frame stitching.
[0020] Furthermore, pixel-domain fusion methods commonly used in related technologies (such as weighted average or Poisson fusion) lack explicit constraints on the boundary structure of photovoltaic modules, resulting in insufficient stitching boundary accuracy. Accumulated calibration errors and image matching errors may cause gaps or overlaps at the boundaries of adjacent fields of view, which is particularly significant at the closure of stitching loops. This greatly affects the overall geometric consistency of the panoramic image, causing existing segmentation models to misidentify broken borders as multiple objects or to classify blurred areas as non-borders, thus affecting positioning accuracy.
[0021] In view of this, this application provides a panoramic image correction method for photovoltaic modules, which aims to improve the geometric consistency and boundary regularity of panoramic images generated by multi-view image stitching. By utilizing the semantic information of the photovoltaic module frame in the stitched panoramic image, combined with the geometric prior of the standard rectangle of the photovoltaic module and the array arrangement rules, a multi-constraint optimization framework is constructed and an adaptive geometric correction transformation is generated, thereby simultaneously improving the distortion of the photovoltaic module frame and the stitching misalignment, and realizing the reliability and consistency of panoramic images in inspection, positioning and analysis tasks.
[0022] The panoramic image correction method for photovoltaic modules provided in this application will be described in detail below with reference to the accompanying drawings, through embodiments and application scenarios.
[0023] This application provides an application scenario where the aforementioned panoramic image correction method for photovoltaic modules can be applied. In this scenario, the method can be used during the inspection of a photovoltaic power station to acquire top-down images of photovoltaic modules from multiple fields of view using a photovoltaic robot or inspection device, and to perform boundary correction on the stitched panoramic image to ensure that the photovoltaic module boundaries are regular, the stitching is continuous, and the array arrangement is consistent, thereby providing high-quality input data for module status detection, local positioning, stain identification, and inspection path planning.
[0024] Of course, the panoramic image correction method for photovoltaic modules provided in this application is not limited to this application scenario, but can also be applied to other scenarios, such as photovoltaic module assembly monitoring, photovoltaic panel production inspection, or other visual analysis tasks that require panoramic image geometric correction and array structure optimization.
[0025] The panoramic image correction method for photovoltaic modules provided in this application can be executed by an electronic device or a functional module or entity within an electronic device capable of implementing the method. This electronic device includes, but is not limited to, an image processing unit on a photovoltaic robot, a dedicated computing module, a general-purpose computer, an embedded controller, or other electronic devices capable of executing the method.
[0026] The following uses an electronic device as an example to illustrate the panoramic image correction method for photovoltaic modules provided in the embodiments of this application.
[0027] Figure 1 This is a flowchart illustrating the panoramic image correction method for photovoltaic modules provided in some embodiments of this application. For example... Figure 1 As shown, the method includes steps 110 to 130.
[0028] Step 110: Obtain the initial panoramic image, which is obtained by stitching together the top-view images of each photovoltaic module in multiple field-of-view areas.
[0029] The initial panoramic image refers to an image generated by stitching together top-down images of photovoltaic modules collected from photovoltaic robots or inspection devices in multiple fields of view. It is used to present multiple photovoltaic modules and their relative positions within the same image coordinate range.
[0030] Different fields of view can correspond to image areas acquired by photovoltaic robots or inspection devices in different directions and within different fields of view. The top-view images under each field of view are stitched together after being aligned with unified coordinates to form an initial panoramic image covering at least part of the photovoltaic module array.
[0031] In some embodiments, in order to enable the initial panoramic image to cover the photovoltaic module image content within the circumferential range of the photovoltaic robot and to enable images acquired from different directions to be subsequently processed within the same image coordinate range, acquiring the initial panoramic image may include: acquiring top-view images of the photovoltaic module from multiple field-of-view directions by multiple image acquisition devices mounted on the photovoltaic robot; wherein the field-of-view directions include at least the front, rear, left, and right sides of the photovoltaic robot; after the photovoltaic robot completes at least one inspection along a predetermined inspection path, the top-view images of the photovoltaic module acquired from multiple field-of-view directions are subjected to perspective conversion and stitching processing to obtain the initial panoramic image.
[0032] Specifically, the photovoltaic robot can inspect the photovoltaic module array along a predetermined inspection path and acquire images of the photovoltaic modules from multiple fields of view using image acquisition devices installed at different locations. Each image acquisition device can correspond to the front, rear, left, and right sides of the photovoltaic robot, enabling the photovoltaic robot to acquire images of the photovoltaic modules from multiple directions around it during a single inspection.
[0033] For example, such as Figure 2 As shown, Figure 2 The image illustrates a scene where a photovoltaic robot inspects an array of photovoltaic modules. Six photovoltaic modules are arranged in a matrix, each module consisting of multiple photovoltaic cell units arranged in a regular grid. The robot has several dotted lines pointing to the surrounding photovoltaic modules, indicating that it collects top-down images of each module from multiple fields of view to generate an initial panoramic image and perform boundary recognition and correction.
[0034] After acquiring images from various image acquisition devices, electronic devices can first perform distortion correction and viewpoint conversion processing on the acquired images to convert the photovoltaic module images from various viewing directions into a top-down view.
[0035] The perspective transformation can include inverse perspective transformation, which is used to map the photovoltaic module area in the acquired image onto the top view image, so that the image content under different viewing directions has a unified perspective basis required for stitching.
[0036] After completing the viewpoint transformation, the electronic device can stitch together the top-view images of the photovoltaic modules from multiple viewing directions to obtain an initial panoramic image. For example, the electronic device can align the top-view images according to the relative positions between different viewing directions, and integrate the photovoltaic module image content from the front, rear, left, and right viewing directions into the same panoramic image coordinate range through a fusion stitching method, thereby obtaining the initial panoramic top-view. This initial panoramic image can serve as the base image for subsequent photovoltaic module boundary recognition, constraint generation, and image correction processing.
[0037] Through the above implementation method, after completing at least one inspection, the photovoltaic robot can obtain image data of photovoltaic modules covering multiple fields of view. The electronic device integrates the image content scattered in different fields of view into an initial panoramic image through viewpoint conversion and stitching processing. In this way, on the one hand, the image coverage corresponding to a single inspection can be expanded, so that the initial panoramic image can reflect multiple photovoltaic modules and their relative positional relationships; on the other hand, subsequent boundary recognition and image correction can be based on the same panoramic image coordinate range, avoiding data fragmentation caused by processing images of different fields of view separately, thereby providing a more complete and unified input basis for the subsequent generation of the target panoramic image.
[0038] Step 120: Identify the boundaries of each photovoltaic module in the initial panoramic image to obtain boundary identification information.
[0039] Boundary identification information refers to data used to characterize the position, extent, and contour of the photovoltaic module boundary in the initial panoramic image.
[0040] For example, the boundary recognition information may include a pixel-level mask of the photovoltaic module boundary and a set of boundary coordinates. The pixel-level mask is used to identify the pixel regions in the initial panoramic image that belong to the photovoltaic module boundary, and the set of boundary coordinates is used to record the coordinate positions of each identified photovoltaic module boundary in the initial panoramic image. The set of boundary coordinates can be represented as a set of coordinates corresponding to multiple photovoltaic module boundaries, with each coordinate set corresponding to one detected photovoltaic module boundary.
[0041] When electronic devices perform boundary recognition on the initial panoramic image, they can extract semantic borders from the image to distinguish between photovoltaic (PV) module boundaries and non-boundary regions, obtaining pixel-level masks of the PV module boundaries. Based on these masks, the set of boundary coordinates corresponding to each PV module boundary is determined. In this way, the electronic device can not only determine which pixels in the initial panoramic image belong to the PV module boundaries but also further obtain the specific distribution location of each PV module boundary in the panoramic image coordinate system. This provides foundational data for subsequent contour analysis, spatial relationship determination, and constraint generation.
[0042] During implementation, the electronic device can also preprocess the initial panoramic image before performing border semantic extraction. Preprocessing may include at least one of grayscale conversion, filtering, contrast enhancement, or noise suppression to reduce the impact of image noise, illumination variations, or stitching artifacts on the boundary recognition results, and improve the accuracy of pixel-level masks and boundary coordinate sets. By obtaining the pixel-level masks and boundary coordinate sets of the photovoltaic module boundaries, subsequent analysis can be performed directly based on the boundary pixels and their coordinate positions to determine the boundary morphology of the photovoltaic modules, the boundary connection relationships between different field-of-view regions, and the spatial distribution relationships within the photovoltaic module array.
[0043] Step 130: Based on the boundary recognition information, determine the shape deviation between the photovoltaic module boundary and the expected boundary shape, the alignment deviation between the photovoltaic module boundaries at the intersection of different field of view areas, and the arrangement deviation between each photovoltaic module in the photovoltaic module array.
[0044] In the initial panoramic image, the top-down views of photovoltaic modules from different fields of view have been stitched together within the same image coordinate range. However, due to potential positional or structural deviations introduced during image acquisition, viewpoint transformation, and stitching, the boundaries of the photovoltaic modules in the initial panoramic image may not accurately reflect the regular boundaries and array arrangement of the actual photovoltaic modules. If the panoramic image is corrected as a whole based solely on image pixels, the structural characteristics of the photovoltaic modules themselves may be overlooked, resulting in a lack of stable geometric basis for the correction process.
[0045] Therefore, after obtaining the boundary recognition information, information that can characterize the boundary shape of a single component, the connection relationship of the boundaries of different fields of view, and the arrangement relationship of multiple component arrays can be further extracted from the boundary recognition information. Based on this, shape deviation, alignment deviation, and arrangement deviation can be determined respectively, so that the subsequent correction process can simultaneously consider the local boundary shape, cross-field boundary relationship, and overall array relationship.
[0046] Among them, shape deviation is used to characterize the degree of deviation of the photovoltaic module boundary from the expected boundary shape; alignment deviation is used to characterize the degree of positional offset between the photovoltaic module boundaries at the intersection of different field of view areas; and arrangement deviation is used to characterize the degree of arrangement consistency between the photovoltaic modules in the photovoltaic module array.
[0047] Specifically, the electronic device analyzes the spatial and shape features of the boundaries of each photovoltaic module in the panoramic image based on previously acquired boundary recognition information to form a variety of correction constraints.
[0048] Here, correction constraints refer to the data conditions used to limit or guide the adjustment direction of the photovoltaic module boundary during subsequent image correction. For example, correction constraints include, but are not limited to, geometric constraints, boundary continuity constraints, and array consistency constraints.
[0049] Since the initial panoramic image is stitched together from top-down images across multiple fields of view, its errors may simultaneously manifest as irregularities in the boundary shapes of individual components, discontinuities at the boundaries between different fields of view, and inconsistent arrangement of the photovoltaic module array. Therefore, relying solely on a single constraint is insufficient to simultaneously address local boundary shapes, cross-field stitching relationships, and the overall array structure. This application's embodiments generate multiple types of correction constraints based on boundary recognition information to jointly constrain the structural relationships at these different levels. Specifically, shape deviation serves as the error basis for geometric shape constraints, guiding the photovoltaic module boundaries towards the expected boundary shape; alignment deviation serves as the error basis for boundary continuity constraints, guiding the photovoltaic module boundaries at the boundaries of different fields of view to align with each other; and arrangement deviation serves as the error basis for array consistency constraints, guiding the photovoltaic modules in the array to maintain a relatively consistent arrangement. Thus, these three types of deviations are not isolated evaluation results but can be further used as the basis for constructing corresponding correction constraints, determining target boundary positions, or constructing optimization functions in subsequent correction processes.
[0050] Electronic devices can determine shape deviations based on the contour information of each photovoltaic (PV) module's boundary, thereby constructing correction constraints to constrain the geometry and ensure that the PV panel frame is as close as possible to a standard rectangle after correction. Since PV modules typically have relatively regular boundary shapes in top-view images, when the initial panoramic image has local distortions or projection deviations, the PV module boundaries may appear curved, skewed, or irregularly deformed. Electronic devices can determine shape deviations based on the difference between the boundary contour and the expected boundary shape, and use this shape deviation as a geometric constraint condition. This constraint is applied to the shape deviation of individual PV module boundaries based on the relationship between the boundary contour and the expected boundary shape, allowing subsequent correction processes to consider the structural regularity of each PV module itself, rather than simply adjusting based on the overall appearance of the image.
[0051] The electronic device can also determine the alignment deviation based on the boundary relationship of photovoltaic modules between different field-of-view areas, thereby constructing a correction constraint to constrain boundary continuity. This constraint ensures that the borders at the boundaries of different camera fields of view are as continuous as possible, without obvious breaks or misalignments. Since the initial panoramic image is formed by stitching together top-down images from multiple field-of-view areas, there may be positional offsets, breaks, or insufficient alignment issues at the boundaries of adjacent field-of-view areas, where the boundaries of the same photovoltaic module or adjacent photovoltaic modules may be at odds. The electronic device can determine the alignment deviation based on the spatial correspondence between the boundaries of photovoltaic modules at the boundaries of different field-of-view areas, and use this alignment deviation as a boundary continuity constraint to describe the spatial correspondence between the boundaries of photovoltaic modules at the boundaries of different field-of-view areas. This allows the subsequent correction process to take into account the boundary connection relationship across field-of-view areas, avoiding the neglect of the overall continuity of the stitched area while only correcting the boundaries within a single field-of-view area.
[0052] Electronic devices can also determine arrangement deviations based on the overall distribution of multiple photovoltaic (PV) module boundaries in the initial panoramic image, thereby constructing correction constraints to ensure the consistency of the PV module array. These constraints ensure that PV panels in the same row or column maintain consistency in size, spacing, and orientation. PV modules are typically arranged in arrays, and the modules within the same array exhibit certain regularities in row-column relationships, relative spacing, and overall arrangement orientation. Correcting only individual module boundaries or local splicing areas may result in seemingly regular local boundaries but an inconsistent overall array structure. Electronic devices can determine arrangement deviations based on the size, spacing, orientation, or row-column relationships between multiple PV module boundaries and use these deviations as array consistency constraints. Starting from the spatial relationships between multiple PV module boundaries, this constraint governs the overall arrangement of PV modules within the array, allowing subsequent correction processes to consider both local boundaries and the global array structure, thus improving the overall consistency of the target panoramic image.
[0053] Furthermore, when correcting local boundaries, focusing solely on boundary positions or array relationships may lead to excessively large correction amplitudes in local areas, resulting in unnatural deformation transitions between adjacent regions in the panoramic image. Therefore, embodiments of this application can also determine spatial smoothing deviations. Spatial smoothing deviations characterize the smoothness and continuity of photovoltaic module boundaries in panoramic images, limiting local abrupt changes between adjacent boundary positions in the panoramic image, ensuring a continuous and smooth boundary adjustment process, and avoiding local distortions or unnatural deformations caused by single-point corrections.
[0054] Since panoramic image correction typically involves the synchronous adjustment of multiple photovoltaic module boundaries and surrounding image regions, if the correction magnitude in a local area differs too much from that in adjacent areas, it may lead to local stretching, bending, or unnatural deformation in the corrected image. By introducing spatial smoothness constraints, the correction process can meet the requirements of geometric shape, boundary continuity, and array consistency while reducing the damage to the overall smoothness of the panoramic image caused by local correction, resulting in a more stable and natural spatial transition effect for the target panoramic image.
[0055] Step 140: Correct the initial panoramic image by combining shape deviation, alignment deviation and arrangement deviation to obtain the target panoramic image.
[0056] Electronic devices can correct an initial panoramic image based on shape deviation, alignment deviation, and arrangement deviation to obtain a target panoramic image. For example, electronic devices can use multiple correction constraints together to correct the initial panoramic image to obtain the target panoramic image. By incorporating correction constraints at different levels, such as geometry, boundary continuity, and array consistency, into the same correction process, deviations at different levels are treated as error terms. This ensures that the final correction result simultaneously considers the shape regularity of individual photovoltaic module boundaries, the boundary connection between different field-of-view regions, and the overall arrangement of the photovoltaic module array.
[0057] In some implementations, the electronic device can determine the target boundary position of each photovoltaic module based on shape deviation, alignment deviation, and arrangement deviation, and establish a spatial mapping relationship between the initial panoramic image and the corrected image based on the correspondence between the initial boundary position represented by boundary recognition information and the target boundary position. The target boundary position can be understood as the position that the photovoltaic module boundary should approach after correction, and it can be represented by corner points, boundary intersections, boundary sampling points, or other key structural positions that can characterize the module boundary structure.
[0058] The electronic device performs a comprehensive transformation on the initial panoramic image based on this spatial mapping relationship, adjusting the boundaries of the photovoltaic modules in the initial panoramic image from their initial boundary positions to positions close to the target boundary, thereby obtaining the target panoramic image. Since the target boundary position is determined based on shape deviation, alignment deviation, and arrangement deviation, the corrected photovoltaic module boundaries not only more closely resemble the expected boundary shape but also improve boundary misalignment at the junctions of different field-of-view regions and maintain the overall arrangement consistency of the photovoltaic module array.
[0059] Through the above correction process, the target panoramic image is not only corrected for a local area, but the boundaries and interrelationships of the photovoltaic modules are uniformly corrected across the entire panoramic image. This reduces structural deviations in the initial panoramic image caused by multi-field stitching, viewpoint switching, or on-site acquisition errors, resulting in a more harmonious panoramic image in terms of boundary morphology, cross-field connection, and overall array relationship. This provides a more reliable image foundation for subsequent photovoltaic module inspection, positioning, identification, and path planning.
[0060] According to the photovoltaic module panoramic image correction method provided in this application embodiment, after acquiring an initial panoramic image formed by stitching together top-view images of each photovoltaic module under multiple field-of-view areas, the boundary of each photovoltaic module in the initial panoramic image is identified to obtain boundary recognition information that can characterize the position, outline, and spatial distribution of the module. This allows the image correction process to start from the structural features of the photovoltaic module itself, rather than relying solely on pixel fusion or ordinary image registration. Then, based on the boundary recognition information, multiple types of correction constraints are generated, and constraints are respectively applied to the geometric shape, boundary continuity, and array consistency of the photovoltaic module boundary. This allows the correction process to simultaneously consider the regularity of the boundary of a single module, the boundary connection relationship between different field-of-view areas, and the overall array relationship between multiple photovoltaic modules. Furthermore, by combining multiple correction constraints to correct the initial panoramic image, local optimization and error accumulation caused by a single constraint or step-by-step correction can be avoided. This improves the overall geometric consistency, boundary continuity, and array arrangement stability of the target panoramic image, thereby enhancing the reliability and usability of the target panoramic image in subsequent visual tasks such as inspection, positioning, recognition, and path planning.
[0061] When a photovoltaic robot performs inspections, images acquired from a single field of view typically only cover a localized area around the robot, making it difficult to reflect the overall distribution of the photovoltaic module array within the robot's circumference. To enable subsequent correction processes to be based on more complete image content, in this embodiment, the photovoltaic robot can acquire top-view images of the photovoltaic modules from multiple fields of view, and after completing an inspection, it uniformly converts and stitches the images from different fields of view to form an initial panoramic image.
[0062] That is, in some embodiments, acquiring the image to be corrected includes: acquiring top-view images of the photovoltaic module from multiple field-of-view directions by multiple image acquisition devices mounted on the photovoltaic robot; wherein the field-of-view directions include at least the front, rear, left and right sides of the photovoltaic robot; after the photovoltaic robot completes at least one inspection along a predetermined inspection path, the top-view images of the photovoltaic module acquired from multiple field-of-view directions are subjected to perspective conversion and stitching processing to obtain an initial panoramic image.
[0063] The photovoltaic robot is equipped with multiple image acquisition devices, each corresponding to a different field of view. The field of view includes at least the front, rear, left, and right sides of the photovoltaic robot, enabling the robot to acquire images of the photovoltaic modules in different directions around it during inspection.
[0064] After the photovoltaic robot completes at least one inspection along the predetermined inspection path, the electronic equipment can acquire images collected by each image acquisition device and perform distortion correction and perspective transformation processing on each image, so that the photovoltaic module images from each field of view are converted into corresponding top-down views. This perspective transformation can include inverse perspective transformation, used to convert the photovoltaic module area in the image to a top-down representation, allowing the photovoltaic module image content from different field of view directions to be stitched together under a unified image reference.
[0065] Subsequently, the electronic device can stitch together the top-view images of the photovoltaic modules from multiple viewpoints to obtain an initial panoramic image. For example, multiple top-view images after viewpoint transformation can be weighted and fused together to form the initial panoramic top-view. The initial panoramic image can cover multiple viewpoints collected by the photovoltaic robot during a single inspection, integrating the photovoltaic module image content from the front, rear, left, and right sides into the same image coordinate range.
[0066] After obtaining the initial panoramic image, the electronic device can further perform semantic extraction on the photovoltaic module bounding boxes in the initial panoramic image to obtain boundary recognition information. For example, the boundary recognition information may include a pixel-level mask of the photovoltaic module bounding box and a set of boundary coordinates. The pixel-level mask is used to represent the pixel regions belonging to the photovoltaic module bounding boxes in the image, and the set of boundary coordinates is used to represent the positions of the detected photovoltaic module bounding boxes in the initial panoramic image. Thus, the initial panoramic image and its corresponding boundary recognition information can jointly serve as the basic input data for subsequent geometric constraint generation, array consistency analysis, and cross-field boundary continuity analysis.
[0067] This initial panoramic image can cover multiple fields of view collected by the photovoltaic robot during a single inspection, integrating images of photovoltaic modules from different directions into a single panoramic image. Subsequent steps can directly perform photovoltaic module boundary recognition and image correction processing on this initial panoramic image, without needing to process each individual field of view separately, thus ensuring the consistency of subsequent processing.
[0068] Therefore, by acquiring top-view images of photovoltaic modules from multiple fields of view using multiple image acquisition devices mounted on the photovoltaic robot, the image coverage that the photovoltaic robot can obtain during a single inspection can be expanded, allowing the initial panoramic image to contain more complete photovoltaic module array information. By performing viewpoint transformation and stitching on the top-view images from multiple fields of view after the photovoltaic robot has completed at least one inspection, the image content acquired from different fields of view can be formed into a panoramic image from a unified perspective, reducing the risk of information loss due to insufficient coverage of a single field of view during subsequent boundary recognition. This processing method provides a more complete and unified image foundation for subsequent multi-constraint correction based on photovoltaic module boundaries, thereby improving the overall reliability of the target panoramic image and the applicability of subsequent inspection analysis.
[0069] In some embodiments, based on boundary recognition information, the shape deviation between the photovoltaic module boundary and the expected boundary shape, the alignment deviation between photovoltaic module boundaries at the intersection of different field-of-view regions, and the arrangement deviation between photovoltaic modules in the photovoltaic module array are determined, including: extracting contour information and distribution statistics of the boundaries of each photovoltaic module based on boundary recognition information, and determining the spatial relationship information of photovoltaic module boundaries in different field-of-view regions; determining the shape deviation based on contour information and the expected boundary shape; wherein the shape deviation is used to characterize the degree of deviation of the photovoltaic module boundary from the expected boundary shape; determining the alignment deviation based on spatial relationship information; wherein the alignment deviation is used to characterize the degree of positional offset between photovoltaic module boundaries at the intersection of different field-of-view regions; and determining the arrangement deviation based on distribution statistics; wherein the arrangement deviation is used to characterize the degree of consistency in the arrangement of photovoltaic modules in the photovoltaic module array.
[0070] In this embodiment, the electronic device can extract the contour information and distribution statistics of the boundaries of each photovoltaic module based on the boundary recognition information, and determine the spatial relationship information of the boundaries of photovoltaic modules in different fields of view.
[0071] Specifically, contour information can be used to characterize the boundary orientation, extent, and shape of each photovoltaic module boundary in the initial panoramic image. Distribution statistics can be used to characterize the overall distribution relationship of multiple photovoltaic module boundaries in the initial panoramic image. Spatial relationship information can be used to characterize the positional correspondence between photovoltaic module boundaries at the boundaries of different field-of-view areas or adjacent areas.
[0072] Electronic devices can determine shape deviations by performing geometric prior processing on the contour information of each photovoltaic module's boundary. Geometric priors refer to the expected boundary shape that a photovoltaic module should typically present in a top-view image. Based on this geometric prior, the electronic device can determine the shape deviation according to the difference between the boundary contour and the expected boundary shape. This allows the subsequent correction process to constrain the shape deviation of individual photovoltaic module boundaries, preventing significant deformation from remaining at the corrected photovoltaic module boundaries.
[0073] Electronic devices can also utilize the spatial relationship information of photovoltaic module boundaries in different field-of-view regions to determine alignment deviations. Since the initial panoramic image is stitched together from top-down images across multiple field-of-view regions, boundary joining deviations may exist at the intersections of different field-of-view regions. Electronic devices can determine the alignment deviation, which characterizes the degree of positional offset between photovoltaic module boundaries at the intersections of different field-of-view regions, based on the relative positions, proximity relationships, and correspondence relationships of the photovoltaic module boundaries in different field-of-view regions. This allows subsequent correction processes to focus on the alignment degree between photovoltaic module boundaries at the field-of-view intersections, thereby reducing the impact of boundary breaks, misalignments, or inconsistent joining between different field-of-view regions on the target panoramic image.
[0074] Electronic devices can also utilize distribution statistics to determine alignment deviations. Since photovoltaic (PV) modules are typically arranged in arrays, there is a relatively stable alignment relationship between multiple PV modules. Electronic devices can determine the alignment deviation, which characterizes the consistency of alignment among PV modules in the array, based on the distribution of each PV module's boundary in the initial panoramic image. For example, alignment deviation can reflect the degree of deviation in size, spacing, orientation, or row / column position of PV modules in the same row or column. Therefore, subsequent correction processes can consider not only individual module boundaries and the field-of-view boundary area, but also the overall alignment relationship of the PV module array, avoiding local corrections that could disrupt the overall array structure.
[0075] Therefore, by further extracting contour information, distribution statistics, and spatial relationship information from boundary recognition information, data that was originally only used to represent boundary positions can be transformed into structured constraints that can participate in image correction. Consequently, subsequent joint correction no longer relies solely on single image features but can simultaneously utilize the boundary morphology of individual components, cross-field boundary relationships, and the overall array structure for correction, thereby improving the geometric consistency of the target panoramic image and the reliability of subsequent inspection and analysis.
[0076] When generating shape deviations based on boundary recognition information, directly treating all photovoltaic (PV) module boundaries in the same way can easily overlook the differences in visibility of different modules in the initial panoramic image. For PV modules that are fully visible, their boundary information is relatively complete and can be directly used to determine the regular boundary shape of the module. However, for PV modules located at the image edge, the boundary of the field of view, or partially obscured, their boundaries may only be partially visible. If geometric constraints are still generated directly based on the visible boundaries, the constraint conditions may be unstable or deviate from the actual module shape. Therefore, it is advisable to first determine the complete visibility state of the boundaries of each PV module, and then use different processing methods to generate shape deviations based on whether the boundaries are fully visible or partially visible.
[0077] To this end, in some embodiments, the shape deviation is determined based on contour information and expected boundary shape, including: detecting whether the boundaries of each photovoltaic module are fully visible; fitting the fully visible photovoltaic module boundaries to obtain a fitted rectangle; fitting the partially visible photovoltaic module boundaries using the array arrangement information of other fully visible photovoltaic module boundaries to generate an inferred rectangle; and determining the shape deviation based on the shape errors between the fitted rectangle and the inferred rectangle and the expected boundary shape, respectively.
[0078] In this embodiment, the electronic device can detect whether the boundaries of each photovoltaic module are completely visible based on the contour information of each photovoltaic module boundary. Complete visibility of the boundary can be understood as the main boundary parts that can characterize the overall shape of the photovoltaic module being identified, while partial visibility can be understood as only part of the boundary being identified, such as only some edge lines, local contours, or missing parts of the boundary that characterize the complete shape of the module.
[0079] For a fully visible photovoltaic (PV) module boundary, the electronic device can fit a fitted rectangle based on the contour information of the PV module boundary. The fitted rectangle is used to characterize the reference boundary shape of the PV module in the initial panoramic image. Since PV modules typically have near-rectangular boundary features in top-view images, a regular boundary reference corresponding to the module can be obtained by performing rectangular fitting on the fully visible PV module boundary.
[0080] For partially visible photovoltaic (PV) module boundaries, electronic devices can use the array arrangement information of other fully visible PV module boundaries to fit the data and generate an inferred rectangle. Array arrangement information can include the arrangement position, orientation, adjacency relationship, and boundary scale relationship of fully visible PV modules within the same PV module array—information that reflects the array's layout pattern. Since PV modules within the same array typically exhibit a relatively stable arrangement pattern, even if some visible PV modules lack partial boundaries, their missing boundaries can be inferred using the array arrangement information of other fully visible PV modules, thus generating an inferred rectangle that more closely approximates the actual module shape.
[0081] After obtaining the fitted rectangle and the inferred rectangle, the electronic device can determine the shape deviations between the fitted rectangle, the inferred rectangle, and the expected boundary shape. The expected boundary shape can be a regular boundary shape that the photovoltaic module should satisfy in the top view image, such as a boundary shape close to a rectangle.
[0082] Therefore, by first detecting whether the boundaries of each photovoltaic module are completely visible, and generating fitting rectangles and inferred rectangles for completely visible boundaries and partially visible boundaries respectively, the problem of inaccurate constraints caused by directly using incomplete boundaries as the basis for geometric constraints can be avoided. Furthermore, the shape deviation can adapt to the boundary states of both complete and incomplete modules, improve the reliability of constraints, make the subsequent panoramic image correction process more stable, and help improve the regularity of photovoltaic module boundaries in the target panoramic image.
[0083] For a fully visible photovoltaic (PV) module boundary, since its contour information is relatively complete, the main extension direction and boundary range of the PV module in the initial panoramic image can be directly determined from the spatial distribution of the boundary pixels. In some embodiments, fitting the fully visible PV module boundary to obtain a fitted rectangle includes: extracting the contour pixel set of the PV module boundary for the fully visible PV module boundary; performing principal component analysis on the contour pixel set to determine the principal and secondary directions of the corresponding PV module; and projecting each contour pixel in the contour pixel set onto the principal and secondary directions respectively to obtain the fitted rectangle.
[0084] For fully visible photovoltaic module boundaries The electronic device can extract the set of contour pixels corresponding to the boundary of the photovoltaic module. The set of contour pixels refers to the set of multiple pixels located on the boundary of the photovoltaic module and their coordinates in the initial panoramic image. It reflects the overall distribution of the photovoltaic module boundary in the image, and can be represented as follows:
[0085] In the formula, ( ) represents the coordinates of the j-th contour point, | | indicates the number of points in the point set.
[0086] Electronic devices can perform principal component analysis (PCA) on the set of contour pixels to determine the principal and secondary directions of the corresponding photovoltaic (PV) module. The principal direction characterizes the direction in which the PV module's boundary extends the most in the image, typically corresponding to the long side of the PV module; the secondary direction characterizes another principal direction distinct from the principal direction, typically corresponding to the short side of the PV module. Determining the principal and secondary directions through PCA avoids directly using image coordinate axes as fitting directions, thus enabling the subsequent fitting process to adapt to the rotational posture of the PV module in the initial panoramic image.
[0087] For example, in order to determine the main orientation of the photovoltaic module, the electronic device sets the point set Perform principal component analysis and calculate the covariance matrix C of the point set:
[0088] In the formula, , Let be the centroid coordinates of the point set.
[0089] The eigenvalues are obtained by performing eigenvalue decomposition on the covariance matrix C. , and the corresponding feature vectors , The principal direction vector of the photovoltaic panel is obtained. (Corresponding to larger eigenvalues, which can be considered as the direction of the longer side of a rectangle) and the secondary direction vector. (The corresponding smaller feature value can be regarded as the direction of the short side of the rectangle).
[0090] After determining the primary and secondary directions, the electronic device can project each contour pixel in the contour pixel set onto the primary and secondary directions respectively, and determine the range of the photovoltaic module boundary in both directions based on the projection results. Specifically, the boundary of the fitting rectangle in the primary direction can be determined based on the distribution range of the contour pixels in the primary direction, and the boundary of the fitting rectangle in the secondary direction can be determined based on the distribution range of the contour pixels in the secondary direction, thus obtaining a fitting rectangle corresponding to the boundary of the complete visible photovoltaic module. This fitting rectangle can serve as a reference boundary shape for subsequent calculations of shape deviation.
[0091] For example, the electronic device will set the contour points Projected to and Construct a fitted rectangle in the direction. The longer side of the rectangle is Parallel, shorter side and The length and width of the parallel rectangle are:
[0092]
[0093] In the formula, Electronic equipment can calculate the shape error between the actual photovoltaic panel outline and the ideal rectangle. , where is the average shortest distance from the contour points to the boundary of the fitted rectangle:
[0094] In the formula, Point To rectangle The shortest Euclidean distance to the boundary. The smaller the value, the closer the frame of the photovoltaic panel is to an ideal rectangle.
[0095] Therefore, by determining the principal and secondary directions of the photovoltaic module boundary through principal component analysis and generating a fitting rectangle based on the projection results, the fitting results can better match the actual posture of the module, reduce the error caused by fixed coordinate axis fitting, and provide a reliable basis for geometric shape correction constraints.
[0096] In the initial panoramic image, due to shading, field of view edges, or image stitching limitations, the boundaries of some photovoltaic modules may only be partially visible (e.g., only two or three sides are clearly discernible). Directly fitting a rectangle based on partial boundaries may lead to shape deviations from the actual shape of the photovoltaic modules, affecting the subsequent correction effect. In order to generate a reliable geometric reference when boundary information is incomplete, it is necessary to infer the arrangement pattern of fully visible photovoltaic modules in the same photovoltaic module array, ensuring that the position and shape of the partially visible modules are reasonably consistent in both the local and overall array.
[0097] Therefore, in some embodiments, for partially visible photovoltaic module boundaries, the array arrangement information of other fully visible photovoltaic module boundaries is used to fit and generate an inferred rectangle, including: for partially visible photovoltaic module boundaries, determining the row and column positions of the corresponding photovoltaic modules in the photovoltaic module array; inferring the position of the missing boundary based on the arrangement information of the fully visible photovoltaic modules in the corresponding row and column, and combining the positions of adjacent fully visible photovoltaic module boundaries in the same row or column to generate an inferred rectangle.
[0098] The electronic device can first determine the row and column positions of some visible photovoltaic (PV) modules within the PV module array, which can be obtained by analyzing the spatial distribution and array arrangement patterns of the complete visible PV modules. Subsequently, the electronic device uses the boundary arrangement information of the complete visible PV modules in its respective row and column, including boundary positions, dimensions, and spacing between adjacent modules, to infer the missing boundaries of the partially visible PV modules. To ensure consistency between the local and overall array, the electronic device can further fine-tune the inferred boundaries by incorporating the positions of the boundaries of adjacent complete PV modules in the same row or column. Through this process, the generated inferred rectangle can reflect the complete boundary shape of the partially visible PV modules as closely as possible and maintain consistency with the overall arrangement of the array.
[0099] For example, the electronic device first performs cluster analysis on the fully visible photovoltaic modules in the panoramic image to identify the row and column structure of the photovoltaic module array. Along the main direction of the modules ( Clustering the centroid of the photovoltaic module yields each row, along the sub-direction ( Clustering yields each column. For a complete photovoltaic module in the same row or column, the average length of the row is calculated for the electronic device. Average width of the column and the average spacing of the rows and the average spacing of the column , used to describe the pattern of array arrangement.
[0100] For the partially visible photovoltaic module boundary (Only some edges are clearly identifiable). The electronic device determines its row or column based on the direction and position of its visible edges, and infers the location of missing edges using statistical information from complete photovoltaic modules in the corresponding row and column. If the centroid and principal direction of the photovoltaic module are known, the location can be determined based on the average length of the row it belongs to. and the average width of the column Constructing the inference rectangle Meanwhile, the electronic equipment further adjusts the position of the inferred rectangle based on the positional relationship (spacing) of adjacent photovoltaic modules to ensure the regularity of the arrangement of some visible modules in the array. The inferred rectangle only supplements the boundaries of the missing parts, while retaining the accurate information of the visible edges.
[0101] The shape error of the inferred rectangle can be defined as:
[0102] In the formula, , This represents the set of actually visible contour points in an incomplete photovoltaic module. This represents the shortest distance from the visible contour point to the inferred rectangle boundary. This shape error is calculated only for the visible edge contour points and is used as an auxiliary reference for subsequent multi-constraint optimization to ensure that the boundary shape of some visible components is reasonable and consistent with the overall array during the geometric correction process.
[0103] Correspondingly, the shape deviation can be expressed as:
[0104] In the formula, η is the geometric constraint term, representing the shape deviation. η is the weighting coefficient of the incomplete photovoltaic panel constraint term, used to adjust its influence in the overall optimization.
[0105] Therefore, by combining the array arrangement pattern of complete photovoltaic modules to infer some visible photovoltaic modules, the electronic device can generate a reasonable inference rectangle when the boundary information is incomplete, fill in the missing boundary, and ensure that the boundary of the local photovoltaic module is consistent with the overall array arrangement. In addition, the generated inference rectangle can serve as an effective reference for geometric constraints, improve the integrity and consistency of the photovoltaic module boundary in the panoramic image, and provide a reliable data foundation for subsequent image correction.
[0106] In some embodiments, if the boundary information of a photovoltaic module is severely missing, making it unsuitable for direct generation of shape deviations or arrangement deviations to construct geometric shape or array consistency constraints, the electronic device can treat this module as a sample with missing local constraints. During the overall panoramic image correction process, a single or small number of such missing samples will not significantly affect the overall correction result, because the correction constraints depend on both the boundary information of other complete photovoltaic modules and the spatial relationships of multiple field-of-view regions. However, when the number of photovoltaic modules with missing boundaries in a local area is large, the structural constraint strength of that area may weaken accordingly. The electronic device can compensate in two ways: first, by re-acquiring images of the photovoltaic modules in that area to obtain complete boundary information; second, by inferring the missing boundaries based on the array statistics of adjacent complete photovoltaic modules in that area, thereby generating auxiliary constraints to ensure the rationality of the local area in terms of geometry, boundary continuity, and array consistency. Through this method, even if the boundary information of some photovoltaic modules is missing, the accuracy and stability of the overall panoramic image correction can be maintained, while improving the usability of structural constraints in local areas.
[0107] Since the initial panoramic image is stitched together from top-down views of photovoltaic modules across multiple fields of view, the boundaries of photovoltaic modules in different fields of view may be offset, broken, or overlapping at the junctions of adjacent fields of view. Ignoring the boundary connections at these junctions can easily lead to discontinuities in the stitched panoramic image.
[0108] Therefore, in some embodiments, the alignment deviation is determined based on spatial relationship information, including: identifying boundary pairs formed by the boundaries of photovoltaic modules located at the intersection of adjacent field of view regions; for each boundary pair, determining the spatial correspondence between the two photovoltaic module boundaries, and generating a boundary alignment error based on the spatial correspondence to obtain the alignment deviation.
[0109] The electronic device can first determine the boundary positions of adjacent field-of-view regions in the initial panoramic image and identify the photovoltaic module boundaries located at the boundaries of adjacent field-of-view regions. During implementation, the electronic device can first determine the stitching seam position of adjacent field-of-view regions in the initial panoramic image based on the field-of-view stitching relationship, and set a neighborhood range on both sides of the stitching seam. Subsequently, the electronic device uses photovoltaic module boundaries falling within this neighborhood range as candidate objects, and combines this with the border pixel set in the aforementioned boundary recognition information to determine whether it is located near the stitching seam by calculating the distance from the border to the stitching seam. For photovoltaic module boundaries that are spatially close, have similar boundary extension directions, or may belong to the same continuous boundary structure at the boundaries of adjacent field-of-view regions, the electronic device can identify them as boundary pairs to characterize boundary objects across the stitching seam.
[0110] For each boundary pair, the electronic device can determine the spatial correspondence between the two photovoltaic module boundaries based on their position, orientation, and extension trend in the initial panoramic image. This spatial correspondence describes the relative position and alignment of the two boundaries within the boundary pair at the boundary between adjacent field-of-view regions. Based on this spatial correspondence, the electronic device can calculate the boundary alignment error, which characterizes the degree of offset between the two photovoltaic module boundaries at the boundary between adjacent field-of-view regions.
[0111] After obtaining the boundary alignment error, the electronic device can use the boundary alignment error as the basis for boundary continuity constraints, so that the subsequent joint correction process can reduce the boundary offset at the junction of adjacent field of view regions and improve the connection consistency of photovoltaic module boundaries across field of view regions.
[0112] For example, for each seam, the electronic device identifies pairs of photovoltaic module frames located near the seam in different field-of-view areas, denoted as... Here, L and R represent the border segments on the left and right sides (or top and bottom sides) of the seam, respectively, and j is the seam index. Subsequently, the electronic device extracts the boundary point set for each boundary and establishes corresponding point pairs by combining spatial proximity and edge direction consistency. ,in , This indicates the number of points corresponding to the seam. and These represent the coordinates of the k-th pair of boundary points on both sides of the seam. If necessary, sub-pixel refinement can be applied to the edge points to improve the accuracy of error calculation.
[0113] Based on the corresponding points, the electronic device can calculate the boundary alignment error of the j-th seam. :
[0114] in, This represents the geometric mapping relationship of the local area of the seam, used to map the right border points. Mapped to the reference coordinate system where the left border is located; This represents the Euclidean distance norm. This index reflects the alignment degree of the two sides of the seam boundary. A smaller value indicates a more continuous boundary and better alignment; conversely, a larger value indicates a significant break or misalignment in the stitching seam. By calculating the boundary alignment error for each stitching seam separately, a continuity error index for the panoramic image stitching quality can be obtained. This index will serve as a stitching seam constraint term in subsequent global optimization, used to generate boundary continuity correction constraints.
[0115] Accordingly, the alignment deviation can be expressed as:
[0116] In the formula, This is the boundary continuity constraint term.
[0117] Therefore, by identifying photovoltaic module boundary pairs at the intersection of adjacent field-of-view regions and generating boundary alignment errors based on the spatial correspondence between these boundary pairs, the boundary connection problem between different field-of-view regions can be transformed into a continuity constraint that can be corrected. This approach avoids the problem of correcting only the boundaries within a single field-of-view region while ignoring the stitching area, resulting in better boundary continuity and overall geometric consistency in subsequent panoramic images of the target at the intersection of adjacent field-of-view regions.
[0118] Since photovoltaic (PV) modules are typically arranged in arrays, performing geometric correction or boundary alignment on each individual module may result in regular local boundaries but inconsistent overall array structure. To ensure that PV modules satisfy both local boundary morphology and maintain array arrangement consistency in the target panoramic image, array consistency constraints need to be generated based on the identified boundary information and module distribution statistics. These constraints are used to constrain the arrangement relationship of each module in the entire PV module array.
[0119] Therefore, in some embodiments, the arrangement deviation is determined based on distribution statistics, including: determining the row and column positions of each photovoltaic module in the photovoltaic module array; determining the average arrangement characteristics of the corresponding row and column based on the arrangement information of the fully visible photovoltaic modules in each row and column; and comparing the boundary of each photovoltaic module with the corresponding average arrangement characteristics of its row and column to obtain the arrangement deviation.
[0120] In this embodiment, the electronic device first determines the row and column positions of each photovoltaic module in the photovoltaic module array based on the positional distribution of the boundaries of each photovoltaic module in the initial panoramic image. Since photovoltaic modules are usually laid out in a regular array, photovoltaic modules in the same row or column are usually consistent in size, spacing, and orientation. Therefore, the average arrangement characteristics of the corresponding row and column can be determined using the arrangement information of the fully visible photovoltaic modules.
[0121] Specifically, the electronic device can determine the average length and average row spacing of photovoltaic (PV) modules in each row based on the boundary information of fully visible PV modules in that row; and determine the average width and average column spacing of PV modules in each column based on the boundary information of fully visible PV modules in that column. Simultaneously, the electronic device can also determine the average orientation characteristics of the PV module array based on the orientation of the boundaries of each PV module. This directional consistency does not require that the boundaries of different PV modules be strictly collinear, but rather that corresponding edges in the same row or column be approximately parallel, and that the overall orientation remains consistent.
[0122] For any photovoltaic (PV) module, the electronic device can compare the actual length, actual width, row spacing with adjacent PV modules in the same row, column spacing with adjacent PV modules in the same column, and boundary orientation of the PV module with the average arrangement characteristics corresponding to its row or column. For example, the actual length of the PV module can be compared with the average length of its row, the actual width with the average width of its column, the row spacing with the average row spacing, the column spacing with the average column spacing, and the boundary orientation with the average orientation of the array, thereby determining the arrangement deviation of the PV module relative to its array.
[0123] In some embodiments, the arrangement deviation can be generated based on the combination of the various arrangement deviations described above. For example, the size deviation, spacing deviation, and orientation deviation of the photovoltaic modules can be comprehensively considered to obtain an arrangement deviation constraint term that characterizes the regularity of the photovoltaic module array. The smaller the value of this constraint term, the closer the photovoltaic module array is to a regular distribution in terms of size, arrangement, and orientation; the larger the value of this constraint term, the more significant the deviation of the corresponding photovoltaic module or local array area from the overall array regularity.
[0124] For example, let the length, width, and orientation angle of the i-th photovoltaic panel be respectively... , and The arrangement deviation can be expressed as:
[0125] in, This represents the average length of the row containing the i-th photovoltaic panel; This represents the average width of the column containing the i-th photovoltaic panel; This represents the actual spacing between the i-th photovoltaic panel and its adjacent photovoltaic panels in the same row along the row direction. This indicates the average spacing within its row; This represents the actual spacing between the i-th photovoltaic panel and its adjacent photovoltaic panels in the same column along the column direction. This indicates the average spacing of the column it belongs to; This represents the average orientation angle of the current photovoltaic array.
[0126] Therefore, by utilizing the arrangement information of complete visible photovoltaic modules to generate arrangement deviations, electronic devices can maintain the regularity of the overall array structure of photovoltaic modules during the correction process. This method not only improves the accuracy of boundary correction for individual modules but also ensures the overall consistency of the photovoltaic module array arrangement in the target panoramic image, providing more reliable panoramic data for subsequent inspection, positioning, and maintenance.
[0127] Furthermore, in some embodiments, the initial panoramic image is corrected by combining shape deviation, alignment deviation, and arrangement deviation to obtain the target panoramic image, including: constructing a multi-constraint optimization function based on shape deviation, alignment deviation, and arrangement deviation; solving the multi-constraint optimization function to determine the target boundary position of each photovoltaic module boundary; and correcting the initial panoramic image based on the correspondence between the initial boundary position of each photovoltaic module boundary and the corresponding target boundary position represented by boundary recognition information to obtain the target panoramic image.
[0128] The electronic device first constructs a multi-constraint optimization function based on shape deviation, alignment deviation, and arrangement deviation. This function comprehensively reflects the deviation between the boundary of a single photovoltaic module and the expected boundary shape, the alignment between the boundaries of photovoltaic modules at the intersection of different field-of-view regions, and the consistency of the arrangement of photovoltaic modules in the array. For example, the multi-constraint optimization function can be expressed as:
[0129] In the formula, This represents the error term corresponding to the geometric shape constraint, i.e., the shape deviation. This represents the error term corresponding to the boundary continuity constraint, i.e., the alignment error. This represents the error term corresponding to the array consistency constraint, i.e., the arrangement error; These are the corresponding weighting coefficients, used to balance the impact of different error terms on the overall optimization result. They can be dynamically adjusted according to the importance of different constraints or the initial image distortion.
[0130] Subsequently, the electronic device solves the multi-constraint optimization function to determine the target boundary positions of each photovoltaic module boundary. The target boundary position refers to the position that each photovoltaic module boundary should approach or reach after correction. It can be represented by the target corner point, target boundary intersection point, target boundary line segment, target boundary sampling point, or other key positions that can characterize the boundary structure. For example, the electronic device can use a nonlinear optimization algorithm (such as the Levenberg-Marquardt algorithm or gradient descent) to solve the multi-constraint optimization function and determine the target control point positions of each photovoltaic module boundary. ,in This indicates the photovoltaic module index, and the target control point corresponds to the corner point or key structural point of the photovoltaic module.
[0131] After obtaining the target boundary positions of each photovoltaic module, the electronic equipment can determine the initial boundary positions of each photovoltaic module based on the boundary identification information. The initial boundary positions can be represented by the boundary coordinate set, boundary corner points, boundary intersection points, or boundary sampling points from the boundary identification information. The electronic equipment then determines the initial boundary positions based on the original positions of the control points. With the target location The correspondence between them is used to construct a spatial mapping function from the initial panoramic image to the target control point location. f The spatial mapping function can be solved using Thin Plate Spline Interpolation (TPS) or Moving Least Squares (MLS). Taking TPS as an example, its goal is to maintain smooth overall image deformation while ensuring accurate alignment of control points. The corresponding energy function can be expressed as:
[0132] In the formula, Indicate control points Mapped coordinates This indicates the corresponding target location, with the first term being the control point fitting term. This represents the smoothing regularization term of the mapping function, used to constrain the smoothness of the overall deformation; This is a regularization parameter used to balance the accuracy of control point fitting with the degree of global smoothness.
[0133] After obtaining the spatial mapping function f Then, the electronic device processes each pixel in the initial panoramic image. Calculate its corresponding position in the corrected image. Pixel-level resampling is performed using bilinear or bicubic interpolation to obtain the corrected panoramic image. .
[0134] Therefore, by constructing a multi-constraint optimization function that combines shape deviation, alignment deviation, and arrangement deviation, and correcting the initial panoramic image based on the solved target boundary positions, the electronic device can maintain the regularity of individual photovoltaic module boundaries while ensuring the continuity of boundaries across the field of view and the consistency of the overall array. In the resulting target panoramic image, the photovoltaic module boundaries are closer to the expected boundary shape, the alignment at the seams is higher, and the photovoltaic module array arrangement is more regular, thus providing reliable visual input for subsequent photovoltaic module status detection, inspection planning, and positioning.
[0135] As mentioned above, when constructing multi-constraint optimization functions, electronic devices not only comprehensively consider shape deviation, alignment deviation, and arrangement deviation, but also introduce spatial smoothing deviation to enhance the smoothness of overall boundary adjustment.
[0136] set up Let represent the local transformation parameters at the m-th control point. Then, the spatial smoothing deviation can be expressed as:
[0137] In the formula, This is the error term corresponding to the spatial smoothness constraint, namely the spatial smoothness deviation. This term is used to limit the rate of change of local transformations in space, making the transformation between adjacent control points smoother, thereby improving the stability and naturalness of the overall panoramic image correction result.
[0138] Accordingly, in some embodiments, the multi-constraint optimization function is also constructed based on several shape deviations, alignment deviations, arrangement deviations, and spatial smoothing deviations.
[0139] That is, the multi-constraint optimization function can also be expressed as:
[0140] In the formula, The weighting coefficients corresponding to spatial smoothing deviations can be dynamically adjusted based on the actual degree of image distortion or the importance of constraints.
[0141] The electronic device then solves the multi-constraint optimization function to obtain the target boundary positions of each photovoltaic module. Based on the initial boundary positions and the target boundary positions, a spatial mapping function is constructed to perform pixel-level resampling of the initial panoramic image, thereby obtaining a target panoramic image with regular boundaries, continuous stitching, and overall smoothness. By introducing a spatial smoothing regularization term, the image distortion caused by drastic local correction transformations can be significantly reduced, improving the overall consistency and visual naturalness of the target panoramic image.
[0142] In this embodiment, when the electronic device performs panoramic image quality optimization, it can simultaneously achieve rectangular regularization of the photovoltaic module boundary and continuous alignment of the cross-field-of-view stitching seam area during the same correction process. Combining the geometric prior that "the photovoltaic module should be a standard rectangle" and the arrangement rules of the photovoltaic panel array, a multi-constraint optimization function is constructed, which includes rectangular structure, stitching seam continuity, and array consistency. This function is then solved using a global optimization method to obtain the optimal geometric correction transformation of the panoramic image.
[0143] During the calibration process, the electronic equipment can simultaneously achieve rectangular regularization of the photovoltaic panel boundaries and continuous alignment across the splicing seam area in the same operation. It also incorporates the size, spacing, and orientation of the photovoltaic panel array to provide auxiliary constraints on some visible or incomplete photovoltaic panels. This one-time joint calibration avoids the error accumulation that may occur with step-by-step processing, improving the overall geometric consistency and structural stability of the panoramic image.
[0144] In the optimized panoramic image, the photovoltaic panels present a standard rectangular shape, with smooth and continuous boundaries at the seams, without any breaks or misalignments. The photovoltaic panel array is arranged at a regular height, providing high-quality and reliable input data for subsequent visual tasks such as local positioning, stain recognition, and condition monitoring.
[0145] This application's embodiments do not rely on precise distortion model parameters; they can directly perform adaptive correction based on bounding box semantics to eliminate residual geometric distortion. Simultaneously, the adoption of a multi-constraint unified optimization framework can simultaneously improve single-plate deformation and stitching misalignment issues, significantly reducing the error accumulation caused by step-by-step processing and improving the overall correction accuracy and visual consistency of the target panoramic image.
[0146] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.
[0147] The photovoltaic module panoramic image correction method provided in this application can be executed by a photovoltaic module panoramic image correction device. This application uses the photovoltaic module panoramic image correction device executing the photovoltaic module panoramic image correction method as an example to illustrate the photovoltaic module panoramic image correction device provided in this application.
[0148] like Figure 3 As shown, the panoramic image correction device 300 for photovoltaic modules includes: a panoramic stitching module 301, a boundary recognition module 302, a constraint generation module 303, and a panoramic correction module 304.
[0149] The panoramic stitching module 301 is used to acquire an initial panoramic image, which is obtained by stitching together the top-view images of each photovoltaic module under multiple field-of-view areas.
[0150] The boundary recognition module 302 is used to identify the boundaries of each photovoltaic module in the initial panoramic image and obtain boundary recognition information.
[0151] The constraint generation module 303 is used to determine, based on boundary recognition information, the shape deviation between the photovoltaic module boundary and the expected boundary shape, the alignment deviation between the photovoltaic module boundaries at the intersection of different field regions, and the arrangement deviation between each photovoltaic module in the photovoltaic module array.
[0152] The panoramic correction module 304 is used to correct the initial panoramic image by combining shape deviation, alignment deviation and arrangement deviation to obtain the target panoramic image.
[0153] According to the photovoltaic module panoramic image correction device provided in the embodiments of this application, after acquiring an initial panoramic image formed by stitching together top-view images of each photovoltaic module under multiple field-of-view areas, the device identifies the boundaries of each photovoltaic module in the initial panoramic image to obtain boundary recognition information that can characterize the position, outline, and spatial distribution of the modules. This allows the image correction process to start from the structural features of the photovoltaic modules themselves, rather than relying solely on pixel fusion or ordinary image registration. Then, based on the boundary recognition information, multiple types of correction constraints are generated, and constraints are imposed on the geometric shape, boundary continuity, and array consistency of the photovoltaic module boundaries, respectively. This allows the correction process to simultaneously consider the regularity of individual module boundaries, the boundary connection relationship between different field-of-view areas, and the overall array relationship between multiple photovoltaic modules. Furthermore, by combining multiple correction constraints to correct the initial panoramic image, the device can avoid local optimization and error accumulation caused by single constraints or step-by-step correction, thereby improving the overall geometric consistency, boundary continuity, and array arrangement stability of the target panoramic image. This enhances the reliability and usability of the target panoramic image in subsequent visual tasks such as inspection, positioning, recognition, and path planning.
[0154] In some embodiments, the constraint generation module is used to extract the contour information and distribution statistics of the boundaries of each photovoltaic module based on boundary recognition information, and determine the spatial relationship information of the boundaries of photovoltaic modules in different fields of view; determine the shape deviation based on the contour information and the expected boundary shape; wherein the shape deviation is used to characterize the degree of deviation of the photovoltaic module boundary from the expected boundary shape; determine the alignment deviation based on the spatial relationship information; wherein the alignment deviation is used to characterize the degree of positional offset between the boundaries of photovoltaic modules at the intersection of different fields of view; and determine the arrangement deviation based on the distribution statistics; wherein the arrangement deviation is used to characterize the degree of arrangement consistency between the photovoltaic modules in the photovoltaic module array.
[0155] In some embodiments, the constraint generation module is used to detect whether the boundaries of each photovoltaic module are fully visible; to fit the fully visible photovoltaic module boundaries to obtain a fitted rectangle; to fit the partially visible photovoltaic module boundaries using the array arrangement information of other fully visible photovoltaic module boundaries to generate an inferred rectangle; and to determine the shape deviation based on the shape errors between the fitted rectangle and the inferred rectangle and the expected boundary shape.
[0156] In some embodiments, the constraint generation module is used to extract the set of contour pixels of the photovoltaic module boundary for a fully visible photovoltaic module boundary; perform principal component analysis on the set of contour pixels to determine the principal direction and secondary direction of the corresponding photovoltaic module; and project each contour pixel in the set of contour pixels onto the principal direction and secondary direction respectively to obtain a fitted rectangle.
[0157] In some embodiments, the constraint generation module is used to determine the row and column positions of the corresponding photovoltaic modules in the photovoltaic module array for partially visible photovoltaic module boundaries; infer the position of the missing boundary based on the arrangement information of the fully visible photovoltaic modules in the corresponding row and column, and generate an inferred rectangle by combining the positions of the adjacent fully visible photovoltaic module boundaries in the same row or column.
[0158] In some embodiments, the constraint generation module is used to identify boundary pairs formed by the boundaries of photovoltaic modules located at the intersection of adjacent field of view regions; for each boundary pair, the spatial correspondence between the two photovoltaic module boundaries is determined, and a boundary alignment error is generated based on the spatial correspondence to obtain the alignment deviation.
[0159] In some embodiments, the constraint generation module is used to determine the row and column positions of each photovoltaic module in the photovoltaic module array; based on the arrangement information of the fully visible photovoltaic modules in each row and each column, determine the average arrangement characteristics of the corresponding row and column; and compare the boundary of each photovoltaic module with the corresponding average arrangement characteristics of its row and column to obtain the arrangement deviation.
[0160] In some embodiments, the panoramic correction module is used to construct a multi-constraint optimization function based on shape deviation, alignment deviation, and arrangement deviation; solve the multi-constraint optimization function to determine the target boundary position of each photovoltaic module boundary; and correct the initial panoramic image based on the correspondence between the initial boundary position of each photovoltaic module boundary and the corresponding target boundary position represented by boundary recognition information to obtain the target panoramic image.
[0161] In some embodiments, the multi-constraint optimization function is also constructed based on spatial smoothing deviation, which is used to characterize the smoothness and continuity of the photovoltaic module boundary in the panoramic image.
[0162] In some embodiments, the panoramic stitching module is used to acquire top-view images of photovoltaic modules from multiple field-of-view directions by multiple image acquisition devices mounted on the photovoltaic robot; wherein the field-of-view directions include at least the front, rear, left and right sides of the photovoltaic robot; after the photovoltaic robot completes at least one inspection along a predetermined inspection path, the top-view images of the photovoltaic modules acquired from multiple field-of-view directions are subjected to perspective conversion and stitching processing to obtain an initial panoramic image.
[0163] The panoramic image correction device for photovoltaic modules in this application embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal, such as a server.
[0164] The photovoltaic module panoramic image correction device provided in this application embodiment can realize all the processes implemented in the above-described photovoltaic module panoramic image correction method embodiment and achieve the same technical effect. To avoid repetition, it will not be described again here.
[0165] In some embodiments, this application also provides a photovoltaic robot. This photovoltaic robot can be used in photovoltaic power plant inspection scenarios to acquire images of photovoltaic modules in multiple fields of view while moving along a photovoltaic module array. It then corrects the initial panoramic image formed by stitching together images from multiple fields of view to obtain a target panoramic image with regular boundaries, stable cross-field connections, and consistent array arrangement. Specifically, the photovoltaic robot may include a robot body, multiple image acquisition devices, and a processing device. The robot body can serve as the mobile support structure for the photovoltaic robot, supporting components such as the walking mechanism, power module, communication module, image acquisition devices, and processing device, and can move along the surface of the photovoltaic modules, inter-module channels, or a preset inspection path. Multiple image acquisition devices are mounted on the robot body to acquire images of the photovoltaic modules in multiple fields of view. For example, the multiple image acquisition devices can be respectively positioned on the front, rear, left, and right sides of the robot body to acquire images of the photovoltaic modules in different circumferential directions. The image acquisition devices can be fisheye cameras, wide-angle cameras, ordinary cameras, or other visual acquisition devices capable of acquiring images of photovoltaic modules.
[0166] The processing device is connected to multiple image acquisition devices to receive photovoltaic module images acquired by the multiple image acquisition devices and execute the photovoltaic module panoramic image correction method described in any of the above embodiments. The processing device may be an embedded processor, image processing unit, edge computing module, controller, or other computing module with image processing capabilities, mounted on the robot body. The multiple image acquisition devices can be connected to the processing device through wired or wireless communication interfaces, such as serial interfaces, Ethernet interfaces, universal serial bus interfaces, mobile industrial processor interfaces, or other image transmission interfaces to transmit image data.
[0167] When a photovoltaic robot performs an inspection task, multiple image acquisition devices can acquire images of photovoltaic modules from multiple fields of view. The processing device performs viewpoint transformation and stitching on the acquired images to obtain an initial panoramic image. The processing device further identifies the boundaries of each photovoltaic module in the initial panoramic image, obtaining boundary recognition information. Based on the boundary recognition information, shape deviation, alignment deviation, and arrangement deviation are determined, and the initial panoramic image is corrected based on these deviations to obtain a target panoramic image. This target panoramic image can be used for subsequent module positioning, stain identification, defect detection, path planning, or inspection result display.
[0168] By integrating multiple image acquisition and processing devices into the photovoltaic robot, the robot can directly acquire multi-field-of-view images and perform panoramic image correction during inspection, reducing reliance on external fixed cameras or offline processing equipment. Simultaneously, because the processing device can perform joint correction based on shape deviations at photovoltaic module boundaries, alignment deviations at field-of-view intersections, and array arrangement deviations, it can improve the geometric consistency and boundary reliability of the robot's inspection images, providing a more stable image foundation for the photovoltaic robot's subsequent autonomous localization, visual recognition, and path decision-making.
[0169] In some embodiments, such as Figure 4 As shown, this application embodiment also provides an electronic device 400, including a processor 401, a memory 402, and a computer program stored in the memory 402 and executable on the processor 401. When the program is executed by the processor 401, it implements the various processes of the above-described photovoltaic module panoramic image correction method embodiment and can achieve the same technical effect. To avoid repetition, it will not be described again here.
[0170] This application provides a non-transitory computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the various processes of the above-described photovoltaic module panoramic image correction method embodiment and achieves the same technical effect. To avoid repetition, it will not be described again here.
[0171] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable media, such as computer read-only memory (ROM), random-access memory (RAM), magnetic disks, or optical disks.
[0172] The computer-readable storage medium may include: read-only memory (ROM), random-access memory (RAM), magnetic disk or optical disk, etc.
[0173] This application provides a computer program product, including a computer program that, when executed by a processor, implements the above-described panoramic image correction method for photovoltaic modules.
[0174] This application provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described photovoltaic module panoramic image correction method embodiments and achieve the same technical effect. To avoid repetition, it will not be described again here.
[0175] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.
[0176] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.
[0177] From the above description of the embodiments, it is clear that the methods described in the embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases, the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the related technology, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.
[0178] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the above-described embodiments. The above-described embodiments are merely illustrative and not restrictive. Under the guidance of this application, many other forms can be made without departing from the spirit and scope of the claims, and all of them are within the protection scope of this application.
[0179] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.
[0180] Although embodiments of this application have been shown and described, it is understood that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the claims and their equivalents.
Claims
1. A panoramic image correction method for photovoltaic modules, characterized in that, include: An initial panoramic image is obtained, which is obtained by stitching together top-view images of each photovoltaic module under multiple field-of-view areas; Identify the boundaries of each photovoltaic module in the initial panoramic image to obtain boundary identification information; Based on the boundary recognition information, the shape deviation between the photovoltaic module boundary and the expected boundary shape, the alignment deviation between photovoltaic module boundaries at the intersection of different field of view regions, and the arrangement deviation between photovoltaic modules in the photovoltaic module array are determined respectively. By combining the shape deviation, alignment deviation, and arrangement deviation, the initial panoramic image is corrected to obtain the target panoramic image.
2. The panoramic image correction method for photovoltaic modules according to claim 1, characterized in that, Based on the boundary recognition information, the determination of the shape deviation between the photovoltaic module boundary and the expected boundary shape, the alignment deviation between photovoltaic module boundaries at the intersection of different field-of-view regions, and the arrangement deviation between photovoltaic modules in the photovoltaic module array includes: Based on boundary recognition information, the contour information and distribution statistics of each photovoltaic module boundary are extracted, and the spatial relationship information of photovoltaic module boundaries in different field-of-view areas is determined. Based on the contour information and the expected boundary shape, the shape deviation is determined; wherein, the shape deviation is used to characterize the degree of deviation of the photovoltaic module boundary from the expected boundary shape; Based on the spatial relationship information, the alignment deviation is determined; wherein, the alignment deviation is used to characterize the degree of positional offset between the boundaries of photovoltaic modules at the intersection of different field-of-view regions; Based on the distribution statistics, the arrangement deviation is determined; wherein the arrangement deviation is used to characterize the degree of consistency in the arrangement of photovoltaic modules in the photovoltaic module array.
3. The panoramic image correction method for photovoltaic modules according to claim 2, characterized in that, Determining the shape deviation based on the contour information and the expected boundary shape includes: Check whether the boundaries of each photovoltaic module are complete and visible; Fit the boundary of the fully visible photovoltaic module to obtain the fitted rectangle; For partially visible photovoltaic module boundaries, the array arrangement information of other fully visible photovoltaic module boundaries is used for fitting to generate an inferred rectangle; The shape deviation is determined based on the shape errors between the fitted rectangle and the inferred rectangle and the expected boundary shape, respectively.
4. The panoramic image correction method for photovoltaic modules according to claim 3, characterized in that, The process of fitting the fully visible photovoltaic module boundary to obtain a fitted rectangle includes: For a fully visible photovoltaic module boundary, extract the set of contour pixels of the photovoltaic module boundary; Principal component analysis is performed on the set of contour pixels to determine the principal and secondary directions of the corresponding photovoltaic module; Each contour pixel in the contour pixel set is projected onto the main direction and the secondary direction respectively to obtain a fitted rectangle.
5. The panoramic image correction method for photovoltaic modules according to claim 3 or 4, characterized in that, The step of fitting the partially visible photovoltaic module boundaries with the array arrangement information of other fully visible photovoltaic module boundaries to generate an inferred rectangle includes: For partially visible photovoltaic module boundaries, determine the row and column positions of the corresponding photovoltaic modules in the photovoltaic module array; The location of the missing boundary is inferred based on the arrangement information of the fully visible photovoltaic modules in the corresponding row and column, and the inferred rectangle is generated by combining the location of the boundary of the adjacent fully visible photovoltaic modules in the same row or column.
6. The panoramic image correction method for photovoltaic modules according to claim 2, characterized in that, Determining the alignment deviation based on the spatial relationship information includes: Identify boundary pairs formed by the boundaries of photovoltaic modules located at the intersection of adjacent field-of-view regions; For each boundary pair, determine the spatial correspondence between the two photovoltaic module boundaries, and generate a boundary alignment error based on the spatial correspondence to obtain the alignment deviation.
7. The panoramic image correction method for photovoltaic modules according to claim 2, characterized in that, Determining the arrangement deviation based on the distribution statistics information includes: Determine the row and column position of each photovoltaic module in the photovoltaic module array; Based on the arrangement information of the fully visible photovoltaic modules in each row and column, the average arrangement characteristics of the corresponding row and column are determined; The arrangement deviation is obtained by comparing the boundary of each photovoltaic module with the average arrangement characteristics of its corresponding row and column.
8. The panoramic image correction method for photovoltaic modules according to claim 1, characterized in that, The initial panoramic image is corrected by combining the shape deviation, alignment deviation, and arrangement deviation to obtain the target panoramic image, including: Based on the shape deviation, the alignment deviation, and the arrangement deviation, a multi-constraint optimization function is constructed. The multi-constraint optimization function is solved to determine the target boundary position of each photovoltaic module. Based on the correspondence between the initial boundary position and the corresponding target boundary position of each photovoltaic module represented by the boundary recognition information, the initial panoramic image is corrected to obtain the target panoramic image.
9. The panoramic image correction method for photovoltaic modules according to claim 8, characterized in that, The multi-constraint optimization function is also constructed based on spatial smoothing deviation, which is used to characterize the smoothness and continuity of the photovoltaic module boundary in the panoramic image.
10. The panoramic image correction method for photovoltaic modules according to claim 1, characterized in that, The process of obtaining the initial panoramic image includes: Multiple image acquisition devices mounted on the photovoltaic robot acquire top-view images of the photovoltaic module from multiple fields of view; wherein, the fields of view include at least the front, rear, left and right sides of the photovoltaic robot; After the photovoltaic robot completes at least one inspection along the predetermined inspection path, the top-view images of the photovoltaic modules collected from the multiple field-of-view directions are subjected to perspective conversion and stitching processing to obtain an initial panoramic image.
11. A photovoltaic robot, characterized in that, include: The robot itself; Multiple image acquisition devices are installed on the robot body, and the multiple image acquisition devices are used to acquire images of photovoltaic modules in multiple fields of view; A processing device is connected to the plurality of image acquisition devices, and the processing device is used to perform the panoramic image correction method for photovoltaic modules as described in any one of claims 1-10.
12. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the panoramic image correction method for photovoltaic modules as described in any one of claims 1-10.
