Intelligent installation method of photovoltaic support based on 3D vision

Abstract

The application relates to a photovoltaic support intelligent installation method based on 3D vision, which comprises the following steps: collecting environment data, obtaining the light reflection condition of a photovoltaic panel, and adjusting the camera parameters of a split binocular 3D camera based on the light reflection condition; performing voxel downsampling and normal vector estimation on 3D point cloud data, extracting photovoltaic support features, extracting edge data based on a 2D plane image, performing key feature point matching of the photovoltaic support features and the edge features, and constructing a three-dimensional photovoltaic scene model; performing accurate registration by using an ICP algorithm with a normal vector constraint term, and combining PID control to realize dynamic convergence of installation errors; and after the installation task is completed, updating the three-dimensional photovoltaic scene model for use in the next installation. According to the application, strong light reflection interference is inhibited by using an improved exposure control algorithm, and a matching and registration mechanism is designed according to photovoltaic support features, so that a high-precision and automatic support installation process is realized.

Description

Technical Field

[0001] This invention relates to the field of photovoltaic bracket installation, and in particular to a smart installation method for photovoltaic brackets based on 3D vision. Background Technology

[0002] With the continuous growth of global demand for renewable energy, photovoltaic (PV) power generation, as a clean and renewable energy source, has been widely applied and promoted. During the installation of PV systems, the installation location and angle of the PV mounting brackets have a crucial impact on the efficiency of the PV panels in receiving solar radiation. Traditional PV mounting bracket installation methods mainly rely on manual measurement and adjustment. This method is not only inefficient but also makes it difficult to guarantee installation accuracy, resulting in the PV panels not maximizing their solar radiation reception and thus affecting the overall power generation efficiency.

[0003] In addition, the surface of photovoltaic panels has a high reflectivity (usually between 0.8 and 0.95), which easily generates glare and high reflectivity interference under strong light conditions, making it difficult for conventional visual positioning to achieve reliable results.

[0004] 3D vision technology and artificial intelligence (AI) algorithms have developed rapidly in recent years. Sensors such as binocular 3D cameras, structured light, and LiDAR are widely used in industrial inspection and positioning. Meanwhile, deep learning models such as YOLO and Mask R-CNN have demonstrated excellent performance in object detection and segmentation. Existing technologies, such as Chinese invention patent CN119217379A, disclose a photovoltaic module grasping method based on point cloud and RGBD images. This method identifies the photovoltaic panel position in a two-step process, guiding a robotic arm to automatically grasp and place the panel. This improves the automation level, accuracy, and efficiency of photovoltaic module installation. By combining the high-precision spatial information of point cloud data with the color and depth information of RGBD images, it achieves rapid identification, positioning, and accurate grasping of photovoltaic modules, thereby reducing manual intervention and improving the safety and reliability of the installation process.

[0005] However, due to the high reflectivity of photovoltaic panels and the complexity of the installation site environment, there are still problems with insufficient identification robustness and positioning accuracy. Summary of the Invention

[0006] To address the technical problems existing in the prior art, the present invention aims to provide a 3D vision-based intelligent installation method for photovoltaic brackets. This method suppresses strong reflective interference through an improved exposure control algorithm and designs a matching and registration mechanism based on the characteristics of photovoltaic brackets, thereby achieving a high-precision and automated bracket installation process.

[0007] To achieve the above-mentioned objectives, this invention provides a smart installation method for photovoltaic brackets based on 3D vision, comprising the following steps:

[0008] Step S1: Collect environmental data, obtain the reflection of the photovoltaic panel, and adjust the camera parameters of the split-type binocular 3D camera based on the reflection.

[0009] Step S2: Acquire 3D depth image and 2D planar image to obtain 3D point cloud data;

[0010] Step S3: Perform voxel downsampling and normal vector estimation on the 3D point cloud data to extract photovoltaic support features. Extract edge data based on the 2D planar image and match key feature points of photovoltaic support features and edge features to construct a three-dimensional photovoltaic scene model.

[0011] Step S4: Based on the three-dimensional photovoltaic scene model, the ICP algorithm with introduced normal vector constraint terms is used for accurate registration, and combined with PID control to achieve dynamic convergence of installation error;

[0012] Step S5: After the installation task is completed, update the 3D photovoltaic scene model for use in the next installation based on the 3D photovoltaic scene model.

[0013] According to a technical solution of the present invention, in step S1, an adaptive exposure control algorithm is adopted to dynamically adjust the intensity of the auxiliary light source based on the reflectivity of the photovoltaic panel surface when the sun trajectory, light direction and intensity are constantly changing, and synchronous acquisition is performed using anti-reflection mode and standard mode.

[0014] Color and histogram analysis were performed on the two acquired images to determine whether the current exposure effect met the requirements of the next analysis step.

[0015] Images that meet the requirements are input into the photovoltaic panel detection model for judgment, and the exposure mode is determined based on whether there are detection results and the confidence level of the results.

[0016] According to one technical solution of the present invention, in step S1, the adaptive exposure control algorithm includes:

[0017] The light source intensity is dynamically adjusted based on the surface reflectivity of the photovoltaic panel and the solar trajectory.

[0018] The exposure value is adjusted based on the baseline exposure parameters and the current solar position variable.

[0019] According to one technical solution of the present invention, step S3 specifically includes:

[0020] The 3D point cloud data was downsampled using voxels, with each voxel being 5 mm in size. 3 or 3mm 3 To reduce the amount of data while retaining the necessary geometric features;

[0021] Based on the downsampled point cloud data, the normal vector of each point is estimated using the least squares method for subsequent surface feature analysis.

[0022] Mask R-CNN was used for instance segmentation to identify and extract the ROI region of the photovoltaic support.

[0023] The contour of the ROI region is extracted using the Canny edge detection algorithm, and key feature points are matched using the SIFT algorithm to achieve precise positioning of the photovoltaic support.

[0024] A three-dimensional photovoltaic scene model is constructed, which includes at least: the location of the photovoltaic support, the installation location of the photovoltaic panel, and the installation angle.

[0025] According to one technical solution of the present invention, step S4 specifically includes:

[0026] Step S41: Determine the number of key feature matching points;

[0027] Step S42: If the number of matching points is greater than or equal to 4, proceed to step S43; if the number of matching points is less than 4, use the template matching method for pose estimation.

[0028] Step S43: Use the SVD method to calculate the pose, requiring the calculation error to be less than 1.5 pixels;

[0029] Step S44: Calculate the solar angle γ based on the solution results. The formula is:

[0030]

[0031] Where α is the solar altitude angle, β is the solar azimuth angle, and θ is the current tilt angle of the support.

[0032] Step S45: Drive the actuator based on the angle difference △θ.

[0033] According to one technical solution of the present invention, step S45 specifically includes:

[0034] Determine whether the angle difference Δθ is greater than the preset angle threshold Δθ0;

[0035] If Δθ > Δθ0, then by adjusting the actuator, the proportional coefficient Kp of the PID controller is set to 2.5;

[0036] If △θ≤△θ0, then maintain the current state and drive the actuator;

[0037] The preset angle threshold △θ0 has a range of 0°<△θ0≤1°.

[0038] According to one technical solution of the present invention, step S5 further includes judging the installation error, specifically including:

[0039] After the actuator operates, check whether the installation error is less than 1mm;

[0040] If the installation error is not less than 1mm, increment the retry counter by 1 and determine whether the number of retries is less than 3.

[0041] If the number of retries is less than 3, return to the synchronous acquisition step;

[0042] If the number of retries reaches or exceeds 3, manual intervention will be performed;

[0043] If the installation error is less than 1mm, the task is completed, and the parameters of the 3D photovoltaic scene model are updated for use in the next installation.

[0044] According to one technical solution of the present invention, the optimization objective function E of the ICP algorithm with introduced normal vector constraint term is expressed as:

[0045]

[0046] Where, n i p Let n represent the source point cloud normal vector. i q Let λ represent the target point cloud normal vector, λ represent the experimental optimization weights, T represent the 3D transformation matrix, and q represent the target point cloud normal vector. i The i-th point in the source point cloud, p i Indicates the target point cloud and q i The corresponding nearest neighbor.

[0047] According to one technical solution of the present invention, the objective function of the ICP algorithm with introduced normal vector constraint term is solved by Levenberg-Marquardt optimization, and the Jacobian matrix is ​​supplemented with a normal vector partial derivative term, expressed as:

[0048]

[0049] Among them, a i Let b represent the coordinates of the i-th point in the target point cloud. i The coordinates of the i-th point in the source point cloud, n a Represents target point a i The normal vector, n b Represents source point b i The normal vector is ξ, which represents the rigid body transformation parameter.

[0050] According to one technical solution of the present invention, the method further includes the following step before step S1:

[0051] Activate the force sensor installed at the end of the installation tool to monitor the contact force and torque information in real time during the installation process;

[0052] When visual data is difficult to recognize due to high reflectivity or partial occlusion, tactile feedback is used to help determine the positional relationship between the bracket and the camera, and to adjust the camera position / posture.

[0053] Compared with the prior art, the present invention has the following beneficial effects:

[0054] This invention discloses an intelligent photovoltaic (PV) bracket installation method based on 3D vision. By combining 3D vision and artificial intelligence technologies, it utilizes a split-type binocular 3D camera to simultaneously acquire high-resolution 3D depth images and 2D color images, and fuses point cloud data to achieve accurate identification of the PV bracket. First, environmental data is collected to determine the reflectivity of the PV panel surface, and camera parameters are dynamically adjusted based on the reflectivity to ensure the quality and stability of visual data acquisition. An improved exposure control algorithm suppresses strong reflective interference, and a matching and registration mechanism is designed specifically for the characteristics of the PV bracket, thereby achieving a high-precision, automated bracket installation process.

[0055] Voxel downsampling and normal vector estimation are used to preprocess point cloud data to reduce computational complexity while preserving geometric details. Combining Mask R-CNN segmentation and Canny edge detection, the region of interest (ROI) of the photovoltaic (PV) support is accurately identified and key feature points are extracted. Subsequently, an improved SIFT algorithm is used to match the key feature points of the PV support features with those of the edge features, achieving precise positioning of the support.

[0056] In the registration stage, for feature point matching, the ICP algorithm with introduced normal vector constraints is used for high-precision registration. This is combined with a PID controller to adjust the actuator and dynamically converge installation errors. Finally, after installation, the 3D photovoltaic scene model is updated to support the efficient and rapid execution of subsequent installation tasks, ensuring efficient and high-precision installation of the same photovoltaic support structure.

[0057] This invention enables photovoltaic bracket installation errors to be controlled within 1mm under complex lighting and high reflectivity environments. This not only improves installation accuracy and automation, but also results in photovoltaic panels installed using the method of this invention having better solar reception efficiency and power generation performance, greatly promoting the intelligent and standardized process of photovoltaic system installation. Attached Figure Description

[0058] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings without any creative effort.

[0059] Figure 1 This is a schematic diagram illustrating a process of a 3D vision-based intelligent installation method for photovoltaic brackets according to an embodiment of the present invention.

[0060] Figure 2 This schematic diagram illustrates a process flow of a 3D vision-based intelligent installation method for photovoltaic brackets according to another embodiment of the present invention. Detailed Implementation

[0061] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0062] This invention discloses a 3D vision-based intelligent installation method for photovoltaic (PV) brackets. By combining 3D vision technology, adaptive exposure control algorithms, and artificial intelligence deep learning models, it achieves rapid and high-precision installation and positioning of PV brackets in highly reflective environments. Improvements to the ICP algorithm and the integration of force sensors further enhance system robustness, maintaining an installation accuracy of ≤1mm even under complex outdoor conditions. This method can be widely applied to the construction of PV power plants of various scales, providing technical support for the intelligent upgrading of the PV industry.

[0063] like Figure 1 and Figure 2 As shown, the present invention provides a smart installation method for photovoltaic brackets based on 3D vision, comprising the following steps:

[0064] Step S1: Collect environmental data, obtain the reflection of the photovoltaic panel, and adjust the camera parameters of the split-type binocular 3D camera based on the reflection.

[0065] Step S2: Acquire 3D depth image and 2D planar image to obtain 3D point cloud data;

[0066] Step S3: Perform voxel downsampling and normal vector estimation on the 3D point cloud data to extract photovoltaic support features. Extract edge data based on the 2D planar image and match key feature points of photovoltaic support features and edge features to construct a three-dimensional photovoltaic scene model.

[0067] Step S4: Based on the three-dimensional photovoltaic scene model, the ICP algorithm with introduced normal vector constraint terms is used for accurate registration, and combined with PID control to achieve dynamic convergence of installation error;

[0068] Step S5: After the installation task is completed, update the 3D photovoltaic scene model for use in the next installation based on the 3D photovoltaic scene model.

[0069] By combining 3D vision and artificial intelligence technologies, a split-type binocular 3D camera simultaneously acquires high-resolution 3D depth images and 2D color images, and fuses point cloud data to achieve accurate identification of photovoltaic (PV) brackets. First, environmental data is collected to determine the reflectivity of the PV panel surface, and camera parameters are dynamically adjusted based on this to ensure the quality and stability of visual data acquisition. Furthermore, voxel downsampling and normal vector estimation are used to preprocess the point cloud data to reduce computational complexity while preserving geometric details. Combining the Mask R-CNN segmentation algorithm and Canny edge detection, the region of interest (ROI) of the PV bracket is accurately identified and key feature points are extracted. Subsequently, an improved SIFT algorithm is used to match the key feature points of the PV bracket features with edge features, achieving precise positioning of the bracket.

[0070] In the registration stage, for feature point matching, the ICP algorithm with introduced normal vector constraints is used for high-precision registration. This is combined with a PID controller to adjust the actuator and dynamically converge installation errors. Finally, after installation, the 3D photovoltaic scene model is updated to support the efficient and rapid execution of subsequent installation tasks, ensuring efficient and high-precision installation of the same photovoltaic support structure.

[0071] This invention enables photovoltaic bracket installation errors to be controlled within 1mm under complex lighting and high reflectivity environments. This not only improves installation accuracy and automation, but also allows photovoltaic panels installed using the method of this invention to have better solar reception efficiency and power generation performance, greatly promoting the intelligent and standardized process of photovoltaic system installation.

[0072] In some embodiments of the present invention, step S1 employs an adaptive exposure control algorithm to dynamically adjust the intensity of the auxiliary light source based on the reflectivity of the photovoltaic panel surface when the sun trajectory, illumination direction, and intensity are constantly changing, and to simultaneously collect data using anti-reflective mode and standard mode.

[0073] Color and histogram analysis were performed on the two acquired images to determine whether the current exposure effect met the requirements of the next analysis step.

[0074] Images that meet the requirements are input into the photovoltaic panel detection model for judgment, and the exposure mode is determined based on whether there are detection results and the confidence level of the results.

[0075] In some embodiments of the present invention, the adaptive exposure control algorithm includes:

[0076] The light source intensity is dynamically adjusted based on the surface reflectivity of the photovoltaic panel and the solar trajectory.

[0077] The exposure value is adjusted based on the baseline exposure parameters and the current solar position variable.

[0078] After obtaining the basic exposure parameters through ambient light detection and synchronous acquisition, the exposure parameters of this operation are used as the benchmark for this operation.

[0079] Each time a photo is taken, the previous exposure parameters are used as a reference, and the angle is adjusted accordingly based on the current sun position and the variables compared to the reference exposure.

[0080] By introducing an adaptive exposure control algorithm, the system dynamically monitors the reflectivity of the photovoltaic panel surface based on real-time changes in the sun's trajectory, illumination direction, and intensity. It then adjusts the intensity of the auxiliary light source and camera exposure parameters accordingly, effectively suppressing high-reflectivity interference. Specifically, the system employs two acquisition modes—anti-reflective mode and standard mode. Through simultaneous image acquisition, color and histogram analysis are used to automatically determine whether the current exposure meets the recognition requirements. The system then combines this with the confidence level of the photovoltaic panel detection model to determine the switching of the exposure mode. This dynamic adjustment mechanism not only optimizes image quality and enhances feature clarity but also effectively avoids image information loss and misidentification caused by strong light reflection.

[0081] In some embodiments of the present invention, when the ambient brightness is detected to be greater than 220 Lux, the anti-reflection mode is entered, the polarizer angle is adjusted to 45°, and the camera exposure time is reduced by 30% to reduce strong light reflection interference.

[0082] When the ambient brightness is detected to be less than or equal to 220 Lux, the system enters standard mode and maintains the camera's default parameters.

[0083] When the ambient light intensity exceeds 220 Lux, the system automatically enters anti-reflective mode, reducing strong light interference by adjusting the polarizer to 45° and shortening the exposure time; while maintaining standard mode and optimizing shooting parameters under low light conditions; if the light intensity is below 200 lux / cm²... 2When the system detects extremely low light conditions, it switches to night mode and activates auxiliary light sources to ensure complete target information acquisition. This greatly enhances the system's adaptability to different lighting conditions, ensuring high-quality visual data can still be obtained in complex natural light environments. This provides stable and reliable input for subsequent identification and positioning processes, significantly improving the accuracy of photovoltaic bracket identification and the robustness of the entire automated installation system.

[0084] In some embodiments of the present invention, step S3 specifically includes:

[0085] The 3D point cloud data was downsampled using voxels, with each voxel being 5 mm in size. 3 or 3mm 3 To reduce data volume while retaining necessary geometric features; preferably, 5mm is used. 3 Experiments have verified that when the voxel downsampling size is 5mm... 3 At the same time, the key geometric features of the photovoltaic support can be preserved, while reducing the amount of data processing by about 70%.

[0086] Based on the downsampled point cloud data, the normal vector of each point is estimated using the least squares method for subsequent surface feature analysis.

[0087] Mask R-CNN was used for instance segmentation to identify and extract the ROI region of the photovoltaic support.

[0088] The contour of the ROI region is extracted using the Canny edge detection algorithm, and key feature points are matched using the SIFT algorithm to achieve precise positioning of the photovoltaic support.

[0089] A three-dimensional photovoltaic scene model is constructed, which includes at least: the location of the photovoltaic support, the installation location of the photovoltaic panel, and the installation angle.

[0090] By performing voxel downsampling and normal vector estimation on 3D point cloud data, the data volume is effectively reduced, achieving a concise representation of the point cloud while retaining necessary geometric features, thus improving the computational efficiency and accuracy of subsequent processing. Based on this preprocessing result, the system adopts the Mask R-CNN instance segmentation algorithm to accurately identify and segment the region of interest (ROI) of the photovoltaic support, overcoming the bottleneck problem of traditional methods' difficulty in identification under feature scarcity and complex backgrounds. Subsequently, the Canny edge detection algorithm is used to extract contour information from the ROI, enhancing the understanding of the support morphology and effectively improving the accuracy of feature point localization. In terms of key feature point matching, the SIFT algorithm is applied. During SIFT matching, an Euclidean distance ratio threshold of 0.8 is used to filter out mismatched points, which not only ensures the stability of matching but also improves the robustness of key point matching, ensuring accurate localization even under changes in illumination and partial occlusion.

[0091] Overall, the data preprocessing and feature extraction matching process of this invention effectively solves the challenges of simple photovoltaic support structure and severe reflective interference through multi-level and multi-algorithm collaborative work, and achieves high-precision detection and positioning of the target, laying a solid data foundation for the subsequent high-precision installation process.

[0092] The Mask R-CNN model was trained on a labeled dataset of photovoltaic brackets (containing 1,000 images under different lighting conditions), with an input image resolution of 1024×1024 and a confidence threshold of 0.7.

[0093] The present invention also constructs a three-dimensional photovoltaic scene model, which includes at least: the position of the photovoltaic support, the installation position and installation angle of the photovoltaic panel, and can provide global environmental information support for subsequent installation path planning, obstacle avoidance and pose optimization.

[0094] In some embodiments of the present invention, step S4 specifically includes:

[0095] Step S41: Determine the number of key feature matching points;

[0096] Step S42: If the number of matching points is greater than or equal to 4, proceed to step S43; if the number of matching points is less than 4, use the template matching method for pose estimation.

[0097] Step S43: Use the SVD method to calculate the pose, requiring the calculation error to be less than 1.5 pixels;

[0098] Step S44: Calculate the solar angle γ based on the solution results. The formula is:

[0099]

[0100] Where α is the solar altitude angle, β is the solar azimuth angle, and θ is the current tilt angle of the support.

[0101] Step S45: Drive the actuator based on the angle difference Δθ, specifically including:

[0102] Determine whether the angle difference Δθ is greater than the preset angle threshold Δθ0;

[0103] If Δθ > Δθ0, then by adjusting the actuator, the proportional coefficient Kp of the PID controller is set to 2.5;

[0104] If △θ≤△θ0, then maintain the current state and drive the actuator;

[0105] The preset angle threshold △θ0 has a value range of 0°<△θ0≤1°, preferably 0.5°.

[0106] An actuator is a component in equipment used for photovoltaic installation that picks up photovoltaic panels; it is usually a robotic arm.

[0107] By judging the number of key feature matching points, the most suitable pose calculation method is intelligently selected to ensure the accuracy and reliability of position and angle calculation.

[0108] For cases with ≥4 matching points, the Singular Value Decomposition (SVD) algorithm is used for high-precision pose calculation, ensuring a calculation error of less than 1.5 pixels, effectively improving the positioning accuracy during bracket installation. Based on the SVD calculation results, the mathematical formula γ=arcsin(cosα·cosβ / √(1+tan 2 θ)) accurately calculates the optimal installation angle γ of the photovoltaic bracket, taking into account the solar altitude angle, solar azimuth angle and current bracket tilt angle, and dynamically reflects the real-time changes in the solar position.

[0109] Subsequently, by determining whether the adjustment angle difference Δθ exceeds the preset threshold (0°<Δθ0≤1°), a PID controller (proportional coefficient Kp=2.5) is used to drive the actuator to achieve precise adjustment of the bracket angle and convergence of dynamic error.

[0110] If there are fewer than 4 matching points, a template matching method is used for pose estimation to ensure reasonable positioning even when data is scarce.

[0111] This invention achieves optimized adjustment of the photovoltaic support angle by combining precise mathematical calculation and intelligent control. When △θ≤△θ0, the current state is maintained and the actuator is driven; when △θ>△θ0, a PID controller (proportional coefficient Kp=2.5) is used to drive the actuator, making the installation process smoother.

[0112] In some embodiments of the present invention, step S5 further includes determining the installation error, specifically including:

[0113] After the actuator operates, check whether the installation error is less than 1mm;

[0114] If the installation error is not less than 1mm, increment the retry counter by 1 and determine whether the number of retries is less than 3.

[0115] If the number of retries is less than 3, return to the synchronous acquisition step;

[0116] If the number of retries reaches or exceeds 3, manual intervention will be performed;

[0117] If the installation error is less than 1mm, the task is completed, and the parameters of the 3D photovoltaic scene model are updated for use in the next installation.

[0118] By implementing an installation error detection mechanism after the task is completed, we ensure that the accuracy of each photovoltaic bracket installation is strictly controlled within 1mm, guaranteeing that the installation quality meets the actual requirements of the project.

[0119] After the actuator moves, the system monitors the installation error in real time. If the error is not less than 1mm, a retry mechanism is automatically triggered, with a maximum of 3 retries. The system re-identifies and adjusts the error by returning to synchronous data collection, ensuring effective correction. If the error requirement is still not met after reaching the maximum number of retries, manual intervention is prompted to prevent continuous errors from affecting the overall process and efficiency. If the error meets the requirements, the installation task is considered complete, and the parameters of the 3D photovoltaic scene model are updated so that the updated 3D photovoltaic scene model can be directly used for the next installation task.

[0120] Meanwhile, the model parameters of this task, including camera adjustment parameters, feature extraction model and pose calculation results, are retained. These parameters can be used when the time interval between the next task and the previous task is less than the preset time, thus providing accumulation and support for rapid installation in similar scenarios in the future.

[0121] Furthermore, the intelligent photovoltaic (PV) bracket installation system dynamically updates the 3D PV scene model. It only requires acquiring a single 3D depth image to install all PV panels on the bracket, improving the efficiency of subsequent tasks and continuously enhancing the accuracy of identification and installation. Through closed-loop error control and iterative scene model updates, it achieves high reliability and intelligence in the installation process, significantly improving the practicality and stability of the automated installation system.

[0122] In some embodiments of the present invention, the optimization objective function E of the ICP algorithm with introduced normal vector constraint term is expressed as:

[0123]

[0124] Where, n i p Let n represent the source point cloud normal vector. i q Let λ represent the target point cloud normal vector, λ represent the experimental optimization weights, T represent the 3D transformation matrix, and q represent the target point cloud normal vector. i The i-th point in the source point cloud (point cloud to be registered), p i Indicates the target point cloud and q i The corresponding nearest neighbor.

[0125] The objective function of the ICP algorithm with introduced normal vector constraints is solved using Levenberg-Marquardt optimization, with the Jacobian matrix incorporating the normal vector partial derivative term, and is expressed as:

[0126]

[0127] Among them, a i Let b represent the coordinates of the i-th point in the target point cloud. i The coordinates of the i-th point in the source point cloud, n a Represents target point a i The normal vector, n b Represents source point b i The normal vector is ξ, which represents the rigid body transformation parameter.

[0128] By introducing a normal vector constraint term to optimize the traditional ICP algorithm, a normal vector similarity evaluation is added to the point cloud registration process. Point pairs with similar shapes and orientations are matched first, which significantly reduces the registration error caused by rough or occluded point cloud surfaces.

[0129] The objective function is optimized by combining normal vector information with spatial positional relationships, and iteratively solved using the Levenberg-Marquardt nonlinear least squares method. Partial derivative terms of the normal vector are added to the Jacobian matrix to enhance the sensitivity of the registration process to geometric details and improve optimization speed. This optimization mechanism not only improves the stability and accuracy of point cloud registration but also effectively controls the stringent requirement of an overall positioning error below 1 mm.

[0130] This demonstrates that the ICP algorithm is more robust in handling complex surfaces and fine-tuning deformations of photovoltaic supports, effectively avoiding the drawbacks of traditional ICP, which is prone to getting trapped in local optima or accumulating coarse registration errors. Point cloud registration technology lays a solid foundation for the high-precision positioning and installation of photovoltaic supports, and also provides algorithmic demonstrations and technical references for other precision industrial assembly in this field.

[0131] In some embodiments of the present invention, the method further includes the following step before step S1:

[0132] Activate the force sensor installed at the end of the installation tool to monitor the contact force and torque information in real time during the installation process;

[0133] When visual data is difficult to recognize due to high reflectivity or partial occlusion, tactile feedback is used to help determine the positional relationship between the bracket and the camera, and to adjust the camera position / posture.

[0134] By installing a force sensor at the end of the tool, multimodal fusion of vision and touch is achieved, significantly enhancing the system's positioning capability in highly reflective and obstructed conditions.

[0135] Force sensors monitor the contact force and torque information on the photovoltaic support in real time during installation, helping to determine the spatial relationship between the camera and the support. When visual data is difficult to identify due to strong reflection, obstruction, or lack of object features, the system uses tactile information to adjust the camera position and posture to supplement the lack of visual information.

[0136] This invention effectively compensates for blind spots in visual perception, ensuring stable capture and positioning of photovoltaic (PV) brackets. Through tactile feedback, the system can dynamically adjust its operational strategies, quickly adapting to complex environmental conditions and improving overall recognition accuracy and installation efficiency. Simultaneously, cross-modal sensor fusion provides a more flexible and intelligent perception solution for automated installation robots, significantly improving the system's robustness and reliability, and effectively supporting the precise deployment of PV brackets in varied real-world environments.

[0137] For example, when the visual recognition confidence level is below 0.6, the system switches to tactile-dominated mode, which detects the direction of the contact force through a force sensor. If the vertical force exceeds 5N, the robotic arm is triggered to retract and reposition itself.

[0138] Industrial application effects:

[0139] After testing and verification at multiple photovoltaic power plants, this invention can maintain high recognition accuracy even in strong light and high reflectivity environments, with an average installation error controllable within 0.8 to 1 mm, and installation efficiency improved by 50 to 80% compared to traditional manual methods.

[0140] It should be noted that although the embodiments described above are illustrative, they are not intended to limit the invention. Therefore, the invention is not limited to the specific embodiments described above. Any other embodiments obtained by those skilled in the art under the guidance of this invention without departing from its principles are considered to be within the protection scope of this invention.

Claims

1. A 3D vision-based intelligent installation method for photovoltaic racks, characterized in that, The method comprises the following steps: Step S1, collecting environmental data, obtaining the reflection condition of the photovoltaic panel, and adjusting the camera parameters of the split binocular 3D camera based on the reflection condition; Step S2, obtaining 3D depth images and 2D plane images to obtain 3D point cloud data; Step S3, performing voxel downsampling and normal vector estimation on the 3D point cloud data, extracting photovoltaic support features, extracting edge data based on the 2D plane images, and performing key feature point matching of the photovoltaic support features and the edge features to construct a three-dimensional photovoltaic scene model; Step S4, based on the three-dimensional photovoltaic scene model, using an ICP algorithm with a normal vector constraint term for accurate registration, and combining a PID controller to realize dynamic convergence of installation errors, specifically comprising: Step S41, judging the number of key feature matching points; Step S42, if the number of matching points is greater than or equal to 4, step S43 is executed; if the number of matching points is less than 4, a template matching method is used for pose estimation; Step S43, using the SVD method for pose solution, requiring that the solution error be less than 1.5 pixels; Step S44, according to the solution result, calculating the solar angle γ, the formula being: wherein α is the solar elevation angle, β is the solar azimuth angle, and θ is the current support inclination angle; Step S45, driving the actuator based on the angle difference Δθ; Step S5, after the installation task is completed, updating the three-dimensional photovoltaic scene model for use in the next installation.

2. The 3D vision based photovoltaic mounting system intelligent installation method according to claim 1, characterized in that, In the step S1, an adaptive exposure control algorithm is used to dynamically adjust the intensity of the auxiliary light source according to the reflectivity of the photovoltaic panel surface under the condition that the sun's trajectory, light direction and intensity are constantly changing, and the anti-reflection mode and the standard mode are used for synchronous collection; The collected images of the two modes are analyzed in terms of color and histogram to determine whether the current exposure effect imaging meets the requirements of the next step of analysis; The images that meet the requirements are input into the photovoltaic panel detection model for judgment, and the exposure mode is determined according to whether there is a detection result and the confidence of the result.

3. The 3D vision based photovoltaic mounting system intelligent installation method according to claim 2, characterized in that, In the step S1, the adaptive exposure control algorithm comprises: dynamically adjusting the light source intensity according to the reflectivity of the photovoltaic panel surface and the sun's trajectory; based on the reference exposure parameters, correcting the exposure value according to the current sun position variable.

4. The 3D vision based photovoltaic mounting system intelligent installation method according to claim 1, characterized in that, In the step S3, specifically comprising: voxel down-sampling the 3D point cloud data with a down-sampling voxel size of 5mm 3 or 3mm 3 to reduce the data volume and preserve necessary geometric features; based on the downsampled point cloud data, estimating the normal vector of each point using the least squares method for subsequent surface feature analysis; using Mask R-CNN for instance segmentation to identify and extract the ROI region of the photovoltaic support; using the Canny edge detection algorithm to extract the contour of the ROI region, and matching the key feature points through the SIFT algorithm to realize accurate positioning of the photovoltaic support; constructing a three-dimensional photovoltaic scene model, the three-dimensional photovoltaic scene model at least comprising: the position of the photovoltaic support, the installation position and installation angle of the photovoltaic panel.

5. The 3D vision based photovoltaic mount intelligent installation method of claim 1, wherein, In the step S45, specifically comprising: judging whether the angle difference Δθ is greater than a preset angle threshold Δθ0; if Δθ>Δθ0, adjusting the actuator, the proportional coefficient Kp of the PID controller being 2.5; if Δθ≤Δθ0, maintaining the current state and driving the actuator; The preset angle threshold △θ0 has a range of 0°<△θ0≤1°.

6. The 3D vision based photovoltaic mount intelligent installation method of claim 1, wherein, Step S5 also includes judging the installation error, specifically including: After the actuator operates, check whether the installation error is less than 1mm; If the installation error is not less than 1mm, increment the retry counter by 1 and determine whether the number of retries is less than 3. If the number of retries is less than 3, return to the synchronous acquisition step; If the number of retries reaches or exceeds 3, manual intervention will be performed; If the installation error is less than 1mm, the task is completed, and the parameters of the 3D photovoltaic scene model are updated for use in the next installation.

7. The 3D vision based photovoltaic mount intelligent installation method of claim 1, wherein, The objective function E of the ICP algorithm with introduced normal vector constraints is expressed as: in, Represents the source point cloud normal vector. Let λ represent the target point cloud normal vector, λ represent the experimental optimization weights, T represent the 3D transformation matrix, and q represent the target point cloud normal vector. i The i-th point in the source point cloud, p i Indicates the target point cloud and q i The corresponding nearest neighbor.

8. The 3D vision based photovoltaic mounting system intelligent installation method according to claim 7, characterized in that, The objective function of the ICP algorithm with introduced normal vector constraints is solved using Levenberg-Marquardt optimization, with the Jacobian matrix incorporating the normal vector partial derivative term, and is expressed as: where a i denotes the coordinates of the i-th point in the target point cloud, b i denotes the coordinates of the i-th point in the source point cloud, n a denotes the normal vector of the target point a i , n b denotes the normal vector of the source point b i , and ξ denotes the rigid body transformation parameters.

9. The 3D vision based photovoltaic mount intelligent installation method of claim 1, wherein, The steps preceding step S1 also include: Activate the force sensor installed at the end of the installation tool to monitor the contact force and torque information in real time during the installation process; When visual data is difficult to recognize due to high reflectivity or partial occlusion, tactile feedback is used to help determine the positional relationship between the bracket and the camera, and to adjust the camera position / posture.

Images