Method and system for positioning junction box of photovoltaic module

Abstract

The embodiment of the invention relates to a positioning method and system for a junction box of a photovoltaic module. The positioning method comprises the steps that an industrial camera with internal and external parameter calibration completed collects image data of a target photovoltaic module; acquiring 3D point cloud data of a target junction box on the target photovoltaic module through the structured light laser sensor subjected to internal and external parameter calibration; marking position information of a target junction box in the image data by adopting a pre-trained deep learning positioning model; extracting contour information of the target junction box based on the 3D point cloud data of the target junction box; performing data fusion on the position information and the contour information; and converting the fused data into the same coordinate system to calculate the target position information of the target junction box. Therefore, advantages of various sensors are integrated, high-precision, high-speed and lossless positioning of the photovoltaic module junction box is realized, and reliable technical support is provided for automatic production of photovoltaic modules.

Description

Technical Field

[0001] The embodiments of the present invention relate to the technical field of photovoltaic module junction box positioning and data processing, and in particular to a method and system for positioning a photovoltaic module junction box. Background Art

[0002] In the automated production and quality control of photovoltaic modules, the precise positioning of the junction box is a key link. The existing technologies mainly use positioning methods based on machine vision, positioning methods based on laser ranging, or positioning methods based on tactile sensing. Among them, the positioning method based on machine vision is easily affected by factors such as lighting conditions and surface reflection, resulting in unstable positioning accuracy; the positioning method based on laser ranging is difficult to identify the specific contour and posture of the junction box, and the measurement points are limited, making it difficult to achieve comprehensive positioning; the positioning method based on tactile sensing has a slow measurement speed and may cause damage to the surface of the junction box. Therefore, how to quickly, accurately, and non-destructively locate the junction box has become an urgent problem to be solved. Summary of the invention

[0003] In view of this, in order to solve the above technical problems or part of the technical problems, an embodiment of the present invention provides a method and system for positioning a junction box of a photovoltaic module.

[0004] In a first aspect, an embodiment of the present invention provides a method for positioning a junction box of a photovoltaic module, comprising:

[0005] The industrial camera, which has been calibrated with internal and external parameters, collects image data of the target photovoltaic module;

[0006] The structured light laser sensor, which has been calibrated with internal and external parameters, collects 3D point cloud data of the target junction box on the target photovoltaic module;

[0007] Annotating location information of a target junction box in the image data using a pre-trained deep learning positioning model;

[0008] Extracting contour information of the target junction box based on the 3D point cloud data of the target junction box;

[0009] Performing data fusion on the position information and the contour information;

[0010] The fused data is converted into the same coordinate system to calculate the target position information of the target junction box.

[0011] In a possible implementation, the method further includes:

[0012] Collecting multiple photovoltaic module images under different environmental variables and marking the positions of junction boxes in the multiple photovoltaic module images;

[0013] Perform data enhancement processing on multiple photovoltaic module images with the locations of junction boxes marked to obtain a model training data set;

[0014] The improved YOLOv5 network is trained based on the model training data set to obtain a deep learning positioning model.

[0015] In a possible implementation, the method further includes:

[0016] The image data is input into the pre-trained deep learning positioning model to obtain a target junction box detection result output by the model, wherein the target junction box detection result includes the model, location information, posture and geometric dimensions of the target junction box.

[0017] In a possible implementation, the method further includes:

[0018] Performing noise reduction processing on the 3D point cloud data;

[0019] The RANSAC algorithm is used to fit the plane image of the target junction box surface without removing background points;

[0020] A three-dimensional Canny operator is used to extract contour information of the target junction box from the plane image.

[0021] In a possible implementation, the method further includes:

[0022] The Kalman filter algorithm is used to dynamically adjust the weights of the data acquired by different sensors according to the image clarity and the 3D point cloud density, and the position information and contour information of the target junction box are aligned in time and space and fused.

[0023] In a possible implementation, the method further includes:

[0024] Perform coordinate transformation on the fused data in different coordinate systems and transform them into the same world coordinate system;

[0025] The target position information of the target junction box is calculated based on the data in the same world coordinate system.

[0026] In a possible implementation, the method further includes:

[0027] Based on preset test conditions and test standards, the calculated target position information of the target junction box is tested and verified to obtain a test verification result;

[0028] The algorithm parameters are optimized based on the test verification results.

[0029] In a second aspect, an embodiment of the present invention provides a positioning system for a junction box of a photovoltaic module, comprising:

[0030] Multi-sensor integrated module, used to collect image data of target photovoltaic modules through industrial cameras calibrated with internal and external parameters;

[0031] The multi-sensor integrated module is also used to collect 3D point cloud data of the target junction box on the target photovoltaic module through the structured light laser sensor that has completed the internal and external parameter calibration;

[0032] An image processing module, used to annotate the location information of the target junction box in the image data using a pre-trained deep learning positioning model;

[0033] A data feature extraction module, used to extract contour information of the target junction box based on the 3D point cloud data of the target junction box;

[0034] A data fusion module, used for fusing the position information and the contour information;

[0035] The coordinate transformation module is used to transform the fused data into the same coordinate system to calculate the target position information of the target junction box.

[0036] In a third aspect, an embodiment of the present invention provides an electronic device, comprising: a processor and a memory, wherein the processor is used to execute a positioning program for a junction box of a photovoltaic component stored in the memory to implement the positioning method for a junction box of a photovoltaic component described in the first aspect.

[0037] In a fourth aspect, an embodiment of the present invention provides a storage medium, including: the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the positioning method of the junction box of the photovoltaic component described in the first aspect above.

[0038] The positioning solution for the junction box of a photovoltaic module provided by an embodiment of the present invention collects image data of a target photovoltaic module through an industrial camera that has completed internal and external parameter calibration; collects 3D point cloud data of a target junction box on a target photovoltaic module through a structured light laser sensor that has completed internal and external parameter calibration; annotates the position information of the target junction box in the image data using a pre-trained deep learning positioning model; extracts contour information of the target junction box based on the 3D point cloud data of the target junction box; performs data fusion on the position information and contour information; and converts the fused data into the same coordinate system to calculate the target position information of the target junction box. Compared with the existing positioning methods for the junction box of photovoltaic modules, it is difficult to simultaneously meet the requirements of high precision, high speed and non-destructive measurement. This system, which combines the advantages of multiple sensors, realizes high-precision, high-speed and non-destructive positioning of the junction box of photovoltaic modules, providing reliable technical support for the automated production of photovoltaic modules. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 A schematic flow chart of a method for positioning a junction box of a photovoltaic module provided by an embodiment of the present invention;

[0040] Figure 2 A schematic diagram of a deep learning network structure provided by an embodiment of the present invention;

[0041] Figure 3 A multi-sensor data fusion flow chart provided by an embodiment of the present invention;

[0042] Figure 4 A coordinate system transformation relationship diagram provided by an embodiment of the present invention;

[0043] Figure 5 A schematic structural diagram of a positioning system for a junction box of a photovoltaic module provided by an embodiment of the present invention;

[0044] Figure 6 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0046] To facilitate understanding of the embodiments of the present invention, specific embodiments will be further explained below in conjunction with the accompanying drawings. The embodiments do not constitute a limitation on the embodiments of the present invention.

[0047] Figure 1 A schematic flow chart of a method for positioning a junction box of a photovoltaic module provided by an embodiment of the present invention is shown in FIG. Figure 1 As shown, the method specifically includes:

[0048] S11. The industrial camera that has completed the internal and external parameter calibration collects image data of the target photovoltaic module.

[0049] S12. The structured light laser sensor, which has been calibrated with internal and external parameters, collects 3D point cloud data of the target junction box on the target photovoltaic module.

[0050] In an embodiment of the present invention, image data of a target photovoltaic module is collected by a high-resolution industrial camera, 3D point cloud data of a target junction box on the target photovoltaic module is collected by a structured light laser sensor, and the spatial position of the sensor is accurately tracked by a high-precision encoder.

[0051] First, coordinate transformation and calibration are performed on high-resolution industrial cameras, structured light laser sensors, and robots carrying high-resolution industrial cameras and structured light laser sensors. The main function and purpose of camera calibration is to establish a geometric model of camera imaging, and to correct image distortion by calibration parameters, so as to finally reconstruct a three-dimensional scene from a two-dimensional image. Camera calibration is performed on high-resolution industrial cameras based on Zhang Zhengyou's calibration method. Laser sensor calibration refers to the process of estimating the internal and external parameters of laser sensors, including internal parameters such as the center of rotation of the laser beam, the tilt angle of the scanning surface, and the divergence of the laser beam, as well as external parameters such as the position and direction of the laser radar relative to the vehicle coordinate system. This process aims to accurately represent the point cloud data collected by the laser radar in the vehicle coordinate system. Robot hand-eye calibration is to determine the pose relationship between the camera coordinate system and the robot coordinate system, including two situations: the camera is installed at the end of the robotic arm and the camera is installed outside the robotic arm. This process involves solving the coordinate relationship between the camera coordinate system and the coordinate system of the end of the robotic arm (or the robot coordinate system) through two movements of the robotic arm.

[0052] S13. Use a pre-trained deep learning positioning model to annotate the location information of the target junction box in the image data.

[0053] In the embodiment of the present invention, a deep learning positioning model is pre-trained. Multiple photovoltaic module images under different environmental variables are collected and the locations of the junction boxes in the multiple photovoltaic module images are marked; data enhancement processing is performed on the multiple photovoltaic module images with the locations of the junction boxes marked to obtain a model training data set; and the improved YOLOv5 network is trained based on the model training data set to obtain a deep learning positioning model. Figure 2The structural diagram of the deep learning positioning model shown in the figure includes: input layer: receiving raw image data; backbone network (Backbone): responsible for extracting basic features of the image, which can be a deep convolutional neural network such as CSPDarknet53; feature pyramid network (FPN): used to fuse features of different scales and enhance the network's detection ability for objects of different sizes; detection head: responsible for the final target detection task, including bounding box regression and category prediction; output layer: output detection results, including the location, size and confidence of the junction box. This network structure diagram shows how the deep learning model processes image information layer by layer to ultimately achieve accurate junction box detection. The deep learning positioning model uses deep learning and parameterized models, combined with online learning mechanisms, to quickly adapt to new models of junction boxes. Only a small number of samples are needed to complete transfer learning, greatly reducing the changeover time.

[0054] Furthermore, the image data is input into a pre-trained deep learning positioning model to obtain a target junction box detection result output by the model, wherein the target junction box detection result includes the model, location information, posture and geometric dimensions of the target junction box.

[0055] S14. Extracting contour information of the target junction box based on the 3D point cloud data of the target junction box.

[0056] The 3D point cloud data is subjected to noise reduction processing; the plane image of the surface of the target junction box is fitted with the RANSAC algorithm, and the plane image with the background points removed is fitted. The RANSAC plane fitting algorithm introduces the normal vector consistency constraint, which can improve the robustness of the plane segmentation; the contour information of the target junction box is extracted from the plane image using the three-dimensional Canny operator. The FPFH (Fast Point Feature Histograms) descriptor is used for feature coarse registration, and the improved ICP algorithm is used for feature fine registration. The contour information of the target junction box is extracted based on the 3D point cloud data of the target junction box.

[0057] S15: performing data fusion on the position information and the contour information.

[0058] The Kalman filter algorithm is used to dynamically adjust the weights of the data obtained by different sensors according to the image clarity and 3D point cloud density, and the location information and contour information of the target junction box are aligned and fused in time and space.

[0059] Specifically, the multi-sensor data fusion algorithm includes the following improvements: a) Data synchronization mechanism based on timestamp: Hardware synchronization: Use external trigger signals to ensure the consistency of acquisition time; Software compensation: Interpolation algorithm based on timestamp to process data with different sampling rates. b) Improved Kalman filter algorithm: State space model: Includes position, attitude, velocity and angular velocity; Observation model: Considers the measurement characteristics and uncertainties of different sensors; Innovation: Introduces adaptive noise covariance matrix to improve the robustness of the filter. c) Adaptive weight adjustment strategy: Evaluation index: Consistency check based on innovation sequence; Weight update: Use exponential sliding average method to smooth weight changes; Abnormal handling: Design fault detection and isolation mechanism to deal with sensor failure.

[0060] like Figure 3 As shown in the figure, the fusion process of multi-sensor data includes: image data input: 2D image data from industrial cameras; point cloud data input: 3D point cloud data from structured light laser sensors. These two data first pass through the feature extraction module: extract key features from the raw data. Then pass through the data synchronization module: ensure that the data from different sensors are aligned in time. The synchronized data is input to the Kalman filter: this is the core data fusion algorithm used to comprehensively analyze multi-source data. Finally, the fusion result output is obtained: including the precise position and posture information of the junction box. The flowchart clearly shows how multi-sensor data can be organically combined to obtain more accurate and reliable positioning results.

[0061] S16: Convert the fused data into the same coordinate system to calculate the target position information of the target junction box.

[0062] The fused data in different coordinate systems are transformed into the same world coordinate system; and the target position information of the target junction box is calculated based on the data in the same world coordinate system.

[0063] Specifically, Figure 4 The various coordinate systems shown and the transformation relationships between them include: World coordinate system: the global reference coordinate system of the entire system. Camera coordinate system: the coordinate system associated with the industrial camera. Laser sensor coordinate system: the coordinate system associated with the structured light laser sensor. Robot coordinate system: the coordinate system associated with the operating robot. Junction box coordinate system: the local coordinate system centered on the junction box. The arrows in the figure indicate the transformation relationship between these coordinate systems. These transformations are determined through the calibration process and are essential for unifying the data from different sensors into the same reference system. This coordinate transformation relationship diagram emphasizes the importance of accurate spatial registration in multi-sensor systems and is the key to achieving high-precision positioning.

[0064] Optionally, the embodiment of the present invention may further add an output and verification phase: including result output and verification process: output positioning results, perform accuracy verification and error analysis. Continuous optimization phase: including real-time processing and optimization process: parallel processing of multiple data streams, real-time parameter adjustment; anomaly detection and processing process: monitoring data validity, detecting and processing anomalies. Static accuracy test: using a high-precision three-dimensional coordinate measuring machine as the gold standard, measure 100 fixed-position junction boxes. Dynamic tracking test: test the system's ability to locate moving junction boxes on a simulated production line. Environmental adaptability test: simulate different lighting, temperature, vibration and other conditions to verify the stability of the system. Long-term reliability test: run the system continuously for 72 hours to monitor the drift of positioning accuracy.

[0065] Through the above implementation, the embodiment of the present invention achieves high-precision, high-speed, and lossless positioning of the photovoltaic module junction box, with a positioning accuracy of up to ±0.2mm and a processing speed of <100ms / piece, providing reliable technical support for the automated production of photovoltaic modules.

[0066] The beneficial effects of the embodiments of the present invention include:

[0067] 1. Positioning accuracy: Through multi-sensor fusion, a three-dimensional positioning accuracy of ±0.1mm is achieved, which is 50% higher than the existing technology.

[0068] 2. Processing speed: Single positioning time <100ms, meeting the needs of high-speed production lines and improving efficiency by more than 30%.

[0069] 3. Environmental adaptability: When the light changes by ±50% and the temperature changes by ±20℃, the performance attenuation is less than 5%.

[0070] 4. Non-destructive testing: Completely non-contact measurement avoids potential damage caused by mechanical contact.

[0071] 5. Comprehensive information acquisition: The precise position, posture and geometric dimensions of the junction box are obtained simultaneously to support subsequent intelligent assembly.

[0072] 6. Flexible production: supports fast switching of various types of junction boxes, with changeover time less than 10 minutes.

[0073] 7. Self-learning capability: Continuously optimize algorithm parameters through online learning, and the long-term accuracy drift is <0.05mm / year.

[0074] 8. High integration: The hardware system is miniaturized and easy to integrate into the existing production line. The installation space is <0.5m 3 .

[0075] The method for positioning the junction box of a photovoltaic module provided in an embodiment of the present invention collects image data of a target photovoltaic module through an industrial camera that has completed internal and external parameter calibration; collects 3D point cloud data of a target junction box on a target photovoltaic module through a structured light laser sensor that has completed internal and external parameter calibration; annotates the position information of the target junction box in the image data using a pre-trained deep learning positioning model; extracts contour information of the target junction box based on the 3D point cloud data of the target junction box; performs data fusion on the position information and contour information; and converts the fused data into the same coordinate system to calculate the target position information of the target junction box. Compared with the existing positioning methods of the junction box of photovoltaic modules, it is difficult to simultaneously meet the requirements of high precision, high speed and non-destructive measurement. This method combines the advantages of multiple sensors to achieve high-precision, high-speed and non-destructive positioning of the junction box of photovoltaic modules, providing reliable technical support for the automated production of photovoltaic modules.

[0076] Figure 5 A schematic structural diagram of a photovoltaic module junction box positioning system provided by an embodiment of the present invention specifically includes:

[0077] The multi-sensor integrated module 501 is used to collect image data of the target photovoltaic module through an industrial camera with internal and external parameter calibration. For detailed description, please refer to the relevant description corresponding to the above method embodiment, which will not be repeated here.

[0078] The multi-sensor integration module 501 is also used to collect 3D point cloud data of the target junction box on the target photovoltaic module through the structured light laser sensor with completed internal and external parameter calibration. For detailed description, please refer to the relevant description corresponding to the above method embodiment, which will not be repeated here.

[0079] The image processing module 502 is used to annotate the location information of the target junction box in the image data using a pre-trained deep learning positioning model. For detailed description, please refer to the relevant description corresponding to the above method embodiment, which will not be repeated here.

[0080] The data feature extraction module 503 is used to extract the contour information of the target junction box based on the 3D point cloud data of the target junction box. For detailed description, please refer to the relevant description corresponding to the above method embodiment, which will not be repeated here.

[0081] The data fusion module 504 is used to perform data fusion on the position information and the contour information. For detailed description, please refer to the relevant description corresponding to the above method embodiment, which will not be repeated here.

[0082] The coordinate transformation module 505 is used to transform the fused data into the same coordinate system to calculate the target position information of the target junction box. For detailed description, please refer to the relevant description corresponding to the above method embodiment, which will not be repeated here.

[0083] The photovoltaic module junction box positioning system provided in the embodiment of the present invention is used to execute the photovoltaic module junction box positioning method provided in the above embodiment. Its implementation method and principle are the same. For details, please refer to the relevant description of the above method embodiment and will not be repeated here.

[0084] Figure 6 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention, Figure 6 The electronic device 600 shown includes: at least one processor 601, a memory 602, at least one network interface 604 and other user interfaces 603. The various components in the electronic device 600 are coupled together via a bus system 605. It is understood that the bus system 605 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 605 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the bus system 605 is not shown in FIG. Figure 6 Various buses are labeled as bus system 605 .

[0085] The user interface 603 may include a display, a keyboard, or a pointing device (eg, a mouse, a trackball, a touch pad, or a touch screen).

[0086] It can be understood that the memory 602 in the embodiment of the present invention can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM bus random access memory (DRRAM). The memory 602 described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0087] In some implementations, the memory 602 stores the following elements, executable units or data structures, or a subset thereof, or an extended set thereof: an operating system 6021 and an application program 6022 .

[0088] The operating system 6021 includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., which are used to implement various basic services and process hardware-based tasks. The application 6022 includes various application programs, such as a media player (Media Player), a browser (Browser), etc., which are used to implement various application services. The program for implementing the method of the embodiment of the present invention can be included in the application 6022.

[0089] In the embodiment of the present invention, by calling the program or instruction stored in the memory 602, specifically, the program or instruction stored in the application 6022, the processor 601 is used to execute the method steps provided by each method embodiment, for example, including:

[0090] An industrial camera that has been calibrated with internal and external parameters collects image data of a target photovoltaic module; a structured light laser sensor that has been calibrated with internal and external parameters collects 3D point cloud data of a target junction box on a target photovoltaic module; a pre-trained deep learning positioning model is used to annotate the position information of the target junction box in the image data; contour information of the target junction box is extracted based on the 3D point cloud data of the target junction box; data fusion is performed on the position information and contour information; and the fused data is converted into the same coordinate system to calculate the target position information of the target junction box.

[0091] In one possible implementation, multiple photovoltaic module images under different environmental variables are collected and the positions of the junction boxes in the multiple photovoltaic module images are marked; data enhancement processing is performed on the multiple photovoltaic module images with the positions of the junction boxes marked to obtain a model training data set; and model training is performed on an improved YOLOv5 network based on the model training data set to obtain a deep learning positioning model.

[0092] In one possible implementation, the image data is input into the pre-trained deep learning positioning model to obtain a target junction box detection result output by the model, wherein the target junction box detection result includes the model, location information, posture, and geometric dimensions of the target junction box.

[0093] In a possible implementation, the 3D point cloud data is subjected to noise reduction processing; a RANSAC algorithm is used to fit a plane image of the surface of the target junction box with background points removed; and a three-dimensional Canny operator is used to extract contour information of the target junction box from the plane image.

[0094] In one possible implementation, a Kalman filter algorithm is used to dynamically adjust the weights of data acquired by different sensors according to image clarity and 3D point cloud density, and perform spatiotemporal alignment and data fusion on the location information and contour information of the target junction box.

[0095] In a possible implementation, coordinate transformation is performed on the fused data in different coordinate systems into the same world coordinate system; and the target position information of the target junction box is calculated based on the data in the same world coordinate system.

[0096] In a possible implementation, the calculated target position information of the target junction box is tested and verified based on preset test conditions and test standards to obtain a test verification result; and the algorithm parameters are optimized based on the test verification result.

[0097] The method disclosed in the above embodiment of the present invention can be applied to the processor 601, or implemented by the processor 601. The processor 601 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit in the processor 601 or the instruction in the form of software. The above processor 601 can be a general processor, a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present invention can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiment of the present invention can be directly embodied as a hardware decoding processor to execute, or the hardware and software units in the decoding processor can be executed. The software unit can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 602, and the processor 601 reads the information in the memory 602 and completes the steps of the above method in combination with its hardware.

[0098] It is understood that the embodiments described herein may be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit may be implemented in one or more application specific integrated circuits (ASIC), digital signal processors (DSP), digital signal processing devices (DSPDevice, DSPD), programmable logic devices (PLD), field programmable gate arrays (FPGA), general purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described in the present application, or a combination thereof.

[0099] For software implementation, the technology described herein can be implemented by a unit that performs the functions described herein. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.

[0100] The electronic device provided in this embodiment may be Figure 6 The electronic device shown in FIG. 1 may perform the following steps: Figure 1 All steps of the photovoltaic module junction box positioning method based on multi-sensor fusion are implemented Figure 1 The technical effect of the positioning method of the photovoltaic module junction box based on multi-sensor fusion is shown in FIG. Figure 1 For the sake of brevity, the relevant description is not repeated here.

[0101] The embodiment of the present invention also provides a storage medium (computer-readable storage medium). The storage medium here stores one or more programs. The storage medium may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a read-only memory, a flash memory, a hard disk or a solid-state drive; the memory may also include a combination of the above-mentioned types of memory.

[0102] When one or more programs in the storage medium can be executed by one or more processors, the above-mentioned photovoltaic module junction box positioning method based on multi-sensor fusion executed on the electronic device side is implemented.

[0103] The processor is used to execute the photovoltaic assembly junction box positioning program based on multi-sensor fusion stored in the memory to implement the following steps of the photovoltaic assembly junction box positioning method based on multi-sensor fusion performed on the electronic device side:

[0104] An industrial camera that has been calibrated with internal and external parameters collects image data of a target photovoltaic module; a structured light laser sensor that has been calibrated with internal and external parameters collects 3D point cloud data of a target junction box on a target photovoltaic module; a pre-trained deep learning positioning model is used to annotate the position information of the target junction box in the image data; contour information of the target junction box is extracted based on the 3D point cloud data of the target junction box; data fusion is performed on the position information and contour information; and the fused data is converted into the same coordinate system to calculate the target position information of the target junction box.

[0105] In one possible implementation, multiple photovoltaic module images under different environmental variables are collected and the positions of the junction boxes in the multiple photovoltaic module images are marked; data enhancement processing is performed on the multiple photovoltaic module images with the positions of the junction boxes marked to obtain a model training data set; and model training is performed on an improved YOLOv5 network based on the model training data set to obtain a deep learning positioning model.

[0106] In one possible implementation, the image data is input into the pre-trained deep learning positioning model to obtain a target junction box detection result output by the model, wherein the target junction box detection result includes the model, location information, posture, and geometric dimensions of the target junction box.

[0107] In a possible implementation, the 3D point cloud data is subjected to noise reduction processing; a RANSAC algorithm is used to fit a plane image of the surface of the target junction box with background points removed; and a three-dimensional Canny operator is used to extract contour information of the target junction box from the plane image.

[0108] In one possible implementation, a Kalman filter algorithm is used to dynamically adjust the weights of data acquired by different sensors according to image clarity and 3D point cloud density, and perform spatiotemporal alignment and data fusion on the location information and contour information of the target junction box.

[0109] In a possible implementation, coordinate transformation is performed on the fused data in different coordinate systems into the same world coordinate system; and the target position information of the target junction box is calculated based on the data in the same world coordinate system.

[0110] In a possible implementation, the calculated target position information of the target junction box is tested and verified based on preset test conditions and test standards to obtain a test verification result; and the algorithm parameters are optimized based on the test verification result.

[0111] The professionals should further realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to the function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0112] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0113] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for positioning a junction box of a photovoltaic module, characterized in that: include: The industrial camera, which has been calibrated with internal and external parameters, collects image data of the target photovoltaic module; The structured light laser sensor, which has been calibrated with internal and external parameters, collects 3D point cloud data of the target junction box on the target photovoltaic module; Annotating location information of a target junction box in the image data using a pre-trained deep learning positioning model; Extracting contour information of the target junction box based on the 3D point cloud data of the target junction box; Performing data fusion on the position information and the contour information; The fused data is converted into the same coordinate system to calculate the target position information of the target junction box.

2. The method according to claim 1, characterized in that Before the pre-trained deep learning positioning model is used to mark the location information of the target junction box in the image data, the method includes: Collecting multiple photovoltaic module images under different environmental variables and marking the positions of junction boxes in the multiple photovoltaic module images; Perform data enhancement processing on multiple photovoltaic module images with the locations of junction boxes marked to obtain a model training data set; The improved YOLOv5 network is trained based on the model training data set to obtain a deep learning positioning model.

3. The method according to claim 2, characterized in that The method of using a pre-trained deep learning positioning model to label the location information of the target junction box in the image data includes: The image data is input into the pre-trained deep learning positioning model to obtain a target junction box detection result output by the model, wherein the target junction box detection result includes the model, location information, posture and geometric dimensions of the target junction box.

4. The method according to claim 1, characterized in that: The extracting contour information of the target junction box based on the 3D point cloud data of the target junction box includes: Performing noise reduction processing on the 3D point cloud data; The RANSAC algorithm is used to fit the plane image of the target junction box surface without removing background points; A three-dimensional Canny operator is used to extract contour information of the target junction box from the plane image.

5. The method according to any one of claims 1 to 4, characterized in that: The data fusion of the position information and the contour information comprises: The Kalman filter algorithm is used to dynamically adjust the weights of the data acquired by different sensors according to the image clarity and the 3D point cloud density, and the position information and contour information of the target junction box are aligned in time and space and fused.

6. The method according to claim 5, characterized in that The step of converting the fused data into the same coordinate system to calculate the target position information of the target junction box includes: Perform coordinate transformation on the fused data in different coordinate systems and transform them into the same world coordinate system; The target position information of the target junction box is calculated based on the data in the same world coordinate system.

7. The method according to claim 6, characterized in that The method further comprises: Based on preset test conditions and test standards, the calculated target position information of the target junction box is tested and verified to obtain a test verification result; The algorithm parameters are optimized based on the test verification results.

8. A positioning system for a junction box of a photovoltaic module, characterized in that: include: Multi-sensor integrated module, used to collect image data of target photovoltaic modules through industrial cameras calibrated with internal and external parameters; The multi-sensor integrated module is also used to collect 3D point cloud data of the target junction box on the target photovoltaic module through the structured light laser sensor that has completed the internal and external parameter calibration; An image processing module, used to annotate the location information of the target junction box in the image data using a pre-trained deep learning positioning model; A data feature extraction module, used to extract contour information of the target junction box based on the 3D point cloud data of the target junction box; A data fusion module, used for fusing the position information and the contour information; The coordinate transformation module is used to transform the fused data into the same coordinate system to calculate the target position information of the target junction box.

9. An electronic device, characterized in that: include: A processor and a memory, wherein the processor is used to execute a positioning program for a junction box of a photovoltaic assembly stored in the memory to implement the positioning method for a junction box of a photovoltaic assembly according to any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the method for positioning a junction box of a photovoltaic assembly according to any one of claims 1 to 7.

Images