Identification grabbing method for photovoltaic module, medium, equipment and system

Abstract

The invention relates to an identification grabbing method, medium, equipment and system for a photovoltaic module, and the method comprises the steps: calling a trained first neural network detection model to carry out the identification of a color image collected by a depth camera, judging whether the color image comprises a photovoltaic module or not, screening out a photovoltaic module region image if the color image comprises the photovoltaic module, and carrying out the recognition grabbing of the photovoltaic module; inputting the photovoltaic module area image into a second neural network detection model; and calling the trained second neural network detection model to perform reasoning on the photovoltaic module area image, if it is judged that the photovoltaic module area image does not have defects, performing spatial transformation based on the depth image to generate a first control command, and controlling a clamping mechanism to grab the current photovoltaic module according to the first control command. According to the scheme, the grabbing process of the photovoltaic module can be more intelligent, and the installation efficiency of the photovoltaic module is improved.

Description

Technical Field

[0001] The present invention relates to the field of photovoltaic components, and in particular to an identification and grasping method, medium, equipment and system for photovoltaic components. Background Art

[0002] With the continuous development of the photovoltaic power generation industry, the construction scale of photovoltaic power stations is constantly increasing and the number of construction is also increasing. Good photovoltaic power station construction efficiency can better contribute to this development trend. The laying and installation of photovoltaic modules is a very important part of the construction of photovoltaic power stations. One of its contents includes placing good photovoltaic modules in batches on fixed mounting brackets for fixing. At present, this process is mainly achieved through manual work. Due to factors such as size and weight, the labor intensity of laying and installation of photovoltaic modules is high and the efficiency is low. On the one hand, it will face high labor costs, and on the other hand, it will extend the construction period of photovoltaic power stations. Summary of the invention

[0003] In view of the above problems, the present invention provides a method, medium, device and system for identifying and grasping photovoltaic components, which solves the problems of low efficiency and high labor costs in the installation process of existing photovoltaic components.

[0004] To achieve the above objectives, in a first aspect, the present application provides a method for identifying and grasping a photovoltaic component, the method comprising the following steps:

[0005] Control the depth camera to collect color images and depth images;

[0006] Calling the trained first neural network detection model to identify the color image, determine whether the color image contains photovoltaic components, and if so, filter out the photovoltaic component area image, and input the photovoltaic component area image into the second neural network detection model;

[0007] Calling the trained second neural network detection model to perform reasoning on the photovoltaic module area image, if it is determined that the photovoltaic module area image does not have defects, calculating the first center point position corresponding to the current photovoltaic module according to the pixel coordinates and pixel size of the photovoltaic module in the reasoning result;

[0008] Performing spatial transformation based on the depth image, obtaining first spatial coordinates of the current photovoltaic component according to the first center point position corresponding to the photovoltaic component, and generating a first control command based on the first spatial coordinates;

[0009] The clamping mechanism is controlled to grab the current photovoltaic component according to the first control command, and the photovoltaic component is grabbed to a first preset position.

[0010] Furthermore, the method further comprises:

[0011] If it is determined that the photovoltaic component area image has defects, the depth camera is controlled to change acquisition parameters according to the location information of the defect, and the color image and the depth image of the current photovoltaic component location are re-acquired using the new acquisition parameters, and the newly acquired color image is input into the second neural network detection model, so that the second neural network detection model again determines whether there are defects in the photovoltaic component area image in the newly acquired color image;

[0012] The acquisition parameters include any one or more of acquisition angle, exposure parameter, focus parameter, and acquisition height.

[0013] Furthermore, the method further comprises:

[0014] If the number of defects in the re-collected photovoltaic component area image reaches a preset number, an edge area of ​​the current photovoltaic component is constructed using an edge detection algorithm based on the depth images collected multiple times, and a second center position of the current photovoltaic component is determined based on the edge area, and a second spatial coordinate of the current photovoltaic component is obtained based on the second center point position, and a second control command is generated based on the second spatial coordinate;

[0015] The clamping mechanism is controlled to grab the current photovoltaic component according to the second control command, and the photovoltaic component is grabbed to a second preset position.

[0016] Furthermore, the second control command includes clamping control information, and controlling the clamping mechanism to grab the current photovoltaic component according to the second control command includes:

[0017] Adjusting the clamping parameters of the clamping mechanism according to the clamping control information and then grabbing the current photovoltaic component;

[0018] The clamping control information is obtained according to the following method:

[0019] The area of ​​the current photovoltaic assembly is determined according to the edge area of ​​the current photovoltaic assembly, the clamping angle and clamping position of the clamping mechanism are determined based on the area and edge area of ​​the photovoltaic assembly, and the clamping control information is generated.

[0020] Furthermore, the first neural network detection model is trained according to the following method:

[0021] Collecting a plurality of photovoltaic module sample images to obtain a first sample data set;

[0022] Using a data enhancement algorithm to expand the sample images in the first sample data set to obtain a second sample data set;

[0023] Inputting the second sample data set into the first neural network detection model to be trained for learning and training, and obtaining the trained first neural network detection model after multiple iterations and updates;

[0024] The second neural network detection model is trained according to the following method:

[0025] Collecting multiple defect images of photovoltaic modules to obtain a third sample data set;

[0026] Using a data enhancement algorithm to expand the sample defect images in the third sample data set to obtain a fourth sample data set;

[0027] The fourth sample data set is input into the second neural network detection model to be trained for learning and training, and after multiple iterative updates, a trained second neural network detection model is obtained.

[0028] Furthermore, the method further comprises:

[0029] If the second neural network detection model determines that there are defects in the photovoltaic component area image, the current defect type is recorded and a prompt message is sent to a preset associated terminal, where the prompt message includes the identification information, location information and defect type of the current photovoltaic component.

[0030] In a second aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the method for identifying and grasping a photovoltaic component as described in the first aspect of the present application is implemented.

[0031] In a third aspect, the present application provides an electronic device having a computer program stored thereon, comprising a processor and a storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by the processor, the method for identifying and grasping photovoltaic components as described in the first aspect of the present application is implemented.

[0032] In a fourth aspect, the present application provides an identification and grasping system for photovoltaic components, comprising:

[0033] An electronic device, which is the electronic device as described in the third aspect of the present application;

[0034] A depth camera, electrically connected to the processor of the electronic device, for collecting color images and depth images;

[0035] The clamping mechanism is communicatively connected to the processor of the electronic device and is used to clamp the photovoltaic assembly according to the control command generated by the processor.

[0036] Furthermore, the clamping mechanism comprises:

[0037] Clamping arm;

[0038] A clamping plate, disposed at one end of the clamping arm;

[0039] A plurality of vacuum suction cups are distributed on the clamping plate in a first preset manner, and ends of the plurality of vacuum suction cups form a clamping portion, and the clamping portion is used to clamp the photovoltaic module;

[0040] The depth camera is arranged on the clamping plate, and the clamping plate is provided with a light source in the circumferential direction surrounding the depth camera.

[0041] Different from the prior art, the above technical solution provides a method, medium, device and system for identifying and grasping photovoltaic components, which includes: controlling the depth camera to collect color images and depth images; calling the trained first neural network detection model to identify the color image, and judging whether the color image contains photovoltaic components, if so, filtering out the photovoltaic component area image, and inputting the photovoltaic component area image into the second neural network detection model; calling the trained second neural network detection model to infer the photovoltaic component area image, if it is judged that there is no defect in the photovoltaic component area image, then calculating the first center point position corresponding to the current photovoltaic component according to the pixel coordinates and pixel size of the photovoltaic component in the inference result; generating a first control command based on the spatial transformation of the depth image, and controlling the clamping mechanism to grasp the current photovoltaic component according to the first control command. The above solution automates and intelligentizes the grasping and placing process of photovoltaic components through visual detection technology, effectively improving the installation efficiency of photovoltaic components.

[0042] The above-mentioned records related to the content of the invention are only an overview of the technical solution of the present invention. In order to enable ordinary technicians in the field to more clearly understand the technical solution of the present invention, and then implement it according to the text of the specification and the contents recorded in the drawings, and in order to make the above-mentioned purpose and other purposes, features and advantages of the present invention easier to understand, the following is a description in combination with the specific implementation mode of the present invention and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The drawings are only used to illustrate the principles, implementations, applications, features and effects of the specific embodiments of the present invention and other related contents, and shall not be considered as limiting the present invention.

[0044] In the drawings of the specification:

[0045] Figure 1 A first flow chart of a method for identifying and grasping a photovoltaic component according to a specific implementation manner;

[0046] Figure 2 A second flow chart of a method for identifying and grasping a photovoltaic component according to a specific implementation manner;

[0047] Figure 3 A third flow chart of a method for identifying and grasping a photovoltaic component according to a specific implementation manner;

[0048] Figure 4 is a flowchart of the first neural network detection model training process described in the specific implementation method;

[0049] Figure 5 A flowchart of the second neural network detection model training process described in the specific implementation method;

[0050] Figure 6 is a schematic diagram of an electronic device according to a specific implementation mode;

[0051] The reference numerals in the above drawings are described as follows:

[0052] 10. Electronic equipment;

[0053] 101. Processor;

[0054] 102. Storage medium. DETAILED DESCRIPTION

[0055] In order to explain in detail the possible application scenarios, technical principles, specific schemes that can be implemented, and the purposes and effects of the present invention, the following is a detailed description in combination with the specific embodiments listed and the accompanying drawings. The embodiments described herein are only used to more clearly illustrate the technical solutions of the present invention, and are therefore only used as examples, and cannot be used to limit the scope of protection of the present invention.

[0056] Reference to "embodiment" herein means that the specific features, structures or characteristics described in conjunction with the embodiment may be included in at least one embodiment of the present invention. The word "embodiment" appearing in various places in the specification does not necessarily refer to the same embodiment, nor does it particularly limit its independence or association with other embodiments. In principle, in the present invention, as long as there is no technical contradiction or conflict, the various technical features mentioned in each embodiment can be combined in any way to form a corresponding implementable technical solution.

[0057] Unless otherwise defined, the technical terms used herein have the same meanings as those generally understood by those skilled in the art to which the present invention belongs; the use of relevant terms herein is only for describing specific embodiments and is not intended to limit the present invention.

[0058] In the description of the present invention, the term "and / or" is an expression used to describe the logical relationship between objects, indicating that three relationships may exist, for example, A and / or B, which means: A exists, B exists, and A and B exist at the same time. In addition, the character " / " in this article generally indicates that the objects before and after are in a logical relationship of "or".

[0059] In the present invention, terms such as "first" and "second" are merely used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship of quantity, priority or sequence between these entities or operations.

[0060] Without further restrictions, in the present invention, the words "include", "comprises", "has" or other similar expressions used in sentences are intended to cover non-exclusive inclusion. These expressions do not exclude the presence of additional elements in the process, method or product including the elements, so that the process, method or product including a series of elements may include not only those limited elements, but also other elements not explicitly listed, or also include elements inherent to such process, method or product.

[0061] In the present invention, expressions such as "greater than", "less than", "exceed" and the like are understood to exclude the number itself; expressions such as "above", "below", "within" and the like are understood to include the number itself. In addition, in the description of the embodiments of the present invention, "multiple" means more than two (including two), and similar expressions related to "multiple" are also understood in this way, such as "multiple groups", "multiple times", etc., unless otherwise clearly and specifically limited.

[0062] In the description of the embodiments of the present invention, the space-related expressions used, such as "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "vertical", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential", etc., indicate the orientation or position relationship based on the orientation or position relationship shown in the specific embodiments or drawings, and are only for the convenience of describing the specific embodiments of the present invention or facilitating the reader's understanding, and do not indicate or imply that the referred device or component must have a specific position, a specific orientation, or be constructed or operated in a specific orientation, and therefore cannot be understood as a limitation on the embodiments of the present invention.

[0063] Unless otherwise expressly specified or limited, in the description of the embodiments of the present invention, the terms such as "install", "connect", "connect", "fix", "set" and the like used should be understood in a broad sense. For example, the "connection" can be a fixed connection, a detachable connection, or an integrated setting; it can be a mechanical connection, an electrical connection, or a communication connection; it can be a direct connection, or an indirect connection through an intermediate medium; it can be the internal connection of two elements or the interaction relationship between two elements. For those skilled in the art of the technical field to which the present invention belongs, the specific meanings of the above terms in the embodiments of the present invention can be understood according to the specific circumstances.

[0064] In the first aspect, Figure 1 As shown, the present application provides a method for identifying and grasping a photovoltaic component, the method comprising the following steps:

[0065] Step S101: Control the depth camera to collect color images and depth images;

[0066] Step S102: calling the trained first neural network detection model to identify the color image, determine whether the color image contains photovoltaic modules, and if so, filter out the photovoltaic module area image, and input the photovoltaic module area image into the second neural network detection model;

[0067] Step S103: calling the trained second neural network detection model to infer the photovoltaic module area image, and if it is determined that the photovoltaic module area image does not have defects, calculating the first center point position corresponding to the current photovoltaic module according to the pixel coordinates and pixel size of the photovoltaic module in the inference result;

[0068] Step S104: performing spatial transformation based on the depth image, obtaining a first spatial coordinate of the current photovoltaic component according to a first center point position corresponding to the photovoltaic component, and generating a first control command based on the first spatial coordinate;

[0069] Step S105: controlling the clamping mechanism to grab the current photovoltaic component according to the first control command, and grabbing the photovoltaic component to a first preset position.

[0070] In step S101, a depth camera is a device capable of capturing distance information of objects in a scene, and can generally provide more information about a three-dimensional spatial structure compared to a traditional two-dimensional camera.

[0071] Color images mainly contain the color information of objects in the scene. They are composed of three channels: red, green, and blue (RGB), each channel representing the intensity of the corresponding color. Such images can be used to identify objects, view scene details, etc. Color images are usually represented in the form of a two-dimensional array, and each pixel contains three values ​​(RGB) representing the color of the pixel.

[0072] The depth image mainly contains the distance information from each pixel in the scene to the camera. The value of each pixel usually represents the distance (depth) to the closest surface of the object. This is a single-channel (grayscale) image. The depth image is usually represented in the form of a two-dimensional array, and each pixel value represents the depth of the point, generally expressed in meters or other units. The range of depth values ​​is usually a continuous value. The smaller the value, the closer it is to the camera, and the larger the value, the farther it is from the camera.

[0073] In steps S102 and S103, the first neural network model and the second neural network model can be a convolutional neural network (CNN) or a recurrent neural network (RNN) and its variants (such as LSTM, GRU). The first neural network model is used to identify whether the collected color image contains a photovoltaic module. On this basis, the second neural network model is used to identify whether the photovoltaic module in the color image has defects. In this embodiment, the defects of the photovoltaic module include cell defects, cracks, hot spot effects in partial or entire areas, etc.

[0074] In step S104, if there is no defect in the photovoltaic component, the first center point position corresponding to the photovoltaic component can be directly identified according to the second neural network model, and then the three-dimensional spatial coordinates of the current photovoltaic component in the actual scene are obtained based on the spatial transformation relationship of the depth image, and the three-dimensional spatial coordinates are converted into a first control command.

[0075] In step S105, the first preset position refers to the position where the qualified photovoltaic module is placed. After the photovoltaic module is clamped to the first preset position, the next step of the installation process can be carried out.

[0076] The above scheme performs intelligent identification of photovoltaic modules based on the visual system. After determining the three-dimensional spatial coordinates of the photovoltaic modules, the photovoltaic modules are automatically grasped and placed through the clamping mechanism, which greatly improves the installation efficiency of the photovoltaic modules. At the same time, it also reduces the labor intensity of the photovoltaic module installation process, thereby reducing human resource costs.

[0077] In some embodiments, Figure 2 As shown, the method also includes:

[0078] First, enter step S201: determine that the photovoltaic module area image has defects, and then control the depth camera to change the acquisition parameters according to the location information of the defects;

[0079] Then, step S202 is entered: the color image and the depth image of the current photovoltaic component location are re-collected using new acquisition parameters, and the newly collected color image is input into the second neural network detection model, so that the second neural network detection model again determines whether there is a defect in the photovoltaic component area image in the newly collected color image;

[0080] In step S201, the acquisition parameters include any one or more of acquisition angle, exposure parameter, focus parameter, and acquisition height.

[0081] In short, if the second neural network model detects that there are defects in the image of the photovoltaic component area, the factor causing this situation may be damage or failure of the photovoltaic component itself, or it may be caused by inappropriate acquisition parameters of the depth camera, that is, the shooting angle, height, exposure parameters, etc. of the depth camera are not adjusted to the appropriate position, resulting in missing color images and depth images, which in turn affects the subsequent judgment of the second neural network model. When this happens, the acquisition parameters of the depth camera can be adjusted and the photo can be re-taken, and then the second neural network detection model can again judge whether there are defects in the photovoltaic component area image in the newly acquired color image, which can effectively reduce the erroneous judgment caused by improper acquisition parameters and improve accuracy.

[0082] In some embodiments, Figure 3 As shown, the method also includes:

[0083] Step S301: if the number of defects in the re-collected photovoltaic component area image reaches a preset number, an edge area of ​​the current photovoltaic component is constructed using an edge detection algorithm based on the depth images collected multiple times, and a second center position of the current photovoltaic component is determined based on the edge area, a second spatial coordinate of the current photovoltaic component is obtained based on the second center point position, and a second control command is generated based on the second spatial coordinate;

[0084] Step S302: controlling the clamping mechanism to grasp the current photovoltaic component according to the second control command, and grasping the photovoltaic component to a second preset position.

[0085] In step S301, if the number of times the re-captured photovoltaic component area image has defects reaches a preset number, it means that although the image has been taken repeatedly, the captured photovoltaic component image still has defects after identification. In this case, the corresponding second center position can also be determined based on the current photovoltaic component area image, and then the three-dimensional spatial coordinates of the current photovoltaic component can be obtained to facilitate subsequent clamping.

[0086] In step S302, the second preset position refers to a position for storing defective photovoltaic modules. After all photovoltaic modules have been identified, the installer can also manually screen the photovoltaic modules at the second preset position again to facilitate subsequent maintenance.

[0087] In some embodiments, the second control command includes clamping control information, and controlling the clamping mechanism to grab the current photovoltaic component according to the second control command includes: adjusting the clamping parameters of the clamping mechanism according to the clamping control information and then grabbing the current photovoltaic component;

[0088] The clamping control information is obtained according to the following method:

[0089] The area of ​​the current photovoltaic assembly is determined according to the edge area of ​​the current photovoltaic assembly, the clamping angle and clamping position of the clamping mechanism are determined based on the area and edge area of ​​the photovoltaic assembly, and the clamping control information is generated.

[0090] In short, if there are defects in the photovoltaic module, for example, the clamping mechanism cannot clamp the photovoltaic module at the original angle and spacing when clamping it, but needs to simulate the actual edge of the current photovoltaic module based on the regional images of the photovoltaic module taken multiple times, and then analyze the edge coordinates and center coordinates corresponding to the current photovoltaic module through edge detection and coordinate conversion, and then select appropriate clamping control information based on the analyzed center coordinates and edge coordinates. In this way, when the degree of defect of the photovoltaic module is different, the clamping mechanism can adaptively adjust the clamping parameters to clamp the defective photovoltaic module, making the operation more intelligent.

[0091] In some embodiments, Figure 4 As shown, the first neural network detection model is trained according to the following method:

[0092] Step S401: collecting a plurality of photovoltaic module sample images to obtain a first sample data set;

[0093] Step S402: using a data enhancement algorithm to expand the sample images in the first sample data set to obtain a second sample data set;

[0094] Step S403: input the second sample data set into the first neural network detection model to be trained for learning and training, and after multiple iterations and updates, obtain a trained first neural network detection model.

[0095] In some embodiments, Figure 5 As shown, the second neural network detection model is trained according to the following method:

[0096] Step S501: collecting a plurality of defect images of photovoltaic modules to obtain a third sample data set;

[0097] Step S502: using a data enhancement algorithm to expand the sample defect images in the third sample data set to obtain a fourth sample data set;

[0098] Step S503: inputting the fourth sample data set into the second neural network detection model to be trained for learning and training, and obtaining a trained second neural network detection model after multiple iterations and updates.

[0099] In short, in the process of enhancing and expanding the image, we can obtain images similar to the photovoltaic module installation scene as much as possible, that is, the background image of the photovoltaic module is similar to the actual installation scene. When processing the image, we can prevent the distortion of the photovoltaic module part in the image as much as possible, which can improve the accuracy of subsequent model recognition. Then, based on the constructed photovoltaic module image dataset and photovoltaic module defect image dataset, we can use the Yolov8 algorithm to learn and train them, so as to obtain the photovoltaic module detection model and photovoltaic module defect detection model.

[0100] In some embodiments, the method further comprises:

[0101] If the second neural network detection model determines that there are defects in the photovoltaic component area image, the current defect type is recorded and a prompt message is sent to a preset associated terminal, where the prompt message includes the identification information, location information and defect type of the current photovoltaic component.

[0102] Preferably, the identification information of the photovoltaic component can be a QR code or a barcode, so that the depth camera can also transmit the captured image of the defective photovoltaic component to the background for identification, thereby determining the identification information of the current photovoltaic component and reporting the second preset position. The preset associated terminal can be a terminal device of the installer or relevant management personnel, or it can be a cloud server. After receiving the prompt information, the preset associated terminal can clearly know the defect type and proportion corresponding to each defective photovoltaic component, and then analyze the cause of the defect and take remedial measures in time. The prompt information can be a sound prompt information, a light prompt information, a vibration prompt information, or text or graphic prompt information on the display interface of the display device.

[0103] In a second aspect, the present invention further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the method for identifying and grasping a photovoltaic component as described in the first aspect of the present invention is implemented.

[0104] The computer-readable storage medium may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories.

[0105] The non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a ferromagnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD ROM); the magnetic surface memory can be a disk memory or a tape memory.

[0106] The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), direct memory bus random access memory (DRRAM). The computer-readable storage medium described in the embodiments of the present invention is intended to include these and any other suitable types of memory.

[0107] like Figure 6 As shown, in a third aspect, the present invention provides an electronic device 10, comprising a processor 101 and a storage medium 102, wherein a computer program is stored on the storage medium, and when the computer program is executed by the processor, the method for identifying and grasping photovoltaic components as described in the first aspect of the present invention is implemented.

[0108] In some embodiments, the processor can be implemented by software, hardware, firmware or a combination thereof, and can use circuits, single or multiple application specific integrated circuits (Application Specific Integrated Circuit, ASIC), digital signal processors (Digital Signal Processor, DSP), digital signal processing devices (Digital Signal Processing Device, DSPD), programmable logic devices (Programmable Logic Device, PLD), field programmable gate arrays (Field Programmable Gate Array, FPGA), central processing units (Central Processing Unit, CPU), controllers, microcontrollers, microprocessors and at least one thereof, so that the processor can execute some or all of the steps or any combination of the steps in the method for identifying and grasping photovoltaic components described in the various embodiments of the present application.

[0109] In a fourth aspect, the present application also provides an identification and grasping system for photovoltaic components, comprising:

[0110] An electronic device, which is the electronic device as described in the third aspect of the present application;

[0111] A depth camera, electrically connected to the processor of the electronic device, for collecting color images and depth images;

[0112] The clamping mechanism is communicatively connected to the processor of the electronic device and is used to clamp the photovoltaic assembly according to the control command generated by the processor.

[0113] Furthermore, the clamping mechanism comprises:

[0114] Clamping arm;

[0115] A clamping plate, disposed at one end of the clamping arm;

[0116] A plurality of vacuum suction cups are distributed on the clamping plate in a first preset manner, and ends of the plurality of vacuum suction cups form a clamping portion, and the clamping portion is used to clamp the photovoltaic module;

[0117] The depth camera is disposed on the clamping plate, and the clamping plate is further provided with a light source in a circumferential direction surrounding the depth camera.

[0118] The above technical solution provides a method, medium, device and system for identifying and grasping photovoltaic components, the method comprising: controlling a depth camera to collect color images and depth images; calling a trained first neural network detection model to identify the color image, judging whether the color image contains photovoltaic components, if so, filtering out the photovoltaic component area image, and inputting the photovoltaic component area image into the second neural network detection model; calling the trained second neural network detection model to infer the photovoltaic component area image, if it is judged that there is no defect in the photovoltaic component area image, then calculating the first center point position corresponding to the current photovoltaic component according to the pixel coordinates and pixel size of the photovoltaic component in the inference result; generating a first control command based on the spatial transformation of the depth image, and controlling the clamping mechanism to grasp the current photovoltaic component according to the first control command. The above solution automates and intelligentizes the grasping and placing process of photovoltaic components through visual detection technology, effectively improving the installation efficiency of photovoltaic components.

[0119] Finally, it should be noted that although the above embodiments have been described in the specification and drawings of this application, this does not limit the scope of patent protection of this application. All technical solutions generated by replacing or modifying equivalent structures or equivalent processes based on the essential concept of this application using the contents recorded in the specification and drawings of this application, as well as directly or indirectly implementing the technical solutions of the above embodiments in other related technical fields, are included in the scope of patent protection of this application.

Claims

1. A method for identifying and grasping a photovoltaic module, characterized in that: The method comprises the following steps: Control the depth camera to collect color images and depth images; Calling the trained first neural network detection model to identify the color image, determine whether the color image contains photovoltaic components, and if so, filter out the photovoltaic component area image, and input the photovoltaic component area image into the second neural network detection model; Calling the trained second neural network detection model to perform reasoning on the photovoltaic module area image, if it is determined that the photovoltaic module area image does not have defects, calculating the first center point position corresponding to the current photovoltaic module according to the pixel coordinates and pixel size of the photovoltaic module in the reasoning result; Performing spatial transformation based on the depth image, obtaining first spatial coordinates of the current photovoltaic component according to the first center point position corresponding to the photovoltaic component, and generating a first control command based on the first spatial coordinates; The clamping mechanism is controlled to grab the current photovoltaic component according to the first control command, and the photovoltaic component is grabbed to a first preset position.

2. The method for identifying and grasping a photovoltaic module according to claim 1, characterized in that: The method further comprises: If it is determined that the photovoltaic component area image has defects, the depth camera is controlled to change acquisition parameters according to the location information of the defect, and the color image and the depth image of the current photovoltaic component location are re-acquired using the new acquisition parameters, and the newly acquired color image is input into the second neural network detection model, so that the second neural network detection model again determines whether there are defects in the photovoltaic component area image in the newly acquired color image; The acquisition parameters include any one or more of acquisition angle, exposure parameter, focus parameter, and acquisition height.

3. The method for identifying and grasping a photovoltaic module according to claim 2, characterized in that: The method further comprises: If the number of defects in the re-collected photovoltaic component area image reaches a preset number, an edge area of ​​the current photovoltaic component is constructed using an edge detection algorithm based on the depth images collected multiple times, and a second center position of the current photovoltaic component is determined based on the edge area, and a second spatial coordinate of the current photovoltaic component is obtained based on the second center point position, and a second control command is generated based on the second spatial coordinate; The clamping mechanism is controlled to grab the current photovoltaic component according to the second control command, and the photovoltaic component is grabbed to a second preset position.

4. The method for identifying and grasping a photovoltaic module according to claim 3, characterized in that: The second control command includes clamping control information, and controlling the clamping mechanism to grab the current photovoltaic component according to the second control command includes: Adjusting the clamping parameters of the clamping mechanism according to the clamping control information and then grabbing the current photovoltaic component; The clamping control information is obtained according to the following method: The area of ​​the current photovoltaic assembly is determined according to the edge area of ​​the current photovoltaic assembly, the clamping angle and clamping position of the clamping mechanism are determined based on the area and edge area of ​​the photovoltaic assembly, and the clamping control information is generated.

5. The method for identifying and grasping a photovoltaic module according to claim 1, characterized in that: The first neural network detection model is trained according to the following method: Collecting a plurality of photovoltaic module sample images to obtain a first sample data set; Using a data enhancement algorithm to expand the sample images in the first sample data set to obtain a second sample data set; Inputting the second sample data set into the first neural network detection model to be trained for learning and training, and obtaining the trained first neural network detection model after multiple iterations and updates; The second neural network detection model is trained according to the following method: Collecting multiple defect images of photovoltaic modules to obtain a third sample data set; Using a data enhancement algorithm to expand the sample defect images in the third sample data set to obtain a fourth sample data set; The fourth sample data set is input into the second neural network detection model to be trained for learning and training, and after multiple iterative updates, a trained second neural network detection model is obtained.

6. The method for identifying and grasping a photovoltaic module according to claim 1, characterized in that: The method further comprises: If the second neural network detection model determines that there are defects in the photovoltaic component area image, the current defect type is recorded and a prompt message is sent to a preset associated terminal, where the prompt message includes the identification information, location information and defect type of the current photovoltaic component.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method for identifying and grasping a photovoltaic component according to any one of claims 1 to 6 is implemented.

8. An electronic device having a computer program stored thereon, characterized in that: It comprises a processor and a storage medium, wherein a computer program is stored on the storage medium, and when the computer program is executed by the processor, the method for identifying and grasping a photovoltaic component according to any one of claims 1 to 6 is implemented.

9. A photovoltaic module identification and grasping system, characterized in that: include: The electronic device is the electronic device as claimed in claim 8; A depth camera, electrically connected to the processor of the electronic device, for collecting color images and depth images; The clamping mechanism is communicatively connected to the processor of the electronic device and is used to clamp the photovoltaic assembly according to the control command generated by the processor.

10. The photovoltaic module identification and grasping system according to claim 9, characterized in that: The clamping mechanism comprises: Clamping arm; A clamping plate, disposed at one end of the clamping arm; A plurality of vacuum suction cups are distributed on the clamping plate in a first preset manner, and ends of the plurality of vacuum suction cups form a clamping portion, and the clamping portion is used to clamp the photovoltaic module; The depth camera is disposed on the clamping plate, and the clamping plate is provided with a light source in a circumferential direction surrounding the depth camera.

Images