Method and device for constructing a solar panel surface property detection model

Abstract

The present application proposes a method and device for constructing a solar panel surface property detection model, including the following steps: constructing a solar panel surface property detection framework, the solar panel surface property detection framework is composed of a feature extraction module, a feature fusion module and a classification head; obtaining multiple solar panel images as training images, using the training images to train the solar panel surface property detection framework, and when the preset training conditions are met, saving the parameters of the solar panel surface property detection framework when the preset training conditions are met to obtain a constructed solar panel surface property detection model. This solution can accurately detect the property category of the solar cell panel by cross-dimensionally fusing the texture features, shape features and deep local features of different dimensions in the solar panel image.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular to a method and device for constructing a solar panel surface property detection model. Background Art

[0002] As a clean and renewable energy, solar energy has been widely used and developed around the world. As the core component that converts solar energy into electrical energy, the production process and quality control of solar cells play a decisive role in the performance and efficiency of the entire solar energy system. The solar cell screen is one of the key components in the solar cell manufacturing process. Its main function is to accurately deposit conductive materials onto the cell through a printing process to form electrodes and other functional structures, thereby achieving efficient collection and transmission of electrical energy.

[0003] In the manufacturing process of solar cell screens, the quality of their surface properties directly affects the final printing effect and battery performance. A high-quality solar cell screen is usually composed of a criss-cross steel mesh as the basic structure, and a waterproof resin layer is covered on the steel mesh. The resin layer will be carved into a specific pattern. During subsequent printing, the ink will pass through the hollow part of the resin layer to form the required pattern. Finally, the yellow PI film will be attached to the surface of the screen using hot pressing technology. However, in actual production, the screen may have a variety of surface property problems, such as damage to the steel mesh (such as steel wire breakage, deformation, etc.), mesh blockage, uneven or incomplete resin coating, etc. These problems will not only lead to a decrease in the accuracy of the printed electrode pattern and affect the electrical performance of the cell, but may also reduce the conversion efficiency and service life of the solar cell.

[0004] Traditional solar cell screen surface property inspection mainly relies on manual inspection, which has many disadvantages. First, manual inspection is inefficient and difficult to meet the needs of large-scale production, especially in modern solar cell production lines, where the inspection speed is far behind the production pace. Second, the accuracy of manual inspection is affected by the experience, fatigue and subjective judgment of the inspectors. Different inspectors may have large differences in their judgments on the same problem, making it difficult to ensure the consistency and reliability of the inspection results. In addition, manual inspection may not be able to accurately identify some subtle surface property problems, such as tiny mesh blockages or local unevenness of the resin coating, which may cause some defective screens to flow into the next production link, thereby affecting the quality of the final product.

[0005] With the rapid development of computer vision and artificial intelligence technology, the use of image analysis technology to automatically detect solar cell screens has become a highly promising alternative. By using high-resolution industrial cameras to capture the surface images of solar screens, combined with image processing and machine learning algorithms, automatic recognition and classification of screen surface properties can be achieved. This method is expected to overcome many of the shortcomings of traditional manual inspections, improve the efficiency, accuracy and consistency of inspections, and provide more reliable guarantees for the production quality control of solar cell screens.

[0006] However, the solar cell screen surface property detection technology based solely on image analysis faces some challenges in practical applications. On the one hand, due to the complex surface structure of the screen, its image feature extraction and analysis are relatively difficult, and it is necessary to accurately capture and process tiny features (such as mesh blockage), different textures (such as resin coating) and structures (such as wire mesh). On the other hand, in order to ensure the reliability of detection, a large amount of image data is required for training, but in actual production, the collected image data may have problems such as insufficient sample diversity and data imbalance, especially for some rare defective samples, the amount of data may be relatively small, which may affect the training effect and generalization ability of the model. Therefore, how to build an efficient, accurate and highly adaptable solar cell screen surface property detection model is an important issue that needs to be studied and solved in the current solar cell production field.

[0007] In summary, developing a solar cell screen surface property detection technology based on image analysis and conducting in-depth research on its key model construction and optimization technology have important practical significance for improving the production quality of solar cells, improving production efficiency and promoting the development of the solar energy industry. Summary of the invention

[0008] The embodiments of the present application provide a method and device for constructing a solar panel surface property detection model, which can accurately detect the property category of the solar cell panel by cross-dimensionally fusing texture features, shape features, and deep local features of different dimensions in the solar panel image.

[0009] In a first aspect, an embodiment of the present application provides a method for constructing a solar panel surface property detection model, the method comprising:

[0010] Constructing a solar panel surface property detection framework, the solar panel surface property detection framework is composed of a feature extraction module, a feature fusion module and a classification head;

[0011] Acquire multiple solar screen images as training images, and each training image is annotated with a property category, wherein the property category includes a mesh state, a resin coating state, a steel screen state, and a PI film lamination state;

[0012] In the feature extraction module, the texture features of each training image are extracted based on the gray level co-occurrence matrix, the shape features of each training image are extracted based on the edge detection algorithm, and the deep local features of each training image are extracted based on multiple convolutional layers;

[0013] In the feature fusion module, the texture features, shape features and deep local features of each training image are fused to obtain the fusion features of each training image;

[0014] The classification head calculates the predicted trait category of each training image based on the fusion features of each training image, and then calculates the trait category error according to the predicted trait category and the labeled trait category of each training image;

[0015] The parameters of the solar panel surface property detection framework are adjusted based on the category error, and the solar panel surface property detection framework after the parameter adjustment is used to continue training with multiple training images until preset training conditions are met, and the parameters of the solar panel surface property detection framework when the preset training conditions are met are saved to obtain a constructed solar panel surface property detection model.

[0016] In a second aspect, an embodiment of the present application provides a device for constructing a solar panel surface property detection model, comprising:

[0017] A construction module is used to construct a solar panel surface property detection framework, wherein the solar panel surface property detection framework is composed of a feature extraction module, a feature fusion module and a classification head;

[0018] An acquisition module, used for acquiring a plurality of solar screen images as training images, and each training image is annotated with a property category, wherein the property category includes a mesh state, a resin coating state, a steel screen state, and a PI film lamination state;

[0019] A feature extraction module extracts the texture features of each training image based on the gray-level co-occurrence matrix, extracts the shape features of each training image based on the edge detection algorithm, and extracts the deep local features of each training image based on multiple convolutional layers;

[0020] A feature fusion module is used to fuse the texture features, shape features and deep local features of each training image to obtain a fusion feature of each training image;

[0021] A classification module calculates the predicted trait category of each training image based on the fusion features of each training image, and then calculates the trait category error based on the predicted trait category and the labeled trait category of each training image;

[0022] A parameter adjustment module adjusts the parameters of the solar panel surface property detection framework based on the category error, continues training with the parameter-adjusted solar panel surface property detection framework and multiple training images until preset training conditions are met, and saves the parameters of the solar panel surface property detection framework when the preset training conditions are met to obtain a constructed solar panel surface property detection model.

[0023] In a third aspect, an embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute a method for constructing a solar panel surface property detection model.

[0024] In a fourth aspect, an embodiment of the present application provides a readable storage medium, wherein the readable storage medium stores a computer program, wherein the computer program includes a program code for controlling a process to execute a process, wherein the process includes a method for constructing a solar panel surface property detection model.

[0025] The main contributions and innovations of the present invention are as follows:

[0026] The feature extraction module of the embodiment of the present application integrates texture features, shape features and deep local features, and can comprehensively capture the subtle defects and complex characteristics of the screen. For example, the texture feature can sensitively perceive the microscopic changes of the resin coating and the mesh, the shape feature can accurately judge the shape of the wire mesh and the mesh, and the deep local feature can mine abstract and rich feature information through deep learning. The three complement each other to make the detection more accurate; the embodiment of the present application solves the problem that different features cannot be merged due to large dimensional differences by aligning the texture features, shape features and deep local features; the feature fusion module of the embodiment of the present application responds to the differences in dimensions, semantics and data distribution through a carefully designed process, effectively integrates the advantages of various features, avoids mutual interference, and improves the comprehensive representation capability of features.

[0027] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0029] Figure 1 is a training flow chart of a solar panel surface property detection model according to an embodiment of the present application;

[0030] Figure 2 is a structural schematic diagram of a texture extraction branch according to an embodiment of the present application;

[0031] Figure 3 is a structural schematic diagram of a shape feature extraction branch according to an embodiment of the present application;

[0032] Figure 4 is a structural schematic diagram of a deep local feature extraction branch according to an embodiment of the present application;

[0033] Figure 5 is a structural schematic diagram of a fusion module according to an embodiment of the present application;

[0034] Figure 6 It is a structural block diagram of a device for constructing a solar panel surface property detection model according to an embodiment of the present application;

[0035] Figure 7 It is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0036] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with one or more embodiments of this specification. Instead, they are merely examples of devices and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0037] It should be noted that: in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; and multiple steps described in this specification may be combined into a single step for description in other embodiments.

[0038] Embodiment 1

[0039] The embodiment of the present application provides a method for constructing a solar panel surface property detection model, by cross-dimensionally fusing texture features, shape features, and depth local features of different dimensions in a solar panel image, so as to accurately detect the property category of a solar cell panel. Specifically, referring to Figure 1, the method comprising:

[0040] Constructing a solar panel surface property detection framework, the solar panel surface property detection framework is composed of a feature extraction module, a feature fusion module and a classification head;

[0041] Acquire multiple solar screen images as training images, and each training image is annotated with a property category, wherein the property category includes a mesh state, a resin coating state, a steel screen state, and a PI film lamination state;

[0042] In the feature extraction module, the texture features of each training image are extracted based on the gray level co-occurrence matrix, the shape features of each training image are extracted based on the edge detection algorithm, and the deep local features of each training image are extracted based on multiple convolutional layers;

[0043] In the feature fusion module, the texture features, shape features and deep local features of each training image are fused to obtain the fusion features of each training image;

[0044] The classification head calculates the predicted trait category of each training image based on the fusion features of each training image, and then calculates the trait category error according to the predicted trait category and the labeled trait category of each training image;

[0045] The parameters of the solar panel surface property detection framework are adjusted based on the category error, and the solar panel surface property detection framework after the parameter adjustment is used to continue training with multiple training images until preset training conditions are met, and the parameters of the solar panel surface property detection framework when the preset training conditions are met are saved to obtain a constructed solar panel surface property detection model.

[0046] In some embodiments, a high-resolution industrial camera is used to photograph the manufactured solar screen in a standardized lighting environment to obtain a solar screen image as a training image.

[0047] Specifically, the training images in this scheme include a large number of solar screen images with normal properties, as well as some solar screen images with defects in properties. During the training process, the model is guided to accurately locate the solar screen images with defects in properties among a large number of images by adjusting parameters to complete the training of the solar screen surface property detection model.

[0048] Specifically, if the number of solar screen images with property defects is small, they can be processed by data enhancement to increase the number of samples.

[0049] In some specific embodiments, the mesh state includes unobstructed mesh and blocked mesh, the resin coating state includes uniform coating and uneven coating, the wire mesh state includes broken wire mesh, deformed wire mesh and normal wire mesh, and the PI film bonding state includes normal bonding and abnormal bonding.

[0050] In this solution, before inputting the training images into the solar panel surface property detection framework for training, the training images are preprocessed, the training images are first cropped to a uniform size and then converted into grayscale images and input into the feature extraction module.

[0051] Furthermore, when cropping the training images, irrelevant background parts should be cropped out first, so as to retain the main body of the solar panel. Specifically, by cropping the training images to a uniform size, irrelevant background in the training images can be removed, and only the main body of the solar panel can be trained. In addition, cropping the training images to a uniform size can also ensure the uniformity of the images input into the solar panel surface property detection architecture, thereby achieving better training effects.

[0052] Specifically, the training image is grayed by using the graying function cv2.cvtColor(image,cv2.COLOR_BGR2GRAY) in the image processing library OpenVC.

[0053] In some embodiments, the feature extraction module includes parallel texture feature extraction branches, shape feature extraction branches and depth local feature extraction branches, and the training image is input into the texture feature extraction branch, the shape feature extraction branch and the depth local feature extraction branch respectively to obtain texture features, shape features and depth local features. The texture features represent the spatial distribution law of pixel gray levels in the training image, the shape features represent the geometric shape information of each mesh in the training image, and the depth local features represent the local high-dimensional abstract features in the training image.

[0054] In other words, this solution first performs analysis and feature extraction at the low-dimensional image level through the texture feature extraction branch and the shape feature extraction branch, thereby capturing the texture features and shape features at the image level, and then uses multiple convolutional layers to perform multiple convolutions on the training image, thereby extracting the abstract representation of the training image in a high dimension. These representations include both texture features and shape features, and these features are subsequently integrated for analysis, so that the trait categories of the training images can be analyzed with high accuracy.

[0055] Furthermore, the structure of the texture extraction branch is as follows: Figure 2As shown, in the texture extraction branch, the training image is first grayscale quantized to obtain a quantized training image, and then the grayscale co-occurrence matrix is ​​calculated based on the quantized training image. The grayscale co-occurrence matrix is ​​subjected to feature extraction through a convolutional layer to obtain grayscale co-occurrence matrix features, and finally, feature selection is performed on the grayscale co-occurrence matrix features to obtain texture features.

[0056] Specifically, this scheme processes the training images through grayscale quantization to reduce the complexity of calculation. When calculating the grayscale co-occurrence matrix, it is necessary to consider the frequency of occurrence of each pair of grayscale levels. A conventional grayscale image has 256 grayscale levels, so a 256×256 matrix calculation is required. This scheme will map the grayscale levels in the training image to the specified quantization level in a mapping manner, thereby completing the quantization of the training image.

[0057] Exemplarily, this solution quantizes the grayscale from 256 levels to 16 levels.

[0058] Specifically, the gray-level co-occurrence matrix features include all the features in the gray-level co-occurrence matrix, such as energy, contrast, correlation, entropy, etc., but the discrimination of most features is not high, so they cannot accurately express the texture information in the training image. This scheme uses the variance threshold method to select features with variance greater than the set threshold in the gray-level co-occurrence matrix features as texture features.

[0059] Specifically, by performing feature selection on gray-level co-occurrence features, the prediction time of the model can be greatly reduced and the running speed of the model can be increased.

[0060] Furthermore, the structure diagram of the shape feature extraction branch is as follows: Figure 3 As shown, in the shape extraction branch, the edge detection algorithm is first used to detect the edges in the training image, and then the detected edges are connected to obtain an edge image, and finally the edge image is described using Fourier calculation to obtain shape features.

[0061] Specifically, during the edge detection process, a non-maximum suppression method is used to remove the detected false edges.

[0062] Specifically, this solution describes the grid edges in the training image from the perspective of frequency domain by means of Fourier calculation to obtain shape features.

[0063] Specifically, this solution obtains shape features through Fourier calculation, which can quickly detect shape defects of solar panels. For example, if there is a significant difference in the high-frequency part of the Fourier descriptor, it may mean that there is slight damage or unevenness in the edge contour of the panel, thereby providing an important basis for the subsequent classification process. This is very critical in the quality control of solar cell panels, and can detect quality problems in a timely manner and reduce the defective rate.

[0064] Furthermore, the structure of the deep local feature extraction branch is as follows: Figure 4 As shown, the deep local feature extraction branch is composed of multiple convolution units and a fully connected layer. The training image is convolved by multiple series-connected convolution units and then output by the fully connected layer to obtain the deep local feature. Each convolution unit has the same structure, which is composed of a convolution layer, an activation function layer and a maximum pooling layer connected in series in sequence.

[0065] Specifically, the local high-dimensional features in the training image, such as edges, corners, etc., can be effectively captured through the deep local feature extraction branch. These high-dimensional features are critical for identifying various details of solar cell screens, and can better reflect the essential characteristics of the image, which helps to improve the accuracy of subsequent classification or detection. In addition, after multiple layers of convolution and pooling operations, higher-level and more abstract features can be gradually extracted from low-level local features. For example, from the initial features such as edges and corners, more complex patterns, structures and other features are gradually combined to enable the network to learn more representative and discriminative feature representations, so as to better cope with different types of solar cell screen image recognition tasks.

[0066] In some specific embodiments, the texture features, shape features, and depth local features are dimensionally aligned and then input into a fusion module.

[0067] Exemplarily, this scheme uses the kernel function analysis method to increase the dimension of the texture features and the shape features, and uses the principal component analysis method to map the deep local features to a low-dimensional space for dimensionality reduction, thereby completing the dimensionality unification of the texture features, shape features, and deep local features.

[0068] Specifically, since texture features and shape features are low-dimensional features directly obtained from training images, and deep local features are high-dimensional features extracted by multi-layer convolutional units, these features with large dimensional differences cannot be fused in the fusion module. However, this solution can splice these features in the fusion module after dimensional alignment, thereby making comprehensive use of various feature information and improving the model's ability to analyze training images.

[0069] In some specific embodiments, the structure of the fusion module is as follows Figure 5As shown, the fusion module includes a semantic association unit, a data alignment unit, a normalization unit and a fusion unit. In the semantic association unit, the semantic association relationship between texture features, shape features and depth local features is learned based on the attention mechanism to obtain texture association features, shape association features and depth local association features. In the data alignment unit, the data distribution of texture association features, shape association features and depth local association features is adjusted to obtain texture adjustment features, shape adjustment features and depth local adjustment features. In the normalization unit, the texture adjustment features, shape adjustment features and depth local adjustment features are normalized and then input into the fusion unit for fusion to obtain fusion features.

[0070] Specifically, this scheme adopts attention calculation method in the semantic association unit to obtain the semantic association weights between texture features, shape features and deep local features, and dynamically adjusts the degree of association between texture features, shape features and deep local features based on the semantic association weights, so that texture features, shape features and deep local features are more closely combined semantically, so that the model can better understand the semantic representation of the same trait under different features.

[0071] Specifically, texture features, shape features, and deep local features are extracted in different ways, so there are significant differences in the representation of these features. For example, the values ​​of texture features may be concentrated in a small range, the values ​​of shape features may vary greatly due to different geometric shapes, and deep local features may be affected by the activation function of the neural network, and their distribution is more complex. This difference in data distribution may cause certain features to dominate the model training process after fusion, affecting the performance of the model.

[0072] Therefore, this scheme estimates the probability density functions of texture features, shape features, and depth local features respectively, and adjusts the data distribution of texture-related features, shape-related features, and depth local-related features based on the probability density function of each feature, so that the obtained texture adjustment features, shape adjustment features, and depth local adjustment features meet the standard normal distribution, thereby reducing the fusion problem caused by data distribution differences.

[0073] In some specific embodiments, the classification head in this solution outputs the probability distribution of each training sample in different trait categories.

[0074] Specifically, since solar panels may have multiple characteristics, such as mesh blockage and wire mesh breakage, output in a probability distribution manner is more suitable for actual applications.

[0075] In some specific embodiments, the probability distribution output by the classification head is compared with the labeled traits of the corresponding training samples to calculate the trait category error, and the parameters of the solar panel surface trait detection architecture are continuously adjusted according to the obtained trait category error to perform iterative training.

[0076] In some specific embodiments, the preset training condition in this solution may be that the loss function meets a threshold. If the set number of iterations is reached, the specific training condition is subject to the actual situation and is not limited here.

[0077] Embodiment 2

[0078] Based on the same idea, refer to Figure 6 The present application also proposes a device for constructing a solar screen surface property detection model, comprising:

[0079] A construction module is used to construct a solar panel surface property detection framework, wherein the solar panel surface property detection framework is composed of a feature extraction module, a feature fusion module and a classification head;

[0080] An acquisition module, used for acquiring a plurality of solar screen images as training images, and each training image is annotated with a property category, wherein the property category includes a mesh state, a resin coating state, a steel screen state, and a PI film lamination state;

[0081] A feature extraction module extracts the texture features of each training image based on the gray-level co-occurrence matrix, extracts the shape features of each training image based on the edge detection algorithm, and extracts the deep local features of each training image based on multiple convolutional layers;

[0082] A feature fusion module is used to fuse the texture features, shape features and deep local features of each training image to obtain a fusion feature of each training image;

[0083] A classification module calculates the predicted trait category of each training image based on the fusion features of each training image, and then calculates the trait category error based on the predicted trait category and the labeled trait category of each training image;

[0084] A parameter adjustment module adjusts the parameters of the solar panel surface property detection framework based on the category error, continues training with the parameter-adjusted solar panel surface property detection framework and multiple training images until preset training conditions are met, and saves the parameters of the solar panel surface property detection framework when the preset training conditions are met to obtain a constructed solar panel surface property detection model.

[0085] Embodiment 3

[0086] This embodiment also provides an electronic device, referring to Figure 7, comprises a memory 404 and a processor 402, wherein the memory 404 stores a computer program, and the processor 402 is configured to run the computer program to execute the steps in any of the above method embodiments.

[0087] Specifically, the processor 402 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0088] Among them, the memory 404 may include a large capacity memory 404 for data or instructions. For example, but not limitation, the memory 404 may include a hard disk drive (HDD), a floppy disk drive, a solid state drive (SSD), a flash memory, an optical disk, a magneto-optical disk, a tape, or a universal serial bus (USB) drive, or a combination of two or more of these. In appropriate cases, the memory 404 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 404 may be inside or outside the data processing device. In a specific embodiment, the memory 404 is a non-volatile memory. In a specific embodiment, the memory 404 includes a read-only memory (ROM) and a random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (Programmable Read-Only Memory, PROM for short), an erasable PROM (Erasable Programmable Read-Only Memory, EPROM for short), an electrically erasable PROM (Electrically Erasable Programmable Read-Only Memory, EEPROM for short), an electrically alterable ROM (Electrically Alterable Read-Only Memory, EAROM for short) or a flash memory (FLASH) or a combination of two or more of these. In appropriate circumstances, the RAM may be a static random access memory (SRAM) or a dynamic random access memory (DRAM), wherein the DRAM may be a fast page mode dynamic random access memory 404 (FPMDRAM), an extended data output dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.

[0089] The memory 404 may be used to store or cache various data files required for processing and / or communication, as well as possible computer program instructions executed by the processor 402 .

[0090] The processor 402 reads and executes the computer program instructions stored in the memory 404 to implement any one of the methods for constructing a solar panel surface property detection model in the above embodiments.

[0091] Optionally, the electronic device may further include a transmission device 406 and an input / output device 408 , wherein the transmission device 406 is connected to the processor 402 , and the input / output device 408 is connected to the processor 402 .

[0092] The transmission device 406 can be used to receive or send data via a network. Specific examples of the above-mentioned network may include a wired or wireless network provided by a communication provider of the electronic device. In one example, the transmission device includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 406 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

[0093] The input and output device 408 is used to input or output information. In this embodiment, the input information may be training images with labeled information, and the output information may be the predicted trait category of each training image.

[0094] Optionally, in this embodiment, the processor 402 may be configured to perform the following steps through a computer program:

[0095] Constructing a solar panel surface property detection framework, the solar panel surface property detection framework is composed of a feature extraction module, a feature fusion module and a classification head;

[0096] Acquire multiple solar screen images as training images, and each training image is annotated with a property category, wherein the property category includes a mesh state, a resin coating state, a steel screen state, and a PI film lamination state;

[0097] In the feature extraction module, the texture features of each training image are extracted based on the gray level co-occurrence matrix, the shape features of each training image are extracted based on the edge detection algorithm, and the deep local features of each training image are extracted based on multiple convolutional layers;

[0098] In the feature fusion module, the texture features, shape features and deep local features of each training image are fused to obtain the fusion features of each training image;

[0099] The classification head calculates the predicted trait category of each training image based on the fusion features of each training image, and then calculates the trait category error according to the predicted trait category and the labeled trait category of each training image;

[0100] The parameters of the solar panel surface property detection framework are adjusted based on the category error, and the solar panel surface property detection framework after the parameter adjustment is used to continue training with multiple training images until preset training conditions are met, and the parameters of the solar panel surface property detection framework when the preset training conditions are met are saved to obtain a constructed solar panel surface property detection model.

[0101] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.

[0102] In general, various embodiments may be implemented in hardware or dedicated circuits, software, logic, or any combination thereof. Some aspects of the invention may be implemented in hardware, while other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor, or other computing device, but the invention is not limited thereto. Although various aspects of the invention may be shown and described as block diagrams, flow charts, or using some other graphical representation, it should be understood that, as non-limiting examples, the boxes, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, dedicated circuits or logic, general-purpose hardware or controllers or other computing devices, or some combination thereof.

[0103] Embodiments of the present invention may be implemented by computer software that is executable by a data processor of a mobile device, such as in a processor entity, or by hardware, or by a combination of software and hardware. Computer software or programs (also referred to as program products) including software routines, applets and / or macros may be stored in any device-readable data storage medium, and they include program instructions for performing specific tasks. A computer program product may include one or more computer executable components configured to perform an embodiment when the program is run. One or more computer executable components may be at least one software code or a portion thereof. In addition, at this point, it should be noted that, for example, Figure 7Any block of the logic flow in the program may represent program steps, or interconnected logic circuits, blocks and functions, or a combination of program steps and logic circuits, blocks and functions. The software may be stored on physical media such as memory chips or storage blocks implemented within a processor, magnetic media such as hard disks or floppy disks, and optical media such as, for example, DVDs and their data variants, CDs, etc. Physical media are non-transitory media.

[0104] Those skilled in the art should understand that the technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0105] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for constructing a solar screen surface property detection model, characterized in that: The following steps are involved: Constructing a solar panel surface property detection framework, the solar panel surface property detection framework is composed of a feature extraction module, a feature fusion module and a classification head; A plurality of solar screen images are obtained as training images, and each training image is annotated with a property category, wherein the property category includes mesh state, resin coating state, wire mesh state, and PI film lamination state, wherein the mesh state includes mesh unblocked and mesh blocked, the resin coating state includes coating uniform and coating uneven, the wire mesh state includes wire mesh breakage, wire mesh deformation, and wire mesh normal, and the PI film lamination state includes lamination normal and lamination abnormal; In the feature extraction module, the texture features of each training image are extracted based on the gray-level co-occurrence matrix, and the features with variance greater than the set threshold are selected as texture features in the gray-level co-occurrence matrix features through the variance threshold method, the shape features of each training image are extracted based on the edge detection algorithm, and the depth local features of each training image are extracted based on multiple convolutional layers. The texture features represent the spatial distribution law of the gray level of pixels in the training image, the shape features represent the geometric shape information of each mesh in the training image, and the depth local features represent the local high-dimensional abstract features in the training image. The probability density functions of the texture features, shape features and depth local features are estimated respectively, and the data distribution of the texture association features, shape association features and depth local association features are adjusted respectively based on the probability density function of each feature, so that the obtained texture adjustment features, shape adjustment features and depth local adjustment features meet the standard normal distribution; In a feature fusion module, the texture features, shape features and deep local features of each training image are fused to obtain a fused feature of each training image, wherein the feature fusion module includes a semantic association unit, a data alignment unit, a normalization unit and a fusion unit, in the semantic association unit, the semantic association relationship between the texture features, shape features and deep local features is learned based on an attention mechanism to obtain texture association features, shape association features and deep local association features, in the data alignment unit, the data distribution of the texture association features, shape association features and deep local association features is adjusted to obtain texture adjustment features, shape adjustment features and deep local adjustment features, in the normalization unit, the texture adjustment features, shape adjustment features and deep local adjustment features are normalized and then input into the fusion unit for fusion to obtain a fused feature; The classification head calculates the predicted trait category of each training image based on the fusion features of each training image, and then calculates the trait category error according to the predicted trait category and the labeled trait category of each training image; The parameters of the solar panel surface property detection framework are adjusted based on the category error, and the solar panel surface property detection framework after the parameter adjustment is used to continue training with multiple training images until preset training conditions are met, and the parameters of the solar panel surface property detection framework when the preset training conditions are met are saved to obtain a constructed solar panel surface property detection model.

2. The method for constructing a solar panel surface property detection model according to claim 1, characterized in that: The feature extraction module includes a parallel texture feature extraction branch, a shape feature extraction branch and a deep local feature extraction branch. The training image is input into the texture feature extraction branch, the shape feature extraction branch and the deep local feature extraction branch respectively to obtain texture features, shape features and deep local features.

3. The method for constructing a solar panel surface property detection model according to claim 2, characterized in that: In the texture extraction branch, the training image is first grayscale quantized to obtain a quantized training image, and then a grayscale co-occurrence matrix is ​​calculated based on the quantized training image. Finally, a convolutional layer is used to extract features in the grayscale co-occurrence matrix to obtain texture features.

4. The method for constructing a solar panel surface property detection model according to claim 2, characterized in that: In the shape extraction branch, the edges in the training image are first detected using an edge detection algorithm, the detected edges are then connected to obtain an edge image, and finally the edge image is described using Fourier calculation to obtain shape features.

5. The method for constructing a solar panel surface property detection model according to claim 2, characterized in that: The deep local feature extraction branch is composed of multiple convolution units and a fully connected layer. The training image is convolved by multiple series-connected convolution units and then output by the fully connected layer to obtain the deep local feature. Each convolution unit has the same structure, which is composed of a convolution layer, an activation function layer and a maximum pooling layer connected in series in sequence.

6. The method for constructing a solar panel surface property detection model according to claim 1, characterized in that: The texture features, shape features and depth local features are dimensionally aligned and then input into a fusion module.

7. A device for constructing a solar screen surface property detection model, characterized in that: include: A construction module is used to construct a solar panel surface property detection framework, wherein the solar panel surface property detection framework is composed of a feature extraction module, a feature fusion module and a classification head; an acquisition module, for acquiring a plurality of solar screen images as training images, and each training image is annotated with a property category, wherein the property category includes mesh state, resin coating state, wire mesh state, and PI film lamination state, wherein the mesh state includes mesh unblocked and mesh blocked, the resin coating state includes uniform coating and uneven coating, the wire mesh state includes wire mesh breakage, wire mesh deformation, and wire mesh normal, and the PI film lamination state includes normal lamination and abnormal lamination; A feature extraction module, which extracts texture features of each training image based on a gray-level co-occurrence matrix, selects features with variances greater than a set threshold as texture features from the gray-level co-occurrence matrix features through a variance threshold method, extracts shape features of each training image based on an edge detection algorithm, and extracts depth local features of each training image based on multiple convolutional layers, wherein the texture features represent the spatial distribution law of the gray levels of pixels in the training image, the shape features represent the geometric shape information of each mesh in the training image, and the depth local features represent the local high-dimensional abstract features in the training image, and estimates the probability density functions of the texture features, shape features, and depth local features respectively, and adjusts the data distribution of the texture association features, shape association features, and depth local association features based on the probability density function of each feature, so that the obtained texture adjustment features, shape adjustment features, and depth local adjustment features meet the standard normal distribution; A feature fusion module, used for fusing the texture features, shape features and deep local features of each training image to obtain a fusion feature of each training image, wherein the feature fusion module includes a semantic association unit, a data alignment unit, a normalization unit and a fusion unit, in which the semantic association relationship between the texture features, shape features and deep local features is learned based on an attention mechanism to obtain texture association features, shape association features and deep local association features, in which the data distribution of the texture association features, shape association features and deep local association features is adjusted to obtain texture adjustment features, shape adjustment features and deep local adjustment features, and in which the texture adjustment features, shape adjustment features and deep local adjustment features are normalized in the normalization unit and then input into the fusion unit for fusion to obtain a fusion feature; A classification module calculates the predicted trait category of each training image based on the fusion features of each training image, and then calculates the trait category error based on the predicted trait category and the labeled trait category of each training image; A parameter adjustment module adjusts the parameters of the solar panel surface property detection framework based on the category error, continues training with the parameter-adjusted solar panel surface property detection framework and multiple training images until preset training conditions are met, and saves the parameters of the solar panel surface property detection framework when the preset training conditions are met to obtain a constructed solar panel surface property detection model.

8. An electronic device comprising a memory and a processor, characterized in that: The memory stores a computer program, and the processor is configured to run the computer program to execute the method for constructing a solar panel surface property detection model as described in any one of claims 1-6.

9. A readable storage medium, characterized in that: The readable storage medium stores a computer program, which includes a program code for controlling a process to execute a process, and the process includes a method for constructing a solar panel surface property detection model according to any one of claims 1-6.

Images