A photovoltaic cell defect detection method, system, terminal and medium based on real EL image and knowledge graph fusion integration discrimination

Abstract

The application discloses a photovoltaic cell defect detection method and system based on real EL image and knowledge graph fusion integration discrimination, a terminal and a medium, and belongs to the technical field of photovoltaic cell detection. The method comprises the following steps: extracting statistical features based on electroluminescence images; constructing a hot spot fault knowledge graph comprising fault nodes, risk nodes, mechanism nodes and maintenance suggestion nodes; generating mechanism priori representation for a to-be-discriminated sample; taking deep semantic features, statistical features and mechanism priori representation as inputs, taking a risk ordered constraint optimization target as an input to drive training of a heterogeneous multi-modal fusion discrimination model, and outputting defect grade classification probability; performing subgraph retrieval, and outputting defect risk assessment scores, risk transmission paths and maintenance treatment suggestions. The application realizes integrated output of defect grade classification and electric-thermal-force coupling damage risk transmission paths by performing heterogeneous multi-modal fusion of mechanism priori representation of the hot spot fault knowledge graph and deep semantic features and statistical features.

Description

Technical Field

[0001] This invention relates to the field of photovoltaic cell testing technology, specifically to a photovoltaic cell defect detection method, system, terminal, and medium based on the fusion and integration of real EL images and knowledge graphs. Background Technology

[0002] Photovoltaic modules, during long-term operation, are susceptible to factors such as microcracks, poor welding, contamination, and material aging, which can induce faults such as increased local resistance, uneven temperature field, and hot spot expansion. Existing defect identification methods based on electroluminescent images typically only extract convolutional features from the image, lacking technical means to unify the deep semantic features and interpretable statistical features of the image with the knowledge of the electro-thermal-mechanical coupling damage mechanism into a single discriminative framework. This makes it difficult for the model to provide risk transmission paths and maintenance and treatment basis based on physical mechanisms while outputting defect level classification results, failing to meet the application requirements of diagnostic and traceable defect identification results in engineering sites. Summary of the Invention

[0003] In view of the above-mentioned problems, the present invention provides a method, system, terminal and medium for detecting defects in photovoltaic cells based on the fusion and integrated discrimination of real EL images and knowledge graphs.

[0004] Therefore, the technical problem solved by this invention is: how to unify the deep semantic features, statistical features and prior features of hot spot fault knowledge graph mechanism of photovoltaic cell electroluminescence image into the same discrimination framework, so as to achieve defect level classification and output risk transmission path and maintenance and disposal suggestions based on the electro-thermal-mechanical coupling damage mechanism, so as to meet the application needs of the engineering site for the defect discrimination results to be diagnosable and traceable.

[0005] To address the aforementioned technical problems, this invention provides the following technical solution: a photovoltaic cell defect detection method based on the fusion and integration of real EL images and knowledge graphs, comprising, Based on the electroluminescent images of photovoltaic cells, statistical features reflecting gray intensity distribution, spatial structure differences and edge gradient response are extracted. A hot spot fault knowledge graph is constructed, which is composed of fault nodes, risk nodes, observation signal nodes, mechanism nodes and maintenance suggestion nodes. The relationship between nodes is used to encode the mechanism relationship of hot spot expansion and damage. Based on the hot spot fault knowledge graph, a priori representation of the generation mechanism of the sample to be judged is provided. Deep semantic features are extracted from the electroluminescent image using a pre-trained visual network. The deep semantic features, statistical features, and prior representations of the mechanism are used as inputs. The training of the heterogeneous multimodal fusion discrimination model is driven by the risk-ordered constraint optimization objective, and the defect level classification probability is output. Based on the hot spot fault knowledge graph, a subgraph retrieval is performed. The retrieved subgraph information and the defect level classification probability are used as the basis to output the defect risk assessment score, risk transmission path and maintenance and handling suggestions.

[0006] As a preferred embodiment of the photovoltaic cell defect detection method based on the fusion and integration of real EL images and knowledge graphs described in this invention, the prior characterization of the mechanism includes: Using the normalized adjacency matrix and learnable node embeddings of the hot spot fault knowledge graph as input, the full graph node embeddings are obtained by encoding through the knowledge graph encoder. From the full-graph node embedding, the context node embedding is extracted using the context node set consisting of dataset nodes, crystal type nodes, risk nodes, and observation signal nodes as an index. Through multi-granularity aggregation operations and dimensional compression mapping, the context nodes are embedded and compressed into a priori representation of the mechanism.

[0007] As a preferred embodiment of the photovoltaic cell defect detection method based on the fusion and integrated discrimination of real EL images and knowledge graphs described in this invention, the heterogeneous multimodal fusion discrimination model includes: The image encoder maps the deep semantic features to image representations; The statistical feature encoder maps the statistical features to statistical representations; The prior representation encoder maps the prior representation of the mechanism to a knowledge representation; The image representation, statistical representation, and knowledge representation are concatenated into a fusion vector. Gating weights are generated based on the fusion vector. The fusion vector is then subjected to residual weighted fusion using the gating weights. Finally, the defect level classification probability is output through the classification head.

[0008] As a preferred embodiment of the photovoltaic cell defect detection method based on the fusion and integration of real EL images and knowledge graphs described in this invention, the risk-ordered constraint optimization objective includes: Based on the imbalance in the distribution of category samples, differential weighting coefficients are applied to the loss terms of each category; Based on the orderliness of defect levels, a level deviation penalty term is applied to the predicted distribution; Using the embedded spatial structure of the hot spot fault knowledge graph as a supervision signal, a structural regularization constraint term is applied to the prediction distribution; Apply parameter norm constraints to the network parameters based on the parameter norm; The differential weighting coefficient, the strength coefficient of the grade deviation penalty term, the strength coefficient of the structural regularization constraint term, and the strength coefficient of the parameter norm constraint term are determined by feedback adjustment of the validation set.

[0009] As a preferred embodiment of the photovoltaic cell defect detection method based on the fusion and integration of real EL images and knowledge graphs described in this invention, the integration decision of the heterogeneous multimodal fusion discrimination model includes: Multiple traditional classifiers are trained using the deep semantic features, the statistical features, and the prior mechanistic representations as common inputs. On the validation set, with classification performance metrics as the optimization objective, the optimal weighted combination of the defect level classification probabilities output by the heterogeneous multimodal fusion discriminant model and the multiple traditional classifiers is searched. The final defect level classification result is determined by the sum of the probabilities of the optimal weighted combination.

[0010] As a preferred embodiment of the photovoltaic cell defect detection method based on the fusion and integration of real EL images and knowledge graphs described in this invention, the statistical features include: A statistic reflecting the distribution of gray intensity; Statistical measures reflecting differences in spatial structure; Statistics reflecting the edge gradient response; The structured numerical encoding of the crystal type is concatenated with the statistics reflecting gray intensity distribution, spatial structure differences, and edge gradient response to form a structured feature vector.

[0011] As a preferred embodiment of the photovoltaic cell defect detection method based on the fusion and integration of real EL images and knowledge graphs described in this invention, wherein: the hot spot fault knowledge graph includes: Nodes in the electrical domain are connected to nodes in the thermal domain via directed relation edges that characterize current-carrying anomalies. Nodes in the thermal domain are connected to nodes in the mechanical domain via directed relation edges that characterize anomalies in the temperature field distribution. The mechanical domain nodes are connected to the hot spot propagation nodes via directed relation edges characterizing material damage accumulation. The directed relation edges of the current-carrying anomaly, the directed relation edges of the temperature field distribution anomaly, and the directed relation edges of the material damage accumulation are connected in series to form a directed conduction path for damage caused by electro-thermal-mechanical coupling.

[0012] This invention provides a photovoltaic cell defect detection system based on the fusion and integrated discrimination of real EL images and knowledge graphs.

[0013] To address the aforementioned technical problems, this invention provides the following technical solution: a photovoltaic cell defect detection system based on the fusion and integrated discrimination of real EL images and knowledge graphs, comprising: The statistical feature extraction module is used to extract statistical features reflecting gray intensity distribution, spatial structure differences, and edge gradient response from electroluminescent images based on photovoltaic cells. The knowledge graph construction module is used to construct a hot spot fault knowledge graph. The hot spot fault knowledge graph is composed of fault nodes, risk nodes, observation signal nodes, mechanism nodes and maintenance suggestion nodes. The relationship between nodes is used to encode the mechanism relationship of hot spot expansion and damage. The prior characterization generation module is used to generate a prior characterization of the mechanism for the sample to be judged based on the hot spot fault knowledge graph. A deep feature extraction module is used to extract deep semantic features from the electroluminescent image using a pre-trained visual network; The fusion discrimination module is used to take the deep semantic features, the statistical features and the mechanism prior representation as inputs, drive the training of the heterogeneous multimodal fusion discrimination model with risk-ordered constraint optimization objective, and output the defect level classification probability. An integrated decision module is used to train multiple traditional classifiers with the deep semantic features, the statistical features, and the prior representation of the mechanism as common inputs. On the validation set, it searches for the optimal weighted combination of the defect level classification probabilities output by the heterogeneous multimodal fusion discriminant model and the multiple traditional classifiers. The final defect level classification result is determined by the sum of the probabilities of the optimal weighted combination. The subgraph retrieval module is used to perform subgraph retrieval based on the hot spot fault knowledge graph. Based on the retrieved subgraph information and the defect level classification probability, it outputs the defect risk assessment score, risk transmission path and maintenance and handling suggestions.

[0014] The present invention provides a computer device, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the photovoltaic cell defect detection method based on the fusion and integration of real EL images and knowledge graphs.

[0015] The present invention provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, it implements the steps of the photovoltaic cell defect detection method based on the fusion and integration of real EL images and knowledge graphs.

[0016] The beneficial effects of this invention are as follows: This invention integrates the prior representation of the mechanism of hot spot fault knowledge graph with deep semantic features and structured feature vectors in a heterogeneous multimodal manner. The training of the heterogeneous multimodal fusion discrimination model is driven by the risk-ordered constraint optimization objective. On the ELPV dataset, it achieves a test set accuracy of 74.86%, a weighted F1 score of 74.02%, and an AUC of 83.12%. Through subgraph retrieval based on the hot spot fault knowledge graph, the defect level classification probability is associated with the directed transmission path of electro-thermal-mechanical coupling damage. This enables the defect discrimination results to trace the complete relationship chain from the classification conclusion to the defect symptoms, mechanism causes, risk expansion, and treatment suggestions, meeting the application needs of diagnostic and traceable defect discrimination results in engineering sites. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating the overall process of a photovoltaic cell defect detection method based on the fusion and integration of real EL images and knowledge graphs, as provided in one embodiment of the present invention.

[0019] Figure 2 This is a KG-RealFusion network architecture diagram for a photovoltaic cell defect detection method based on the fusion and integrated discrimination of real EL images and knowledge graphs, provided as an embodiment of the present invention.

[0020] Figure 3 This is a multimodal statistical feature distribution analysis diagram of electroluminescence images provided in an embodiment of the present invention for a photovoltaic cell defect detection method based on the fusion and integration of real EL images and knowledge graphs.

[0021] Figure 4 This is a schematic diagram of the hot spot fault knowledge graph structure of a photovoltaic cell defect detection method based on the fusion and integration of real EL images and knowledge graphs, provided as an embodiment of the present invention.

[0022] Figure 5 This is an overview of the model's comprehensive performance and knowledge retrieval results for a photovoltaic cell defect detection method based on the fusion and integration of real EL images and knowledge graphs, provided as an embodiment of the present invention.

[0023] Figure 6This is a test set risk ranking and category diagnosis analysis diagram for a photovoltaic cell defect detection method based on the fusion and integration of real EL images and knowledge graphs, provided as an embodiment of the present invention.

[0024] Figure 7 This is a validation set model selection and soft voting weight configuration diagram for a photovoltaic cell defect detection method based on the fusion and integration of real EL images and knowledge graphs, provided as an embodiment of the present invention.

[0025] Figure 8 This is a confusion matrix diagram of the test set for a photovoltaic cell defect detection method based on the fusion and integration of real EL images and knowledge graphs, provided as an embodiment of the present invention. Detailed Implementation

[0026] To make the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0027] Example 1, referring to Figures 1-2 This is one embodiment of the present invention, which provides a photovoltaic cell defect detection method based on the fusion and integrated discrimination of real EL images and knowledge graphs, including: S1, based on the electroluminescent image of photovoltaic cells, extracts statistical features reflecting gray intensity distribution, spatial structure differences and edge gradient response.

[0028] It should be noted that the electroluminescent image is accompanied by a corresponding defect level label and crystal type information. The processor establishes a correspondence between the electroluminescent image and the defect category based on the defect level label and crystal type information.

[0029] In some embodiments, the processor converts the electroluminescent image into a grayscale image, adjusts the grayscale image to a preset size, and then executes steps S101 to S102.

[0030] In some embodiments, the statistical features in step S1 include steps S101 to S102, wherein: S101 extracts statistics reflecting gray intensity distribution, statistics reflecting spatial structure differences, and statistics reflecting edge gradient response.

[0031] In some embodiments, the statistics reflecting the grayscale intensity distribution include EL mean, EL standard deviation, EL entropy, EL contrast, P10 quantile, P25 quantile, median, P75 quantile, P90 quantile, dark pixel proportion, and bright pixel proportion, totaling 11 items; the statistics reflecting spatial structure differences include EL edge response, central region mean, edge region mean, and central edge brightness difference, totaling 4 items; the statistics reflecting edge gradient response include gradient mean and gradient standard deviation, totaling 2 items; the statistics reflecting grayscale intensity distribution, spatial structure differences, and edge gradient response total 17 items, all of which are directly calculated from the grayscale image.

[0032] Understandably, the EL mean and EL standard deviation are the arithmetic mean and root mean square deviation of the grayscale values ​​of all pixels in the image, respectively; the EL entropy is calculated based on the histogram of the grayscale values ​​of the entire image, reflecting the complexity of the image's grayscale distribution; based on the ascending order of the grayscale values ​​of all pixels in the image, the P10 quantile is the grayscale value at the 10th percentile, the P25 quantile is the grayscale value at the 25th percentile, the median is the grayscale value at the 50th percentile, the P75 quantile is the grayscale value at the 75th percentile, and the P90 quantile is the grayscale value at the 90th percentile; the proportion of dark pixels and the proportion of bright pixels are both determined by the grayscale values. The grayscale image is directly calculated, reflecting the proportion of dark and bright pixels in the total number of pixels in the image. The EL edge response is calculated by the response amplitude after the edge detection operator is applied to the grayscale image. The mean of the central region and the mean of the edge region are the arithmetic mean of the grayscale values ​​of all pixels in the central and edge regions of the battery, respectively. The brightness difference between the central and edge regions is the difference between the mean of the central region and the mean of the edge regions. The gradient mean and gradient standard deviation are the arithmetic mean and root mean square deviation of the gradient amplitude of each pixel in the entire image, respectively. The gradient amplitude is calculated by the gradient operator applied to the grayscale image.

[0033] S102, the structured numerical code of the crystal type is concatenated with the statistics reflecting the gray intensity distribution, the statistics reflecting the spatial structure difference, and the statistics reflecting the edge gradient response to form a structured feature vector.

[0034] In some embodiments, crystal types are divided into single crystal and polycrystalline types. The processor converts the crystal type information into a numerical code to form a 1D crystal type code. The processor concatenates the 1D crystal type code with statistics reflecting gray intensity distribution, statistics reflecting spatial structure differences, and statistics reflecting edge gradient response to form an 18-dimensional structured feature vector.

[0035] It is understandable that monocrystalline and polycrystalline cells have systematic differences in luminescence uniformity, internal boundary structure, and defect evolution mechanism. By incorporating the structured numerical encoding of crystal type into the structured feature vector, the heterogeneous multimodal fusion discrimination model can distinguish the defect distribution patterns under different crystal types in the statistical feature space.

[0036] S2. Construct a hot spot fault knowledge graph. The hot spot fault knowledge graph is composed of fault nodes, risk nodes, observation signal nodes, mechanism nodes and maintenance suggestion nodes. The relationship between nodes is used to encode the mechanism relationship of hot spot expansion and damage.

[0037] In some embodiments, the processor first obtains candidate knowledge from statistical results of real images, annotations of public datasets, fault mechanism texts, and manually compiled maintenance experience; secondly, it extracts defect types, risk levels, observed signals, mechanistic causes, and maintenance and disposal suggestions into "entity-relationship-entity" triples; thirdly, it merges, resolves, and standardizes the naming of synonymous nodes, duplicate relations, and conflict relations to complete the knowledge fusion process; finally, it generates a computable graph structure from the fused knowledge, which is stored as an adjacency matrix, triple table, or graph database structure according to engineering needs. In some embodiments, the processor transforms the hot spot fault knowledge graph into a normalized adjacency matrix of the hot spot fault knowledge graph.

[0038] Understandably, the hot spot fault knowledge graph includes seven node categories: dataset nodes, crystal type nodes, fault nodes, risk nodes, observed signal nodes, mechanism nodes, and maintenance suggestion nodes. Dataset nodes include ELPV datasets; crystal type nodes include single crystal and polycrystalline; fault nodes include normal, mild, moderate, and severe defects; risk nodes include hot spot risk, microcrack and current-carrying anomaly risk, welding and interconnect anomaly risk, and shielding and contamination risk; mechanism nodes include local resistance increase, current redistribution, local temperature rise, temperature field anomaly, thermal stress concentration, microcrack propagation, material degradation, and hot spot propagation; and maintenance suggestion nodes include arranging EL re-inspection and inspecting solder strips and interconnects.

[0039] Furthermore, the processor combines the dominant crystal type, signal change trend, and mechanism hints under different defect levels to perform data-driven expansion of the hot spot fault knowledge graph, resulting in a hot spot fault knowledge graph structure containing 53 nodes and 62 relational edges.

[0040] In some embodiments, the hot spot fault knowledge graph in step S2 includes steps S201 to S204, wherein: S201, the electrical domain node is connected to the thermal domain node via a directed relation edge representing the current-carrying anomaly.

[0041] S202, the thermal domain nodes are connected to the mechanical domain nodes via directed relation edges that characterize the anomalies in the temperature field distribution.

[0042] S203, the mechanical domain node is connected to the hot spot propagation node via a directed relation edge characterizing the accumulation of material damage.

[0043] S204, the directed relation edges of current-carrying anomaly, temperature field distribution anomaly, and material damage accumulation are connected in series to form a directed conduction path for damage caused by electro-thermal-mechanical coupling.

[0044] It is understandable that defects or contamination in photovoltaic cells lead to increased local resistance, which in turn causes uneven current distribution and enhanced Joule heating. Enhanced Joule heating leads to local temperature rise and widening of the temperature gradient, which in turn induces thermal stress concentration. Thermal stress concentration causes microcrack propagation or deterioration of encapsulation materials, thus forming a positive feedback process of hot spot propagation. The directed conduction path of electro-thermal-mechanical coupling damage explicitly encodes the above physical processes as nodes and directed relation edges in the hot spot fault knowledge graph.

[0045] S3, based on the hot spot fault knowledge graph, serves as a priori representation of the generation mechanism of the sample to be judged.

[0046] In some embodiments, the mechanism prior characterization in step S3 includes steps S301 to S303, wherein: S301 takes the normalized adjacency matrix and learnable node embeddings of the hot spot fault knowledge graph as input, and encodes them by the knowledge graph encoder to obtain the full graph node embeddings.

[0047] In some embodiments, the knowledge graph encoder includes two graph convolutional layers connected in series. Each graph convolutional layer is followed by ReLU activation, Dropout, and LayerNorm processing. The normalized adjacency matrix of the hot spot fault knowledge graph and the learnable node embeddings are used as inputs to obtain the full graph node embeddings.

[0048] Understandably, the learnable node embeddings are updated synchronously with the network parameters during training, enabling the semantic information of each node in the hot spot fault knowledge graph to be vectorized. The knowledge graph encoder aggregates the information of neighboring nodes through two layers of graph convolutional layers, so that the embedding representation of each node simultaneously contains its own semantics and neighborhood structure information.

[0049] S302, extract the context node embedding from the full graph node embedding, using the context node set consisting of dataset nodes, crystal type nodes, risk nodes and observation signal nodes as the index.

[0050] In some embodiments, the context node set consists of 1 dataset node, 1 crystal type node, 1 risk node, and 3 observation signal nodes, for a total of 6 nodes. The crystal type node is determined based on the crystal type information of the sample to be judged, and the observation signal node is determined based on the statistical anomaly degree of the sample to be judged. The corresponding 6 context node embeddings are extracted from the full-map node embedding by the node number index of the context node set.

[0051] S303, through multi-granularity aggregation operations and dimensional compression mapping, embeds and compresses context nodes into mechanistic prior representations.

[0052] In some embodiments, mean pooling and max pooling are performed on the embeddings of the six context nodes respectively. The mean pooling result and the max pooling result are concatenated into an aggregate vector, which is then compressed into a 128-dimensional mechanism prior representation through a fully connected layer.

[0053] Understandably, mean pooling captures the overall distribution features of the six context node embeddings, while max pooling captures the significant response features in the six context node embeddings. The mean pooling result and the max pooling result are concatenated and then compressed through a fully connected layer, so that the 128-dimensional mechanism prior representation retains both the overall distribution information and the local saliency information.

[0054] S4 utilizes a pre-trained visual network to extract deep semantic features from electroluminescent images. Using deep semantic features, statistical features, and prior mechanistic representations as inputs, the training of the heterogeneous multimodal fusion discrimination model is driven by the risk-ordered constraint optimization objective, and the defect level classification probability is output.

[0055] In some embodiments, the single-channel grayscale image of the electroluminescent image is copied as a three-channel input, the size of the three-channel input is adjusted to 224×224, and input into a pre-trained ResNet18 model. The output of the global pooling layer after removing the final classification layer of the pre-trained ResNet18 model is truncated to obtain 512-dimensional deep semantic features.

[0056] Understandably, the pre-trained ResNet18 model is pre-trained on a large-scale image dataset, enabling it to extract high-level semantic information such as crack morphology, grayscale degradation, and local dark spots. By truncating the output of the global pooling layer instead of the final classification layer, the 512-dimensional deep semantic features retain the general semantic expression of the image and are not limited by the original classification task.

[0057] In some embodiments, such as Figure 2 As shown, the heterogeneous multimodal fusion discrimination model is implemented using the KG-RealFusion network, which includes a knowledge graph encoder, an image encoder, a statistical feature encoder, a knowledge graph projector, a fusion gating unit, and a classifier.

[0058] It should be noted that the input to the KG-RealFusion network includes three categories: the first input is a 512-dimensional deep semantic feature obtained by removing the final classification layer and global pooling from the electroluminescent image using a pre-trained ResNet18 model; the second input is an 18-dimensional structured feature vector composed of 17 statistical terms calculated from the electroluminescent image and a 1-dimensional crystal type code; the third input is the context node set number corresponding to the sample to be judged and the normalized adjacency matrix of the hot spot fault knowledge graph. The output of the KG-RealFusion network is a 4-dimensional defect level classification probability vector and its corresponding defect risk assessment score. The defect risk assessment score is obtained by subtracting the normal class defect level classification probability from 1, thus forming a computable intrinsic relationship between the gray-scale anomalies, statistical anomalies of the electroluminescent image, and the mechanism nodes of the hot spot fault knowledge graph.

[0059] It should be noted that the knowledge graph encoder takes the normalized adjacency matrix and learnable node embeddings of the hotspot fault knowledge graph as input, and obtains the full graph node embeddings through two layers of graph convolutional layers, ReLU activation function, Dropout, and LayerNorm encoding; the knowledge graph projector compresses the six context node embeddings through mean pooling, max pooling, and fully connected layers, and outputs a 128-dimensional mechanism prior representation; the image encoder maps 512-dimensional deep semantic features into a 256-dimensional image representation through two layers of fully connected nonlinear mapping; the statistical feature encoder maps 18-dimensional structured feature vectors into a 128-dimensional statistical representation through two layers of fully connected nonlinear mapping; and the prior representation encoder maps the 128-dimensional mechanism prior representation into a 128-dimensional knowledge representation through two layers of fully connected nonlinear mapping.

[0060] In some embodiments, the heterogeneous multimodal fusion discrimination model in step S4 includes steps S401 to S404, wherein: S401, the image encoder maps deep semantic features to image representations.

[0061] In some embodiments, the image encoder includes two fully connected layers that non-linearly map 512-dimensional deep semantic features into a 256-dimensional image representation.

[0062] S402, the statistical feature encoder maps statistical features to statistical representations.

[0063] In some embodiments, the statistical feature encoder includes two fully connected layers that nonlinearly map 18-dimensional structured feature vectors into 128-dimensional statistical representations.

[0064] Understandably, the image encoder and statistical feature encoder perform dimensional transformation on the 512-dimensional deep semantic features and the 18-dimensional structured feature vector, respectively, mapping the two heterogeneous features to a unified representation space, providing dimension-aligned feature inputs for the adaptive weighting of the subsequent fusion gating unit.

[0065] S403, the prior representation encoder maps the mechanism prior representation to the knowledge representation.

[0066] In some embodiments, the prior representation encoder includes two fully connected layers that nonlinearly map a 128-dimensional mechanistic prior representation into a 128-dimensional knowledge representation.

[0067] It should be noted that the prior representation encoder and the knowledge graph encoder have different functions: the knowledge graph encoder is responsible for encoding the topological structure of the hot spot fault knowledge graph to obtain the full graph node embedding; the prior representation encoder is responsible for mapping the mechanism prior representation to the same feature space as the image representation and statistical representation, so that the subsequent fusion gating unit can perform adaptive weighted fusion.

[0068] S404 concatenates image representation, statistical representation, and knowledge representation into a fusion vector, generates gating weights based on the fusion vector, performs residual weighted fusion on the fusion vector using the gating weights, and outputs the defect level classification probability through the classification head.

[0069] In some embodiments, a 256-dimensional image representation, a 128-dimensional statistical representation, and a 128-dimensional knowledge representation are concatenated into a 512-dimensional fusion vector; gating weights with values ​​ranging from 0 to 1 are generated based on the 512-dimensional fusion vector; residual weighted fusion is performed on the 512-dimensional fusion vector using the gating weights to obtain the fused representation; the classification head is processed sequentially through a 512-dimensional fully connected layer, a ReLU activation function, and Dropout, then through a 256-dimensional fully connected layer, a ReLU activation function, and then through a 4-dimensional fully connected layer to map the fused representation into 4-dimensional logits, and then outputs the classification probabilities of four defect levels: normal, mild defect, moderate defect, and severe defect through Softmax.

[0070] Understandably, the fusion gating unit adaptively weights the three heterogeneous features by generating gating weights, enabling the KG-RealFusion network to dynamically adjust the contributions of image representation, statistical representation, and knowledge representation according to the feature distribution of different samples. The residual weighted fusion introduces gating modulation while preserving the original fusion vector information, avoiding the loss of other modal information caused by the dominance of a single feature.

[0071] In some embodiments, the risk-ordered constraint optimization objective in step S4 includes steps S405 to S409, wherein: In some embodiments, the complete form of the risk-ordered constraint optimization objective is: Ltotal = Lfocal + λ1Lorder + λ2Lkg + λ3Lreg Where Lfocal is the focus loss term with category weights, Lorder is the risk level order constraint term, Lkg is the knowledge graph consistency constraint term, Lreg is the parameter norm constraint term, and λ1, λ2 and λ3 are adjustable strength coefficients.

[0072] S405 applies differential weighting coefficients to each category loss term based on the imbalance in the distribution of category samples.

[0073] In some embodiments, the differential weight coefficients of each category are calculated according to the frequency of samples of the four categories of normal, mildly defective, moderately defective, and severely defective in the training set. The differential weight coefficients of each category are inversely related to the frequency of their samples in the training set, so that minority classes, weakly defective classes, and difficult-to-classify samples receive higher loss weights during training. At the same time, a weighted sampler with replacement is constructed so that the probability of mildly defective and moderately defective samples entering the training batch is proportional to their differential weight coefficients.

[0074] S406 imposes a level deviation penalty term on the predicted distribution based on the orderliness of the defect levels.

[0075] In some embodiments, the grade deviation penalty term applies an increasing penalty to cross-grade misjudgments between normal, minor, moderate, and severe defects, such that the KG-RealFusion network penalizes adjacent grade misjudgments less than the penalty for distant grade misjudgments.

[0076] S407 uses the embedded spatial structure of the hot spot fault knowledge graph as a supervision signal to apply a structural regularization constraint term to the prediction distribution.

[0077] In some embodiments, structural regularization constraints construct a consistency penalty using a distance metric between the predicted category and the embedding of mechanism nodes in the hot spot fault knowledge graph, constraining the predicted distribution to maintain consistency with the embedding space structure of the hot spot fault knowledge graph.

[0078] S408 applies parameter norm constraints to network parameters based on the parameter norm.

[0079] In some embodiments, the parameter norm constraint term is implemented through the weight decay mechanism of the AdamW optimizer, with a weight decay coefficient of 1×10⁻⁶. -4 .

[0080] S409, the differential weighting coefficient, the strength coefficient of the grade deviation penalty term, the strength coefficient of the structural regularization constraint term, and the strength coefficient of the parameter norm constraint term are determined by feedback adjustment of the validation set.

[0081] In some embodiments, candidate values ​​for the differential weighting coefficient, the strength coefficient of the grade deviation penalty term, the strength coefficient of the structural regularization constraint term, and the strength coefficient of the parameter norm constraint term constitute a discrete grid, with initial candidate values ​​taken from the set {0, 0.05, 0.1, 0.2, 0.5, 1.0}. After each group of candidate values ​​is trained on the same training set, the accuracy, weighted F1 score, and AUC are calculated on the validation set, and the group with the best comprehensive index and stable validation loss is selected as the final strength coefficient. In the current ELPV embodiment, the strength coefficients of the grade deviation penalty term, the strength coefficients of the structural regularization constraint term, and the strength coefficients of the parameter norm constraint term are all set to 0, and the risk-ordered constraint optimization objective degenerates into an optimization objective with a weighted focus loss with a focus loss factor γ=1.5 as the main term.

[0082] Understandably, the risk-ordered constraint optimization objective incorporates the ordered risk relationship of photovoltaic cell defect levels and the structural constraints of the hot spot fault knowledge graph into the optimization objective of the KG-RealFusion network, so that the defect level classification probability output by the KG-RealFusion network is consistent with the physical risk law of photovoltaic hot spot faults.

[0083] In some embodiments, the training process of the KG-RealFusion network includes: dividing 2624 samples into a training set of 1574 samples, a validation set of 525 samples, and a test set of 525 samples using an image-level hierarchical approach; standardizing the 512-dimensional deep semantic features, 18-dimensional structured feature vectors, and 128-dimensional mechanistic prior representations using the mean and standard deviation of the training set; calculating the weights of each category according to the frequency of samples in the four categories of normal, mildly defective, moderately defective, and severely defective samples in the training set, and constructing a weighted sampler with replacement; and using a batch size of 64, a maximum number of training epochs of 18, an early stopping patience value of 6, a fusion hidden dimension of 256, an AdamW optimizer, and a learning rate of 8×10⁻⁶. -4 Weight decay 1×10 -4 The KG-RealFusion network was trained with a focus loss factor γ=1.5. After each training round, the validation loss, accuracy, weighted F1 score and AUC were calculated on the validation set, and the model parameters with the highest weighted F1 score were saved. The defect level classification probability was output on the test set using the optimal model parameters.

[0084] It should be noted that the batch size is 64, the maximum number of training epochs is 18, the early stopping patience value is 6, the fusion hidden dimension is 256, and the learning rate is 8×10. -4 Weight decay 1×10 -4The focus loss factor γ=1.5 is a reproducible experimental setting in the current embodiment. While keeping the input, output and connection relationships of the KG-RealFusion network unchanged, the learning rate, batch size, number of training rounds, focus loss factor γ value and λ1, λ2 and λ3 can be adjusted according to the training set size, class distribution and validation set index.

[0085] In some embodiments, the integration decision of the heterogeneous multimodal fusion discriminant model in step S4 includes steps S410 to S412, wherein: S410 uses deep semantic features, statistical features, and mechanistic prior representations as common inputs to train multiple traditional classifiers.

[0086] In some embodiments, multiple traditional classifiers include a logistic regression classifier, an RBF kernel support vector machine classifier, and an extreme random tree classifier. The logistic regression classifier, the RBF kernel support vector machine classifier, and the extreme random tree classifier are all trained with a spliced ​​feature of 512-dimensional deep semantic features, 18-dimensional structured feature vectors, and 128-dimensional mechanistic prior representations as input.

[0087] S411 searches for the optimal weighted combination of the defect level classification probabilities output by the heterogeneous multimodal fusion discriminant model and multiple traditional classifiers on the validation set, with classification performance indicators as the optimization objective.

[0088] In some embodiments, using validation set accuracy, weighted F1 score, and AUC as classification performance metrics, the optimal weighted combination of defect level classification probabilities output by the KG-RealFusion network, logistic regression classifier, RBF kernel support vector machine classifier, and extreme random tree classifier is searched within a discrete space with a weight step size of 0.1. In the current ELPV embodiment, the weights of the optimal weighted combination are 0.2 for KG-RealFusion network, 0.2 for logistic regression classifier, 0.3 for RBF kernel support vector machine classifier, and 0.3 for extreme random tree classifier.

[0089] S412 determines the final defect level classification result by summing the probabilities of the optimal weighted combination.

[0090] In some embodiments, the final defect level classification probability is calculated as follows: Pfinal = 0.2×Pdeep + 0.2×Plr + 0.3×Psvm + 0.3×Pet Where Pdeep is the defect level classification probability output by the KG-RealFusion network, Plr is the defect level classification probability output by the logistic regression classifier, Psvm is the defect level classification probability output by the RBF kernel support vector machine classifier, and Pet is the defect level classification probability output by the extreme random tree classifier; the category corresponding to the maximum value in Pfinal is taken as the final defect level classification result; the defect risk assessment score is obtained by subtracting the normal class defect level classification probability from 1.

[0091] Understandably, the logistic regression classifier, RBF kernel support vector machine classifier, and extreme random tree classifier share the same feature inputs as the KG-RealFusion network. Through the soft voting weight search mechanism based on validation set feedback, they balance the semantic expressive power of the KG-RealFusion network with the robustness of traditional classifiers, thereby improving the overall defect level classification performance.

[0092] Furthermore, since the original annotations of the ELPV dataset do not contain a real time field or acquisition sequence information that can be directly used for time series modeling, the current embodiment adopts an image-level hierarchical partitioning method and does not construct pseudo-time series samples.

[0093] In other embodiments, when the field detection system can simultaneously acquire multi-source data such as real infrared thermal images, visible light images, ambient temperature, irradiance, component current, voltage, or real-time power loss, and the multi-source data has a real timestamp or real acquisition order, a multi-source data access branch and a spatiotemporal trend analysis branch can be added without changing the main framework of the invention. The spatiotemporal trend analysis branch can use a time series model or a time graph convolutional network to learn the dynamic characteristics of the hot spot region changing over time, and combine the physical variable relationships in the hot spot fault knowledge graph to analyze the hot spot propagation path.

[0094] S5 performs subgraph retrieval based on the hot spot fault knowledge graph. Based on the retrieved subgraph information and defect level classification probability, it outputs defect risk assessment score, risk transmission path and maintenance and handling suggestions.

[0095] In some embodiments, the defect category corresponding to the defect level classification probability is used as the search condition to perform subgraph matching search in the hot spot fault knowledge graph to obtain subgraph information corresponding to the defect category; the subgraph information includes risk nodes, mechanism nodes and maintenance suggestion nodes associated with the defect category; based on the subgraph information and the defect level classification probability, the defect risk assessment score, risk transmission path and maintenance and disposal suggestions are output.

[0096] Understandably, the defect risk assessment score is obtained by subtracting the normal defect level classification probability from 1, reflecting the degree to which the sample to be judged deviates from the normal state; the risk transmission path is generated based on the directed relationship edges between mechanism nodes in the subgraph information, describing the complete transmission process from defect symptoms through mechanism causes to risk expansion; the maintenance and handling suggestions are output based on the maintenance suggestion nodes in the subgraph information, providing maintenance personnel with recommended re-inspection methods and handling plans.

[0097] Furthermore, in the current ELPV embodiment, the output includes a model results overview diagram, a test set confusion matrix, a validation set model selection diagram, a test set risk ranking diagram, a model weight file, and an indicator file; the test set risk ranking diagram arranges the test samples in descending order according to the defect risk assessment score.

[0098] Understandably, maintenance personnel can obtain not only classification conclusions for normal, minor, moderate, and severe defects through subgraph retrieval output, but also a complete chain of relationships corresponding to the classification conclusions, including risk level, possible mechanism, recommended re-inspection method, defect symptoms, mechanism causes, risk expansion, and handling suggestions.

[0099] Example 2, refer to Figures 3 to 8 This invention provides a photovoltaic cell defect detection method based on the fusion and integration of real EL images and knowledge graphs. To verify the beneficial effects of this invention, scientific demonstration is carried out through experiments on the ELPV public dataset, including dataset construction, statistical feature distribution analysis, hot spot fault knowledge graph visualization, model performance evaluation, and risk ranking result analysis.

[0100] In some embodiments, the experiment uses the ELPV public dataset as input. The ELPV public dataset consists of 2624 real photovoltaic cell electroluminescence image files and their corresponding annotation index files. The annotation index files record the relative path, defect probability, and crystal type information of each electroluminescence image. The processor maps a defect probability of 0.0 to a normal label, a defect probability of 0.333333 to a mild defect label, a defect probability of 0.666667 to a moderate defect label, and a defect probability of 1.0 to a severe defect label, thereby establishing the correspondence between electroluminescence images and four defect level labels: normal, mild, moderate, and severe.

[0101] Furthermore, the 2624 samples were divided into a training set of 1574 samples, a validation set of 525 samples, and a test set of 525 samples using an image-level stratification method, with a ratio of 6:2:2. This image-level stratification ensures a consistent distribution of the four defect levels—normal, mild, moderate, and severe—across the training, validation, and test sets. To ensure the authenticity of the experimental data, this embodiment does not introduce any fictitious timestamps, pseudo-evolutionary sequences, or manually spliced ​​time-series samples. Since the original annotations of the ELPV public dataset do not contain real time fields or acquisition sequence information that can be directly used for time-series modeling, this embodiment maintains the image-level stratified evaluation method.

[0102] In some embodiments, refer to Figure 3 The processor extracts 18-dimensional structured feature vectors from steps S101 to S102 of Example 1 from 2624 electroluminescent images, and performs multimodal statistical feature distribution analysis on the 18-dimensional structured feature vectors.

[0103] Understandable Figure 3 The fault category distribution subplot shows that the number of normal samples is greater than the number of samples with mild, moderate, and severe defects, indicating a significant imbalance in the distribution of samples across the four defect levels. This imbalance is the basis for constructing the weighted sampler with replacement and applying differential weighting coefficients in step S405 of Example 1. The multimodal feature correlation knowledge matrix subplot shows varying degrees of correlation between the statistics of gray intensity distribution, the statistics of spatial structure differences, and the statistics of edge gradient response. The multimodal feature PCA visualization subplot shows that samples of different defect levels partially overlap in the principal component space of the 18-dimensional structured feature vector, indicating that relying solely on statistical features is insufficient to completely distinguish boundary samples such as mild and moderate defects. It is necessary to combine the prior representation of the mechanism in step S3 and the deep semantic features in step S4 of Example 1 for joint discrimination.

[0104] In some embodiments, refer to Figure 4 The processor constructs a hot spot fault knowledge graph structure containing 53 nodes and 62 relation edges according to step S2 in Example 1. Figure 4 The different colored nodes correspond to seven types of nodes: dataset nodes, crystal type nodes, fault nodes, risk nodes, observation signal nodes, mechanism nodes, and maintenance suggestion nodes. The directed relationship between the nodes reflects the directed conduction path of the electro-thermal-mechanical coupling damage in step S204 of Example 1 and the causal relationship between the various types of nodes.

[0105] In some embodiments, for each sample to be judged in the training set of 1574 samples, the validation set of 525 samples, and the test set of 525 samples, a mechanism prior representation is generated according to steps S301 to S303 in Embodiment 1; the context node set consists of 1 dataset node, 1 crystal type node, 1 risk node, and 3 observation signal nodes, for a total of 6 nodes; the 128-dimensional mechanism prior representation, the 18-dimensional structured feature vector, and the 512-dimensional deep semantic features together constitute the three inputs of the KG-RealFusion network.

[0106] It should be noted that the experimental environment of the current embodiment includes Python 3.11, PyTorch 2.10, scikit-learn, networkx, and matplotlib; the experimental environment is only used to illustrate the feasibility of the present invention and does not constitute a limitation on the scope of protection of the present invention.

[0107] In some embodiments, refer to Figure 7 The processor evaluates the defect level classification performance of the KG-RealFusion network, logistic regression classifier, RBF kernel support vector machine classifier, extreme random tree classifier, and ensemble model on the validation set. Figure 7 The validation set model selection subgraph reflects that the validation set accuracy and weighted F1 score of the ensemble model are both higher than those of the individual classifiers. Figure 7 The soft voting weight configuration subgraph reflects the optimal weighted combination selected on the validation set. The validation set AUC of the KG-RealFusion network is 81.5%, and the validation set AUC of the ensemble model is 84.5%. In the current ELPV embodiment, the optimal soft voting weights selected on the validation set are: KG-RealFusion network 0.2, logistic regression classifier 0.2, RBF kernel support vector machine classifier 0.3, and extreme random tree classifier 0.3.

[0108] In some embodiments, the ensemble model achieves an accuracy of 76.57%, a weighted F1 score of 75.45%, and an AUC of 84.46% on the validation set; and an accuracy of 74.86%, a weighted F1 score of 74.02%, and an AUC of 83.12% on the test set.

[0109] Furthermore, referring to Figure 8 , Figure 8 The confusion matrix of the test set reflects the distribution of the four defect level classification results of the ensemble model on the test set; among the defect level classification results of each category in the test set, the F1 value of the normal class is 82.48%, the F1 value of the severe defect class is 74.73%, the F1 value of the mild defect class is 47.17%, and the F1 value of the moderate defect class is 22.86%.

[0110] It is understandable that the defect classification performance of the mild and moderate defect classes is lower than that of the normal and severe defect classes. This is because the number of mild and moderate defect samples is less than that of normal and severe defect samples, and the inter-class boundaries between mild and moderate defects are closer, resulting in significant overlap in the 18-dimensional structured feature vector space and the 512-dimensional deep semantic feature space. The KG-RealFusion network, through the differential weight coefficients and weighted samplers in step S405 of Example 1, the grade deviation penalty term in step S406, and the structure regularization constraint term in step S407, can still maintain good overall defect classification performance under multi-branch fusion conditions.

[0111] In some embodiments, refer to Figure 6 The processor sorts the defect risk assessment scores of 525 samples in the test set in descending order and generates a risk ranking chart for the test set. The defect risk assessment score is obtained by subtracting the classification probability of the normal defect level from 1, and the value ranges from 0 to 1. Figure 6 The subplot of diagnostic indicators by category reflects the recall and precision of the integrated model at each defect level of normal, mild, moderate and severe defects in the test set. Figure 6 The test set prediction confidence distribution subplot reflects the maximum defect level classification probability distribution of correctly predicted samples and incorrectly predicted samples. The maximum defect level classification probability of correctly predicted samples is generally greater than that of incorrectly predicted samples, indicating that the defect level classification probability output of the KG-RealFusion network has confidence discrimination ability.

[0112] Furthermore, referring to Figure 5 , Figure 5 The performance comparison subplots of the various models show that the KG-RealFusion network scores higher than external baseline models such as logistic regression, random forest, gradient boosting, and RBF kernel support vector machine in terms of weighted F1 score and AUC. Figure 5 The importance ranking subgraph of random forest features is reflected in the 18-dimensional structured feature vector. Among the statistics reflecting spatial structure differences, the center-edge brightness difference and P10 quantile, as well as the EL contrast among the statistics reflecting gray intensity distribution, are ranked highly in importance. This indicates that the statistics reflecting spatial structure differences and the statistics reflecting gray intensity distribution have a high discriminative contribution to defect level classification. Figure 5 The PR curve subplot reflects that the KG-RealFusion network outperforms all external baseline models in terms of precision and recall for both defect and normal categories. Figure 5The subgraph of the knowledge graph relationship retrieval results reflects the example of the fault relationship chain returned by the hot spot fault knowledge graph after performing the subgraph matching retrieval in step S5 of Example 1 on the severe defect samples in the test set. It includes risk nodes, mechanism nodes and maintenance suggestion nodes corresponding to severe defects, forming a complete relationship chain of defect symptoms, mechanism causes, risk expansion and disposal suggestions.

[0113] Example 3 is an embodiment of the present invention, which provides a photovoltaic cell defect detection system based on the fusion and integrated discrimination of real EL images and knowledge graphs, including: The statistical feature extraction module is used to extract statistical features reflecting gray intensity distribution, spatial structure differences, and edge gradient response from electroluminescent images based on photovoltaic cells. The knowledge graph construction module is used to construct a hot spot fault knowledge graph. The hot spot fault knowledge graph is composed of fault nodes, risk nodes, observation signal nodes, mechanism nodes, and maintenance suggestion nodes. The relationship between nodes is used to encode the mechanism relationship of hot spot expansion and damage. The prior characterization generation module is used to generate prior characterizations of the mechanism for samples to be judged based on hot spot fault knowledge graphs. The deep feature extraction module is used to extract deep semantic features from electroluminescent images using a pre-trained visual network; The fusion discrimination module is used to train the heterogeneous multimodal fusion discrimination model by taking deep semantic features, statistical features and prior mechanistic representations as inputs and driving the optimization objective of risk-ordered constraints, and outputting the defect level classification probability. The integrated decision module is used to train multiple traditional classifiers with deep semantic features, statistical features and prior mechanistic representations as common inputs. On the validation set, it searches for the optimal weighted combination of the defect level classification probabilities output by the heterogeneous multimodal fusion discriminant model and multiple traditional classifiers. The final defect level classification result is determined by the sum of the probabilities of the optimal weighted combination. The subgraph retrieval module is used to perform subgraph retrieval based on the hot spot fault knowledge graph. Based on the retrieved subgraph information and defect level classification probability, it outputs defect risk assessment scores, risk transmission paths, and maintenance and handling suggestions.

[0114] This embodiment also provides an electronic device applicable to a photovoltaic cell defect detection method based on the fusion and integration of real EL images and knowledge graphs, comprising: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to realize the photovoltaic cell defect detection method based on the fusion and integration of real EL images and knowledge graphs as proposed in the above embodiment.

[0115] This embodiment also provides a storage medium storing a computer program that, when executed by a processor, implements a photovoltaic cell defect detection method based on the fusion and integration of real EL images and knowledge graphs as proposed in the above embodiments.

[0116] The storage medium proposed in this embodiment belongs to the same inventive concept as the photovoltaic cell defect detection method based on the fusion and integration of real EL images and knowledge graphs proposed in the above embodiments. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0117] Based on the above description of the implementation methods, those skilled in the art can clearly understand that the present invention can be implemented using software and necessary general-purpose hardware, and of course, it can also be implemented using hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk, or optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of the various embodiments of the present invention.

[0118] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A photovoltaic cell defect detection method based on the fusion and integration of real EL images and knowledge graphs, characterized in that, include: Based on the electroluminescent images of photovoltaic cells, statistical features reflecting gray intensity distribution, spatial structure differences and edge gradient response are extracted. A hot spot fault knowledge graph is constructed, which is composed of fault nodes, risk nodes, observation signal nodes, mechanism nodes and maintenance suggestion nodes. The relationship between nodes is used to encode the mechanism relationship of hot spot expansion and damage. Based on the hot spot fault knowledge graph, a priori representation of the generation mechanism of the sample to be judged is provided. Deep semantic features are extracted from the electroluminescent image using a pre-trained visual network. The deep semantic features, statistical features, and prior representations of the mechanism are used as inputs. The training of the heterogeneous multimodal fusion discrimination model is driven by the risk-ordered constraint optimization objective, and the defect level classification probability is output. Based on the hot spot fault knowledge graph, a subgraph retrieval is performed. The retrieved subgraph information and the defect level classification probability are used as the basis to output the defect risk assessment score, risk transmission path and maintenance and handling suggestions.

2. The photovoltaic cell defect detection method based on the fusion and integration of real EL images and knowledge graphs as described in claim 1, characterized in that, The prior characterization of the mechanism includes: Using the normalized adjacency matrix and learnable node embeddings of the hot spot fault knowledge graph as input, the full graph node embeddings are obtained by encoding through the knowledge graph encoder. From the full-graph node embedding, the context node embedding is extracted using the context node set consisting of dataset nodes, crystal type nodes, risk nodes, and observation signal nodes as an index. Through multi-granularity aggregation operations and dimensional compression mapping, the context nodes are embedded and compressed into a priori representation of the mechanism.

3. The photovoltaic cell defect detection method based on the fusion and integration of real EL images and knowledge graphs as described in claim 2, characterized in that, The heterogeneous multimodal fusion discriminant model includes: The image encoder maps the deep semantic features to image representations; The statistical feature encoder maps the statistical features to statistical representations; The prior representation encoder maps the prior representation of the mechanism to a knowledge representation; The image representation, statistical representation, and knowledge representation are concatenated into a fusion vector. Gating weights are generated based on the fusion vector. The fusion vector is then subjected to residual weighted fusion using the gating weights. Finally, the defect level classification probability is output through the classification head.

4. The photovoltaic cell defect detection method based on the fusion and integration of real EL images and knowledge graphs as described in claim 3, characterized in that, The optimization objectives of the risk-ordered constraint include: Based on the imbalance in the distribution of category samples, differential weighting coefficients are applied to the loss terms of each category; Based on the orderliness of defect levels, a level deviation penalty term is applied to the predicted distribution; Using the embedded spatial structure of the hot spot fault knowledge graph as a supervision signal, a structural regularization constraint term is applied to the prediction distribution; Apply parameter norm constraints to the network parameters based on the parameter norm; The differential weighting coefficient, the strength coefficient of the grade deviation penalty term, the strength coefficient of the structural regularization constraint term, and the strength coefficient of the parameter norm constraint term are determined by feedback adjustment of the validation set.

5. The photovoltaic cell defect detection method based on the fusion and integration of real EL images and knowledge graphs as described in claim 4, characterized in that, The integrated decision-making of the heterogeneous multimodal fusion discriminant model includes: Multiple traditional classifiers are trained using the deep semantic features, the statistical features, and the prior mechanistic representations as common inputs. On the validation set, with classification performance metrics as the optimization objective, the optimal weighted combination of the defect level classification probabilities output by the heterogeneous multimodal fusion discriminant model and the multiple traditional classifiers is searched. The final defect level classification result is determined by the sum of the probabilities of the optimal weighted combination.

6. The photovoltaic cell defect detection method based on the fusion and integration of real EL images and knowledge graphs as described in claim 5, characterized in that, The statistical characteristics include: A statistic reflecting the distribution of gray intensity; Statistical measures reflecting differences in spatial structure; Statistics reflecting the edge gradient response; The structured numerical encoding of the crystal type is concatenated with the statistics reflecting gray intensity distribution, spatial structure differences, and edge gradient response to form a structured feature vector.

7. The photovoltaic cell defect detection method based on the fusion and integration of real EL images and knowledge graphs as described in claim 6, characterized in that, The hot spot fault knowledge graph includes: Nodes in the electrical domain are connected to nodes in the thermal domain via directed relation edges that characterize current-carrying anomalies. Nodes in the thermal domain are connected to nodes in the mechanical domain via directed relation edges that characterize anomalies in the temperature field distribution. The mechanical domain nodes are connected to the hot spot propagation nodes via directed relation edges characterizing material damage accumulation. The directed relation edges of the current-carrying anomaly, the directed relation edges of the temperature field distribution anomaly, and the directed relation edges of the material damage accumulation are connected in series to form a directed conduction path for damage caused by electro-thermal-mechanical coupling.

8. A photovoltaic cell defect detection system based on the fusion and integration of real EL images and knowledge graphs, employing the photovoltaic cell defect detection method based on the fusion and integration of real EL images and knowledge graphs as described in any one of claims 1 to 7, characterized in that, include: The statistical feature extraction module is used to extract statistical features reflecting gray intensity distribution, spatial structure differences, and edge gradient response from electroluminescent images based on photovoltaic cells. The knowledge graph construction module is used to construct a hot spot fault knowledge graph. The hot spot fault knowledge graph is composed of fault nodes, risk nodes, observation signal nodes, mechanism nodes and maintenance suggestion nodes. The relationship between nodes is used to encode the mechanism relationship of hot spot expansion and damage. The prior characterization generation module is used to generate a prior characterization of the mechanism for the sample to be judged based on the hot spot fault knowledge graph. A deep feature extraction module is used to extract deep semantic features from the electroluminescent image using a pre-trained visual network; The fusion discrimination module is used to take the deep semantic features, the statistical features and the mechanism prior representation as inputs, drive the training of the heterogeneous multimodal fusion discrimination model with risk-ordered constraint optimization objective, and output the defect level classification probability. An integrated decision module is used to train multiple traditional classifiers with the deep semantic features, the statistical features, and the prior representation of the mechanism as common inputs. On the validation set, it searches for the optimal weighted combination of the defect level classification probabilities output by the heterogeneous multimodal fusion discriminant model and the multiple traditional classifiers. The final defect level classification result is determined by the sum of the probabilities of the optimal weighted combination. The subgraph retrieval module is used to perform subgraph retrieval based on the hot spot fault knowledge graph. Based on the retrieved subgraph information and the defect level classification probability, it outputs the defect risk assessment score, risk transmission path and maintenance and handling suggestions.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the photovoltaic cell defect detection method based on the fusion and integration of real EL images and knowledge graphs as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the photovoltaic cell defect detection method based on the fusion and integration of real EL images and knowledge graphs as described in any one of claims 1 to 7.

Images