Twin network photovoltaic fault detection method and device based on graph similarity
Through the twin network photovoltaic fault detection method based on graph similarity, the problem of picture quality problems and insufficient sample number during photovoltaic power station inspection is solved, and high-precision photovoltaic panel fault detection is achieved.
Patent Information
- Application Number
- CN202411521292.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-29
- Publication Date
- 2025-02-18
AI Technical Summary
During the intelligent inspection of photovoltaic power stations, the number of samples for image quality problems and photovoltaic inspection fault types during the image acquisition process of drones is too small, resulting in low diagnostic and recognition accuracy of photovoltaic panels.
Using the twin network photovoltaic fault detection method based on graph similarity, by obtaining the picture data of the normal state of the photovoltaic panel, building a weighted undirected attribute diagram, forming training samples, training a target twin network, and judging the fault of the picture data to be tested on the photovoltaic panel Probability.
Unstructured defect detection of photovoltaic panels is realized, and the fault detection accuracy of photovoltaic panels is improved without large-scale label data.
Smart Images
Figure CN119474808A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of photovoltaic fault identification, and in particular to a twin network photovoltaic fault detection method and device based on graph similarity. Background Art
[0002] Intelligent inspection of photovoltaic power stations generally uses drones as flight platforms, equipped with infrared, visible light and other cameras to scan and photograph photovoltaic panels, and uses computer vision-related algorithms to refer to historical fault types to diagnose and identify faults. However, the image quality problems that may occur during drone image acquisition, and the small number of samples in photovoltaic inspection fault types, lead to low diagnostic identification accuracy of photovoltaic panels. Therefore, a more reliable twin network photovoltaic fault detection method based on graph similarity is urgently needed. Summary of the invention
[0003] The present invention aims to solve one of the technical problems in the related art at least to a certain extent.
[0004] To this end, the first purpose of the present invention is to propose a twin network photovoltaic fault detection method based on graph similarity. The twin network trained based on graph similarity does not require large-scale labeled data, realizes unstructured defect detection of photovoltaic panels, and improves the fault detection accuracy of photovoltaic panels.
[0005] The second object of the present invention is to propose a twin network photovoltaic fault detection device based on graph similarity.
[0006] A third objective of the present invention is to provide an electronic device.
[0007] A fourth object of the present invention is to provide a non-transitory computer-readable storage medium storing computer instructions.
[0008] A fifth object of the present invention is to provide a computer program product.
[0009] To achieve the above-mentioned purpose, the first embodiment of the present invention proposes a twin network photovoltaic fault detection method based on graph similarity, the method comprising:
[0010] Acquire image data of the photovoltaic panel in a normal state, and select a set of key points in the grayscale image corresponding to the photo in the image data that are invariant to scale and rotation, and a SIFT feature descriptor of each key point, wherein the SIFT feature descriptor includes position, direction and scale information;
[0011] Based on the SIFT feature descriptor and key points of each key point, a weighted undirected attribute graph is constructed;
[0012] A training sample is formed by combining two weighted undirected attribute graphs, and the training sample is marked with a label, and the label is used to judge the feature similarity of the two weighted undirected attribute graphs;
[0013] The two weighted undirected attribute graphs of the training samples are used as the input of the twin network SN, and the feature similarity is used as the output to train a target twin network that judges the similarity of the corresponding features of the two weighted undirected attribute graphs;
[0014] Inputting two target weighted undirected attribute graphs corresponding to the photovoltaic panel to-be-tested image data into the target twin network, and determining the target feature similarity of the two target weighted undirected attribute graphs;
[0015] The target feature similarity is passed through a fully connected layer and a softmax layer in sequence to determine the failure probability of the photovoltaic panel.
[0016] To achieve the above-mentioned purpose, the second embodiment of the present invention proposes a twin network photovoltaic fault detection device based on graph similarity, the device comprising:
[0017] An acquisition module is used to acquire image data of the photovoltaic panel in a normal state, and select a set of key points in the grayscale image corresponding to the photo in the image data that are invariant to scale and rotation, and a SIFT feature descriptor of each key point, wherein the SIFT feature descriptor includes position, direction and scale information;
[0018] A construction module for constructing a weighted undirected attribute graph based on the SIFT feature descriptor of each key point and the key point;
[0019] A building module, used for building a training sample composed of two weighted undirected attribute graphs, wherein the training sample is marked with a label, and the label is used for judging the feature similarity of the two weighted undirected attribute graphs;
[0020] A training module is used to take two weighted undirected attribute graphs of the training sample as input of the twin network SN, and the feature similarity as output, so as to train a target twin network that judges the similarity of the corresponding features of the two weighted undirected attribute graphs;
[0021] A judgment module is used to input two target weighted undirected attribute graphs corresponding to the photovoltaic panel to be tested image data into the target twin network, and judge the target feature similarity of the two target weighted undirected attribute graphs;
[0022] The determination module is used to sequentially pass the target feature similarity through a fully connected layer and a softmax layer to determine the failure probability of the photovoltaic panel.
[0023] To achieve the above-mentioned purpose, the third aspect of the present invention proposes an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method described in the first aspect.
[0024] In order to achieve the above-mentioned purpose, the fourth aspect of the present invention proposes a non-transitory computer-readable storage medium storing computer instructions, where the computer instructions are used to enable the computer to execute the method described in the first aspect.
[0025] In order to achieve the above-mentioned purpose, a fifth aspect of the present invention provides a computer program product, and when the computer program is executed by a processor, it implements the method described in the first aspect.
[0026] The twin network photovoltaic fault detection method, device, electronic device, storage medium and computer program product based on graph similarity provided by the embodiment of the present invention construct a weighted undirected attribute graph according to the image data of the photovoltaic panel in the normal state; form a training sample composed of two weighted undirected attribute graphs, and the training sample is marked with a label, and the label is used to judge the feature similarity of the two weighted undirected attribute graphs; train a target twin network that judges the feature similarity of the two weighted undirected attribute graphs; input the two target weighted undirected attribute graphs corresponding to the photovoltaic panel to be tested image data into the target twin network to judge the target feature similarity; pass the target feature similarity through the fully connected layer softmax layer in sequence to determine the failure probability of the photovoltaic panel. Therefore, the twin network trained based on graph similarity does not require large-scale label data, realizes the detection of unstructured defects of photovoltaic panels, and improves the fault detection accuracy of photovoltaic panels.
[0027] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0028] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:
[0029] Figure 1 A schematic flow chart of a twin network photovoltaic fault detection method based on graph similarity provided by an embodiment of the present invention;
[0030] Figure 2 A schematic structural diagram of a twin network photovoltaic fault detection device based on graph similarity provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0031] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.
[0032] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals involved in the present invention are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data need to comply with the relevant laws, regulations and standards of relevant countries and regions.
[0033] The following describes a twin network photovoltaic fault detection method and device based on graph similarity according to an embodiment of the present invention with reference to the accompanying drawings.
[0034] Figure 1 A schematic flow chart of a twin network photovoltaic fault detection method based on graph similarity provided in an embodiment of the present invention.
[0035] like Figure 1 As shown, the method comprises the following steps:
[0036] Step 101, obtain image data of the photovoltaic panel in a normal state, and select a group of key points in the grayscale image corresponding to the photo in the image data that are invariant to scale and rotation, and the SIFT feature descriptor of each key point, wherein the SIFT feature descriptor includes position, direction and scale information.
[0037] In some embodiments, normal data is selected because it is more available and representative of normal operating conditions. Using samples of image data under normal conditions to establish a baseline helps to more clearly define standard operating characteristics, simplify model training, and enhance its interpretability. This strategy also helps alleviate the problem of data imbalance, which is particularly common when failure events are rare.
[0038] Specifically, through the algorithm, the key point detection step can find a set of key points in the grayscale image of the photo that are invariant to scale and rotation. Each key point can be regarded as a node in the photo. Each node carries information such as position, direction and scale (i.e. SIFT feature descriptor), and the SIFT feature descriptor can be used as the attribute of the node.
[0039] Step 102: construct a weighted undirected attribute graph based on the SIFT feature descriptor of each key point and the key point.
[0040] In some embodiments, an implementation method of constructing a weighted undirected attribute graph based on the SIFT feature descriptor and key points of each key point can be to calculate the feature similarity between any two key points based on the SIFT feature descriptor of each key point; construct edges between the key points based on the feature similarity, use the feature similarity as the weight of the edge, and generate a weighted undirected attribute graph.
[0041] Furthermore, based on the feature similarity as the key point (A i ,B i ) and use the feature similarity (the inverse of the Euclidean distance (d), with n set to 128 and the feature similarity threshold set to 0.3) as the edge weight to quickly generate a weighted undirected attribute graph. The feature similarity is calculated as formula (1):
[0042]
[0043] Step 103: construct a training sample composed of two weighted undirected attribute graphs, and the training sample is marked with a label, and the label is used to determine the feature similarity of the two weighted undirected attribute graphs.
[0044] In some embodiments, the labeled training sample may be (Y, X1, X2), where X1 and X2 represent a pair of weighted undirected attribute graphs after SIFT feature extraction, and Y represents the label, which is used to determine the feature similarity of X1 and X2.
[0045] Step 104, using the two weighted undirected attribute graphs of the training samples as inputs of the twin network SN, and using the feature similarity as output, to train a target twin network that determines the feature similarity corresponding to the two weighted undirected attribute graphs.
[0046] In some embodiments, an implementation method for training a target twin network for judging the similarity of features corresponding to two weighted undirected attribute graphs can be to input the two weighted undirected attribute graphs into two identical spatial feature extractors in the twin network SN respectively to extract the first embedding vector and the second embedding vector corresponding to each of the two weighted undirected attribute graphs; take the average of the first embedding vector and the second embedding vector as the category center of normal and faulty photovoltaic panels; calculate the first distance between the first embedding vector and the category center of normal and faulty, respectively, and the second distance between the second embedding vector and the category center of normal and faulty, respectively; calculate the first distance and the second distance through the Euclidean distance function at the intersection of the ends of the two spatial feature extractors to obtain the similarity between the first embedding vector and the second embedding vector, and use it as the training feature similarity of the two weighted undirected attribute graphs; based on the difference between the feature similarity and the training feature similarity, train a target twin network for calculating the feature similarity of the two weighted undirected attribute graphs.
[0047] Furthermore, the Siamese Network (SN) contains two identical sub-networks, namely the spatial feature extractor, that is, the Graph Attention Networks (GAT). After X1 and X2 pass through the spatial feature extractor, the first embedding vector G can be obtained. W (X1) and the second embedding vector G W (X2). Then, G is calculated by the Euclidean distance function at the intersection of the two sub-network ends. W (X1) and G W The similarity E between (X2) W , as shown in formula (2):
[0048] E W (X1,X2)=||G W (X1)-G W (X2)||2 (2)
[0049] Among them, the optimization goal of the twin network SN is to continuously update E W To minimize the loss function, the distance E between similar training sample pairs is W Reduce, rather than the distance E between similar training sample pairs W Increase.
[0050] Step 105, input the two target weighted undirected attribute graphs corresponding to the photovoltaic panel to be tested image data into the target twin network, and determine the target feature similarity of the two target weighted undirected attribute graphs.
[0051] In some embodiments, two target weighted undirected attribute graphs corresponding to the photovoltaic panel test picture data are input into the target twin network, and one implementation method for determining the target feature similarity of the two target weighted undirected attribute graphs can be to input the two target weighted undirected attribute graphs corresponding to the photovoltaic panel test picture data into two identical spatial feature extractors in the twin network SN respectively to extract the third embedding vector and the fourth embedding vector corresponding to each of the two target weighted undirected attribute graphs; the third distance between the third embedding vector and the normal and faulty category centers, and the fourth distance between the fourth embedding vector and the normal and faulty category centers, are calculated by the Euclidean distance function at the intersection of the two spatial feature extractors; the third distance and the fourth distance are calculated by the Euclidean distance function at the intersection of the two spatial feature extractors to obtain the similarity between the third embedding vector and the fourth embedding vector, and use it as the target feature similarity of the two target weighted undirected attribute graphs. Real-time prediction of test picture data is achieved to ensure the safety of photovoltaic panels.
[0052] Step 106 , the target feature similarity is passed through the fully connected layer and the softmax layer in sequence to determine the failure probability of the photovoltaic panel.
[0053] In some embodiments, the target feature similarity is passed through a fully connected layer and a softmax layer in sequence to determine the failure probability of the photovoltaic panel. One implementation method is to obtain the probability that two target weighted undirected attribute graphs belong to the same category after passing the target feature similarity through a fully connected layer and a softmax layer in sequence; when it is known that any one of the target weighted undirected attribute graphs is faulty, predict the failure probability of the photovoltaic panel corresponding to the other target weighted undirected attribute graph.
[0054] Furthermore, the probability P(X1, X2) that two target weighted undirected attribute graphs belong to the same category is calculated as shown in formula (3):
[0055] P(X1,X2)=softmax(FC(E W (X1,X2))) (3)
[0056] Among them, softmax() and FC() represent the softmax layer and fully connected layer functions respectively.
[0057] In addition, the two target weighted undirected attribute graphs can be the same fault category or the normal category. When the first target weighted undirected attribute graph is the fault category or the normal category, based on the feature similarity between the first target weighted undirected attribute graph and the second target weighted undirected attribute graph, it is determined whether they belong to the same category, and the failure probability of the photovoltaic panel corresponding to the second target weighted undirected attribute graph is predicted.
[0058] The twin network photovoltaic fault detection method based on graph similarity of the embodiment of the present invention constructs a weighted undirected attribute graph according to the image data of the photovoltaic panel in the normal state; forms a training sample composed of two weighted undirected attribute graphs, and the training sample is marked with a label, and the label is used to judge the feature similarity of the two weighted undirected attribute graphs; trains a target twin network that judges the feature similarity of the two weighted undirected attribute graphs; inputs the two target weighted undirected attribute graphs corresponding to the image data of the photovoltaic panel to be tested into the target twin network to judge the target feature similarity; passes the target feature similarity through the fully connected layer softmax layer in sequence to determine the failure probability of the photovoltaic panel. Therefore, the twin network trained based on graph similarity does not require large-scale label data, realizes the detection of unstructured defects of photovoltaic panels, and improves the fault detection accuracy of photovoltaic panels.
[0059] In order to implement the above embodiment, the present invention also proposes a twin network photovoltaic fault detection device based on graph similarity.
[0060] Figure 2A schematic structural diagram of a twin network photovoltaic fault detection device based on graph similarity provided in an embodiment of the present invention.
[0061] like Figure 2 As shown, the twin network photovoltaic fault detection device 20 based on graph similarity includes: an acquisition module 21, a construction module 22, a formation module 23, a training module 24, a judgment module 25 and a determination module 25.
[0062] The acquisition module 21 is used to acquire the image data of the photovoltaic panel in a normal state, and select a set of key points in the grayscale image corresponding to the photo in the image data that are invariant to scale and rotation, and the SIFT feature descriptor of each key point, wherein the SIFT feature descriptor includes position, direction and scale information;
[0063] A construction module 22 is used to construct a weighted undirected attribute graph based on the SIFT feature descriptor of each key point and the key point;
[0064] A forming module 23, used for forming a training sample composed of two weighted undirected attribute graphs, and the training sample is marked with a label, and the label is used to judge the feature similarity of the two weighted undirected attribute graphs;
[0065] The training module 24 is used to use the two weighted undirected attribute graphs of the training sample as inputs of the twin network SN and the feature similarity as output, so as to train a target twin network that judges the feature similarity corresponding to the two weighted undirected attribute graphs;
[0066] A judgment module 25 is used to input two target weighted undirected attribute graphs corresponding to the photovoltaic panel to be tested image data into the target twin network, and judge the target feature similarity of the two target weighted undirected attribute graphs;
[0067] The determination module 26 is used to sequentially pass the target feature similarity through a fully connected layer and a softmax layer to determine the failure probability of the photovoltaic panel.
[0068] Furthermore, in a possible implementation of the embodiment of the present invention, the construction module 22 is specifically used to:
[0069] Based on the SIFT feature descriptor of each key point, the feature similarity between any two key points is calculated;
[0070] Edges are constructed between key points based on feature similarity, and feature similarity is used as the weight of the edge to generate a weighted undirected attribute graph.
[0071] Furthermore, in a possible implementation of the embodiment of the present invention, the training module 24 is specifically used to:
[0072] The two weighted undirected attribute graphs are respectively input into two identical spatial feature extractors in the twin network SN to extract the first embedding vector and the second embedding vector corresponding to the two weighted undirected attribute graphs;
[0073] The average of the first embedding vector and the second embedding vector is used as the category center of the normal and faulty photovoltaic panels;
[0074] Calculate first distances between the first embedding vector and the normal and faulty category centers, and second distances between the second embedding vector and the normal and faulty category centers, respectively;
[0075] The first distance and the second distance are calculated by using a Euclidean distance function at the intersection of the two spatial feature extractors to obtain a similarity between the first embedding vector and the second embedding vector, and used as a training feature similarity of the two weighted undirected attribute graphs;
[0076] Based on the difference between feature similarity and training feature similarity, a target twin network is trained to calculate the feature similarity of two weighted undirected attribute graphs.
[0077] Furthermore, in a possible implementation of the embodiment of the present invention, the judging module 25 is configured to:
[0078] The two target weighted undirected attribute graphs corresponding to the photovoltaic panel test image data are respectively input into two identical spatial feature extractors in the twin network SN to extract the third embedding vector and the fourth embedding vector corresponding to the two target weighted undirected attribute graphs respectively;
[0079] By using the Euclidean distance function at the intersection of the two spatial feature extractors, a third distance between the third embedding vector and the center of the normal and fault categories, and a fourth distance between the fourth embedding vector and the center of the normal and fault categories are calculated;
[0080] The third distance and the fourth distance are calculated by the Euclidean distance function at the intersection of the two spatial feature extractors to obtain the similarity between the third embedding vector and the fourth embedding vector, and used as the target feature similarity of the two target weighted undirected attribute graphs.
[0081] Further, in a possible implementation manner of the embodiment of the present invention, the determining module 26 is specifically configured to:
[0082] After the target feature similarity passes through the fully connected layer and the softmax layer in sequence, the probability that the two target weighted undirected attribute graphs belong to the same category is obtained;
[0083] When any target weighted undirected attribute graph is known to be faulty, the fault probability of the photovoltaic panel corresponding to another target weighted undirected attribute graph is predicted.
[0084] The twin network photovoltaic fault detection device based on graph similarity of the embodiment of the present invention constructs a weighted undirected attribute graph according to the image data of the photovoltaic panel in the normal state; forms a training sample composed of two weighted undirected attribute graphs, and the training sample is marked with a label, and the label is used to judge the feature similarity of the two weighted undirected attribute graphs; trains a target twin network that judges the feature similarity corresponding to the two weighted undirected attribute graphs; inputs the two target weighted undirected attribute graphs corresponding to the image data to be tested of the photovoltaic panel into the target twin network to judge the target feature similarity; passes the target feature similarity through the fully connected layer softmax layer in sequence to determine the failure probability of the photovoltaic panel. Therefore, the twin network trained based on graph similarity does not require large-scale label data, realizes the detection of unstructured defects of photovoltaic panels, and improves the fault detection accuracy of photovoltaic panels.
[0085] In order to implement the above embodiment, the present invention further provides an electronic device, including:
[0086] at least one processor; and
[0087] a memory communicatively connected to the at least one processor; wherein,
[0088] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the aforementioned method.
[0089] In order to implement the above embodiment, the present invention further proposes a non-transitory computer-readable storage medium storing computer instructions, where the computer instructions are used to enable the computer to execute the above method.
[0090] In order to implement the above embodiments, the present invention further proposes a computer program product, comprising a computer program, wherein the computer program implements the above method when executed by a processor.
[0091] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.
[0092] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.
[0093] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present invention belong.
[0094] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute the instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.
[0095] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.
[0096] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.
[0097] In addition, each functional unit in each embodiment of the present invention may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.
[0098] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present invention. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.
Claims
1. A twin network photovoltaic fault detection method based on graph similarity, characterized in that: The method comprises: Acquire image data of the photovoltaic panel in a normal state, and select a set of key points in the grayscale image corresponding to the photo in the image data that are invariant to scale and rotation, and a SIFT feature descriptor of each key point, wherein the SIFT feature descriptor includes position, direction and scale information; Based on the SIFT feature descriptor and key points of each key point, a weighted undirected attribute graph is constructed; A training sample is formed by combining two weighted undirected attribute graphs, and the training sample is marked with a label, and the label is used to judge the feature similarity of the two weighted undirected attribute graphs; The two weighted undirected attribute graphs of the training samples are used as the input of the twin network SN, and the feature similarity is used as the output to train a target twin network that judges the similarity of the corresponding features of the two weighted undirected attribute graphs; Inputting two target weighted undirected attribute graphs corresponding to the photovoltaic panel to-be-tested image data into the target twin network, and determining the target feature similarity of the two target weighted undirected attribute graphs; The target feature similarity is passed through a fully connected layer and a softmax layer in sequence to determine the failure probability of the photovoltaic panel.
2. The method according to claim 1, characterized in that The method of constructing a weighted undirected attribute graph based on the SIFT feature descriptor of each key point and the key point includes: Based on the SIFT feature descriptor of each key point, the feature similarity between any two key points is calculated; Edges are constructed between key points based on feature similarity, and feature similarity is used as the weight of the edge to generate a weighted undirected attribute graph.
3. The method according to claim 1, characterized in that The two weighted undirected attribute graphs of the training samples are used as inputs of the twin network SN, and the feature similarity is used as output to train a target twin network for judging the feature similarity corresponding to the two weighted undirected attribute graphs, including: The two weighted undirected attribute graphs are respectively input into two identical spatial feature extractors in the twin network SN to extract the first embedding vector and the second embedding vector corresponding to the two weighted undirected attribute graphs; The average of the first embedding vector and the second embedding vector is used as the category center of the normal and faulty photovoltaic panels; Calculate first distances between the first embedding vector and the normal and faulty category centers, and second distances between the second embedding vector and the normal and faulty category centers, respectively; The first distance and the second distance are calculated by using a Euclidean distance function at the intersection of the two spatial feature extractors to obtain a similarity between the first embedding vector and the second embedding vector, and used as a training feature similarity of the two weighted undirected attribute graphs; Based on the difference between feature similarity and training feature similarity, a target twin network is trained to calculate the feature similarity of two weighted undirected attribute graphs.
4. The method according to claim 3, characterized in that The step of inputting two target weighted undirected attribute graphs corresponding to the photovoltaic panel to-be-tested image data into the target twin network and determining target feature similarity of the two target weighted undirected attribute graphs includes: The two target weighted undirected attribute graphs corresponding to the photovoltaic panel test image data are respectively input into two identical spatial feature extractors in the twin network SN to extract the third embedding vector and the fourth embedding vector corresponding to the two target weighted undirected attribute graphs respectively; By using the Euclidean distance function at the intersection of the two spatial feature extractors, a third distance between the third embedding vector and the center of the normal and fault categories, and a fourth distance between the fourth embedding vector and the center of the normal and fault categories are calculated; The third distance and the fourth distance are calculated by the Euclidean distance function at the intersection of the two spatial feature extractors to obtain the similarity between the third embedding vector and the fourth embedding vector, and used as the target feature similarity of the two target weighted undirected attribute graphs.
5. The method according to claim 1, characterized in that The target feature similarity is sequentially passed through a fully connected layer and a softmax layer to determine the failure probability of the photovoltaic panel, including: After the target feature similarity passes through the fully connected layer and the softmax layer in sequence, the probability that the two target weighted undirected attribute graphs belong to the same category is obtained; When any target weighted undirected attribute graph is known to be faulty, the fault probability of the photovoltaic panel corresponding to another target weighted undirected attribute graph is predicted.
6. A twin network photovoltaic fault detection device based on graph similarity, characterized in that: The device comprises: An acquisition module is used to acquire image data of the photovoltaic panel in a normal state, and select a set of key points in the grayscale image corresponding to the photo in the image data that are invariant to scale and rotation, and a SIFT feature descriptor of each key point, wherein the SIFT feature descriptor includes position, direction and scale information; A construction module for constructing a weighted undirected attribute graph based on the SIFT feature descriptor of each key point and the key point; A building module, used for building a training sample composed of two weighted undirected attribute graphs, wherein the training sample is marked with a label, and the label is used for judging the feature similarity of the two weighted undirected attribute graphs; A training module is used to take two weighted undirected attribute graphs of the training sample as input of the twin network SN, and the feature similarity as output, so as to train a target twin network that judges the similarity of the corresponding features of the two weighted undirected attribute graphs; A judgment module is used to input two target weighted undirected attribute graphs corresponding to the photovoltaic panel to be tested image data into the target twin network, and judge the target feature similarity of the two target weighted undirected attribute graphs; The determination module is used to sequentially pass the target feature similarity through a fully connected layer and a softmax layer to determine the failure probability of the photovoltaic panel.
7. The device according to claim 6, characterized in that The building blocks are specifically used for: Based on the SIFT feature descriptor of each key point, the feature similarity between any two key points is calculated; Edges are constructed between key points based on feature similarity, and feature similarity is used as the weight of the edge to generate a weighted undirected attribute graph.
8. The device according to claim 6, characterized in that The training module is specifically used for: The two weighted undirected attribute graphs are respectively input into two identical spatial feature extractors in the twin network SN to extract the first embedding vector and the second embedding vector corresponding to the two weighted undirected attribute graphs; The average of the first embedding vector and the second embedding vector is used as the category center of the normal and faulty photovoltaic panels; Calculate first distances between the first embedding vector and the normal and faulty category centers, and second distances between the second embedding vector and the normal and faulty category centers, respectively; The first distance and the second distance are calculated by using a Euclidean distance function at the intersection of the two spatial feature extractors to obtain a similarity between the first embedding vector and the second embedding vector, and used as a training feature similarity of the two weighted undirected attribute graphs; Based on the difference between feature similarity and training feature similarity, a target twin network is trained to calculate the feature similarity of two weighted undirected attribute graphs.
9. The device according to claim 8, characterized in that The judging module is used for: The two target weighted undirected attribute graphs corresponding to the photovoltaic panel test image data are respectively input into two identical spatial feature extractors in the twin network SN to extract the third embedding vector and the fourth embedding vector corresponding to the two target weighted undirected attribute graphs respectively; By using the Euclidean distance function at the intersection of the two spatial feature extractors, a third distance between the third embedding vector and the center of the normal and fault categories, and a fourth distance between the fourth embedding vector and the center of the normal and fault categories are calculated; The third distance and the fourth distance are calculated by the Euclidean distance function at the intersection of the two spatial feature extractors to obtain the similarity between the third embedding vector and the fourth embedding vector, and used as the target feature similarity of the two target weighted undirected attribute graphs.
10. The device according to claim 6, characterized in that The determination module is specifically used for: After the target feature similarity passes through the fully connected layer and the softmax layer in sequence, the probability that the two target weighted undirected attribute graphs belong to the same category is obtained; When any target weighted undirected attribute graph is known to be faulty, the fault probability of the photovoltaic panel corresponding to another target weighted undirected attribute graph is predicted.
11. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 5.
12. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-5.
13. A computer program product, characterized in that The invention comprises a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 5.