Photovoltaic backlight side part identification method, fault detection method and related device

Abstract

The invention relates to the field of photovoltaic technology, and discloses a photovoltaic backlight side part identification method, a fault detection method and a related device, and the method comprises the steps: obtaining a to-be-identified part image at the backlight side of a photovoltaic panel; part key points in the to-be-recognized part image are extracted; using a pre-training classifier to process the patch feature data of any to-be-processed key point in the component key points to obtain a category prediction result; and identifying the parts in the to-be-identified part image according to the category prediction result of each part key point and the corresponding coordinate data. According to the method, the to-be-identified part image on the backlight side of the photovoltaic panel is acquired, the key points of the parts are extracted, and the key information in the part image on the backlight side of the photovoltaic panel is accurately analyzed and processed by using the pre-training classifier, so that the interference and error influence in the image are effectively reduced, the prediction category is generated, and the prediction accuracy is improved. And accurately identifying the part and the position of the part according to the coordinate data corresponding to the prediction category of each key point.

Description

Technical Field

[0001] The present disclosure relates to the field of photovoltaic technology, and in particular to a photovoltaic backlight side component identification method, a fault detection method and related devices. Background Art

[0002] Photovoltaic technology originated from the discovery of the photoelectric effect in the 19th century. In the 1950s, the development of silicon photovoltaic cells marked the beginning of modern photovoltaic technology. Since then, photovoltaic technology has continued to develop and its cost has continued to decrease. At present, photovoltaic technology has become an important renewable energy solution and is widely used in residential, commercial and large-scale solar power plants.

[0003] In the photovoltaic field, there are many difficulties in the inspection, maintenance and repair of the backlight side components of photovoltaic panels. Augmented reality technology has the advantages of intuitively presenting virtual information and breaking through the limitations of real observation. Therefore, it is extremely important to combine augmented reality technology with photovoltaic panels. Through augmented reality, the limitations of direct observation can be broken, and the complex backlight side components of photovoltaic panels can be presented in a more intuitive way, providing efficient and convenient operation guidance for operation and maintenance personnel. In this process, it is very important to accurately identify the location information of components in real scenes. Only by accurately obtaining the location information of components can accurate virtual information superposition be achieved in the augmented reality system, so as to timely discover potential faults, improve maintenance and repair efficiency, optimize equipment design, improve the overall performance of photovoltaic systems, reduce costs, and promote the further development of the photovoltaic industry. Summary of the invention

[0004] In view of the above situation, the embodiments of the present disclosure provide a photovoltaic backlight side component identification method, a fault detection method and related devices, aiming to solve the above problems or at least partially solve the above problems.

[0005] In a first aspect, an embodiment of the present disclosure provides a method for identifying components on a photovoltaic backlight side, the method comprising:

[0006] Acquire the image of the component to be identified on the backlight side of the photovoltaic panel;

[0007] Extracting key points of components in the image of the component to be identified;

[0008] Using a pre-trained classifier, processing the patch feature data of any key point to be processed among the key points of the component to obtain a category prediction result;

[0009] The components in the to-be-identified component image are identified according to the category prediction results of the key points of each component and the corresponding coordinate data.

[0010] In a second aspect, the embodiments of the present disclosure further provide a method for detecting a fault of a photovoltaic backlight component, the method comprising:

[0011] Obtaining the power supply status of the photovoltaic backlight side components; the power supply status is used to indicate whether the external power supply is normal;

[0012] If the power supply status is that the external power supply is not supplying power normally, the whole machine energy storage module is started to supply power, and fault detection is performed based on the fault detection algorithm to determine the faulty parts of the photovoltaic backlight side components and the fault causes corresponding to the faulty parts;

[0013] If the fault cause is a line fault, the sub-energy storage module corresponding to the faulty component is started.

[0014] In a third aspect, the present disclosure also provides a device for identifying components on the backlight side of a voltmeter, the device comprising:

[0015] An image acquisition module, used to acquire images of components to be identified on the backlight side of the photovoltaic panel;

[0016] An extraction module, used to extract key points of components in the image of the component to be identified;

[0017] A classification module, used to process the surface feature data of any to-be-processed key point among the key points of the component using a pre-trained classifier to obtain a category prediction result;

[0018] The recognition module is used to recognize the components in the image of the component to be recognized according to the category prediction results of the key points of each component and the corresponding coordinate data.

[0019] In a fourth aspect, an embodiment of the present disclosure further provides a photovoltaic backlight component fault detection device, the device comprising:

[0020] An acquisition module is used to acquire the power supply status of the photovoltaic backlight side components; the power supply status is used to indicate whether the external power supply is normal;

[0021] A first processing module is used for starting the whole machine energy storage module to supply power if the power supply status is that the external power supply is not supplying power normally, and performing fault detection based on a fault detection algorithm to determine the faulty parts of the photovoltaic backlight side components and the fault causes corresponding to the faulty parts;

[0022] The second processing module is used to start the sub-energy storage module corresponding to the faulty component if the fault cause is a line fault.

[0023] In a fifth aspect, an embodiment of the present disclosure further provides an electronic device, comprising: a processor; and a memory arranged to store computer executable instructions, wherein when the executable instructions are executed, the processor executes the steps of the method described in the first aspect or the method described in the second aspect.

[0024] In the sixth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, which stores one or more programs. When the one or more programs are executed by an electronic device including multiple application programs, the electronic device executes the steps of the method described in the first aspect or the second aspect above.

[0025] By means of the above-mentioned technical scheme, the photovoltaic backlight side component identification method, fault detection method and related devices provided by the embodiments of the present disclosure obtain the image of the component to be identified on the backlight side of the photovoltaic panel, extract the key points of the component, and use the pre-trained classifier to accurately analyze and process the key information in the image of the component on the backlight side of the photovoltaic panel, effectively reduce the interference and error effects in the image, generate predicted categories, and then accurately identify the components and the positions of the components based on the coordinate data corresponding to the predicted categories of each key point.

[0026] The above description is only an overview of the technical solution of the present disclosure. In order to more clearly understand the technical means of the present disclosure, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present disclosure more obvious and easy to understand, the specific implementation methods of the present disclosure are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The drawings described herein are used to provide a further understanding of the present disclosure and constitute a part of the present disclosure. The illustrative embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation on the present disclosure. In the drawings:

[0028] Figure 1 A schematic diagram showing a flow chart of a method for identifying photovoltaic backlight side components provided by an embodiment of the present disclosure;

[0029] Figure 2 A schematic diagram showing a flow chart of a photovoltaic backlight side component fault detection method provided by an embodiment of the present disclosure;

[0030] Figure 3 A schematic diagram of the structure of a photovoltaic backlight side component identification device provided by an embodiment of the present disclosure is shown;

[0031] Figure 4 A schematic structural diagram of a photovoltaic backlight component fault detection device provided by an embodiment of the present disclosure is shown;

[0032] Figure 5A schematic structural diagram of an electronic device provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solutions and advantages of the present disclosure clearer, the technical solutions of the present disclosure will be clearly and completely described below in combination with the specific embodiments of the present disclosure and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present disclosure.

[0034] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0035] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that such usage is interchangeable where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. In addition, the term "including" and its variants are to be interpreted as open-ended terms meaning "including but not limited to".

[0036] Before introducing the specific embodiments, the professional terms involved in the embodiments of the present disclosure are explained first:

[0037] 1) Homography Matrix is ​​an important concept in computer vision. It describes the mapping relationship between two planes. In AR applications, virtual objects need to be accurately superimposed on the image of the real scene. By calculating the homography matrix, the mapping relationship between the real scene image plane and the virtual coordinate system can be determined, thereby achieving the correct placement of virtual objects.

[0038] 2) Model matrix, which is used to perform matrix operations on the position of the model in the model scene, that is, to place the model in a fixed position in the scene with a fixed posture. Once fixed, the relative position of the model in the scene remains unchanged regardless of how the scene changes.

[0039] 3) Projection matrix, which is used to transform the current scene. As the viewpoint of a person changes, the scene view seen is different. The projection matrix is ​​the matrix used to re-project the scene view according to the change of the human eye's perspective.

[0040] As mentioned above, augmented reality can break through the limitations of direct observation and present the complex situation of photovoltaic panel backlight side components in a more intuitive way, providing efficient and convenient operation guidance for operation and maintenance personnel. In this process, it is very important to accurately identify the location information of components in real scenes. Only by accurately obtaining the location information of components can accurate virtual information superposition be achieved in the augmented reality system, so as to timely discover potential faults, improve maintenance and repair efficiency, optimize equipment design, enhance the overall performance of photovoltaic systems, reduce costs, and promote the further development of the photovoltaic industry. Based on this, the present invention proposes a method for identifying photovoltaic backlight side components, a fault detection method and related devices. The present disclosure is described in detail through specific embodiments below.

[0041] To facilitate understanding of this embodiment, a photovoltaic backlight side component identification method disclosed in the embodiment of the present disclosure is first introduced in detail. The execution subject of the photovoltaic backlight side component identification method provided in the embodiment of the present disclosure is generally a computer device with certain computing capabilities, and the computer device includes, for example: a terminal device or a server or other processing device, and the terminal device can be a user equipment (User Equipment, UE), a mobile device, a user terminal, a terminal, a personal digital assistant (Personal Digital Assistant, PDA), etc. In some possible implementations, the photovoltaic backlight side component identification method can be implemented by a processor calling a computer-readable instruction stored in a memory.

[0042] Figure 1 The schematic diagram of the process of identifying photovoltaic backlight side components provided by the embodiment of the present disclosure is shown. Figure 1 It can be seen that the embodiment of the present disclosure at least includes steps S101-S104:

[0043] S101: Acquire an image of a component to be identified on the backlight side of a photovoltaic panel.

[0044] Here, the types of components in the component image to be identified include, but are not limited to: a junction box, a bus bar, a diode, a bypass switch, or a connector.

[0045] During implementation, for example, a specially designed inspection robot can be equipped with an image acquisition device and move along the backlight side of the photovoltaic panel to acquire images of the components to be identified. The present disclosure does not limit the method for acquiring the images of the components to be identified.

[0046] S102: Extracting key points of components in the image of the component to be identified.

[0047] In this step, key points refer to characteristic points in the image that are unique, stable, and distinguishable. For example, for an image of a component to be identified that includes a photovoltaic cell, the key points of the component may be the edge corners of the cell, the location of a specific identification symbol on the cell, the contact point between the wire connecting the cell and the cell, etc.

[0048] In specific implementation, for example, algorithms such as SIFT (Scale-Invariant Feature Transform), SURF (Speeded Up Robust Features), and ORB (Oriented FAST and Rotated BRIEF) can be used to extract one or more key points of parts from the image of the part to be identified.

[0049] S103: Using a pre-trained classifier, the patch feature data of any to-be-processed key point among the key points of the component are processed to obtain a category prediction result.

[0050] This embodiment does not limit the specific structure of the pre-trained classifier, as long as it can achieve pixel classification.

[0051] S104: Identify the components in the component image to be identified according to the category prediction results of the key points of each component and the corresponding coordinate data.

[0052] The disclosed embodiment obtains an image of a component to be identified on the backlight side of a photovoltaic panel and extracts key points of the component therein, and uses a pre-trained classifier to accurately analyze and process key information in the image of the component on the backlight side of the photovoltaic panel, thereby effectively reducing interference and error effects in the image, generating predicted categories, and then accurately identifying the components and their positions based on the coordinate data corresponding to the predicted category of each key point.

[0053] Furthermore, in order to better illustrate the process of the photovoltaic backlight side component identification method, as a refinement and extension of the above embodiment, the embodiment of the present invention provides several optional embodiments, but is not limited thereto, as shown below.

[0054] In a possible implementation manner, the extracting of the key points of the parts in the image of the part to be identified includes: for any unclassified pixel on the image of the part to be identified, randomly determining a first neighboring pixel and a second neighboring pixel whose connecting line passes through the center of a circle with the unclassified pixel as the center and a first preset distance as the radius; calculating the grayscale value difference between the unclassified pixel and the first neighboring pixel and the second neighboring pixel, respectively, to obtain a first grayscale difference and a second grayscale difference; if the first grayscale difference and the second grayscale difference meet a first preset condition, and the first grayscale difference satisfies .... If the actual number of times that the first grayscale difference and the second grayscale difference satisfy the first preset condition is greater than or equal to the first preset number, the unclassified pixel point is determined as the key point of the component; the first preset condition is: the first grayscale difference and / or the second grayscale difference is greater than a first preset threshold; if the first grayscale difference and the second grayscale difference satisfy the first preset condition, and the actual number of times is less than the first preset number, then return to randomly determining the first neighboring pixel point and the second neighboring pixel point on the circumference of a circle with the unclassified pixel point as the center and the first preset distance as the radius, and the line passing through the center of the circle.

[0055] This implementation converts the problem of extracting key points of image parts into a binary classification problem of key points and non-key points. During implementation, for any unclassified pixel point on the image of the part to be identified, such as pixel point m, a circle is drawn with pixel point m as the center and the first preset distance as the radius, and two pixel points whose connecting lines pass through the center of the circle are randomly determined on the circumference, and are named as the first adjacent pixel point and the second adjacent pixel point, respectively. Here, the first preset distance can be set according to actual needs, and this embodiment does not limit this.

[0056] Then, the grayscale value of the first adjacent pixel is subtracted from the grayscale value of the pixel m to obtain the first grayscale difference; the grayscale value of the second adjacent pixel is subtracted from the grayscale value of the pixel m to obtain the second grayscale difference. Next, the relationship between the grayscale value of the pixel m, the first grayscale difference, and the second grayscale difference is determined. Specifically, the determination can be made according to the following formula:

[0057] The gray value of the pixel, represents the gray value of the second neighboring pixel, d is a unit vector, R represents the first preset distance, and α represents the angle between the line between the first neighboring pixel and the second neighboring pixel and the horizontal diameter of the circle. Here, the first preset threshold can be set according to actual needs and is not limited in this embodiment.

[0058] If the first grayscale difference and the second grayscale difference are both less than the first preset threshold, it indicates that the difference between the unclassified pixel m and its first neighboring pixel and the second neighboring pixel in grayscale value is not large, and it can be considered that they have certain similarities in vision or texture, and the unclassified pixel m, the first neighboring pixel and the second neighboring pixel are determined as non-critical points; if the first grayscale difference is greater than the first preset threshold, or the second grayscale difference is greater than the first preset threshold, or the first grayscale difference and the second grayscale difference are both greater than the first preset threshold, it indicates that the unclassified pixel m and its first neighboring pixel and the second neighboring pixel have certain differences in grayscale value, and it is further determined whether the actual number of times the first grayscale difference and the second grayscale difference meet the first preset condition is greater than or equal to the first preset number. Here, the first preset number can be set according to actual needs. For example, it can be set to 4 times. If the actual number of times is greater than or equal to the first preset number of times, the unclassified pixel point m is determined as a key point of the component; if the actual number of times is less than the first preset number of times, the process returns to the step of determining the first neighboring pixel point and the second neighboring pixel point on a circle with the unclassified pixel point m as the center and the first preset distance as the radius. Continuously iterate, and finally realize the binary classification of the pixel point m in the image. By analogy, it can be determined whether the pixel points at other positions in the image are key points.

[0059] This embodiment transforms the key point extraction problem into a pixel binary classification problem. Specifically, on a circle with an unclassified pixel as the center and a first preset distance as the radius, two adjacent pixels are randomly determined, and the grayscale difference between the unclassified pixel and the two adjacent pixels is calculated respectively, and the category of the pixel is judged in combination with the first preset condition and the first preset number of times. By means of repeated and random selection of adjacent pixels and accurate judgment of grayscale value differences, the robustness of component key point recognition is significantly improved, ensuring accurate, stable and efficient screening of key pixels in a high noise environment.

[0060] In a possible implementation manner, the method further includes: performing a multi-scale Gaussian convolution operation and a downsampling operation on the image of the component to be identified to obtain a first set of images to be processed having a Gaussian pyramid structure; and calculating a first criticality score of any candidate key point among the key points of the component using the following formula:

[0061]

[0062] Wherein, score(m) represents the first criticality score of the candidate key point m, n represents the number of elements in the first set of images to be processed, represents the gray value of the candidate key point m in the first image to be processed at the i-th layer of the Gaussian pyramid, represents the gray value of the third neighboring pixel corresponding to the candidate key point m in the first image to be processed at the i-th layer of the Gaussian pyramid, represents the gray value of the fourth neighboring pixel point corresponding to the candidate key point m in the first to-be-processed image of the i-th layer of the Gaussian pyramid, d is a unit vector, α represents the angle between the line between the third neighboring pixel point and the fourth neighboring pixel point and the horizontal diameter of a circle with the candidate key point m as the center and the first preset length R as the radius; according to each of the first criticality scores, each of the component key points is screened, and the screened component key points are used as the component key points of the component image to be identified.

[0063] In this embodiment, the total number of layers of the Gaussian pyramid can be set according to actual needs. For example, it can be set to 4. During implementation, for example, the key point with a larger first criticality score can be taken as the final component key point. Exemplarily, if there are 10 component key points, the first criticality scores can be sorted in descending order, and the first 5 key points can be taken as the final component key points.

[0064] This embodiment first uses the binary classification idea to quickly narrow the search range of component key points, and then uses the Gaussian-Laplace-based scoring mechanism to further refine the selection of key points. The scoring mechanism can help identify feature points that remain stable and significant at different scales, further ensuring the quality of the extracted component key points.

[0065] In a possible implementation, the use of a pre-trained classifier to process the patch feature data of any to-be-processed key point among the component key points to obtain a category prediction result includes: obtaining a patch to be classified centered on the to-be-processed key point, and performing feature extraction on the patch to be classified based on a random Ferns algorithm to obtain feature data of the patch to be classified; using the pre-trained classifier to process the feature data of the patch to be classified to obtain a corresponding category prediction result; the pre-trained classifier is a semi-naive Bayes classifier.

[0066] In this embodiment, for any key point to be processed among the key points of the parts, a patch to be classified centered on it is first obtained. For example, a square patch with a side length of 32 pixels centered on the key point to be processed can be used as the patch to be classified. Then, for example, the random Ferns algorithm can be used to use the grayscale value brightness comparison results between the pixel point pairs on the patch to be classified, that is, multiple binary values, as the feature data of the patch to be classified corresponding to the patch to be classified. Finally, the trained semi-naive Bayes classifier is used to process the feature data of the patch to be classified to obtain the predicted category with the largest posterior probability.

[0067] Here, the core algorithm of the semi-naive Bayes classifier is as follows: The purpose of the classifier is to find the classification number C with the highest probability corresponding to the patch feature data, as shown in formula (1):

[0068]

[0069] The patch feature data is input into the classifier, and the feature value is used to represent the patch. Formula (1) is equivalent to formula (2):

[0070]

[0071] According to the Bayesian formula, we can get formula (3):

[0072]

[0073] The denominator in the right formula has nothing to do with the classification, while the numerator P(C=c k ) factor can be regarded as a constant, so equation (3) is approximately equal to equation (4):

[0074]

[0075] Divide N eigenvalues ​​into M groups, assuming that the features between the M groups are independent, the matching problem is finally transformed into formula (5):

[0076]

[0077] This embodiment uses a Frens-based semi-naive Bayes classifier to quickly provide a more reliable classification result.

[0078] In one possible implementation, the pre-trained classifier is obtained according to the following method: obtaining a data set; the features of the samples in the data set are sample key point patch feature data, and the labels of the samples are corresponding key point categories; and the initial classifier is trained using the data set to obtain the pre-trained classifier.

[0079] In this embodiment, a data set can be obtained first. Specifically, in a possible implementation, the data set is generated according to the following method: multiple sample component images are obtained; the components in each of the sample component images are the components to be identified; for any pixel to be classified on any image in the multiple sample component images, a first sample neighboring pixel point and a second sample neighboring pixel point whose connecting line passes through the center of the circle with the pixel to be classified as the center and the second preset distance as the radius are randomly determined; the grayscale value difference between the pixel to be classified and the first sample neighboring pixel point and the second sample neighboring pixel point is calculated, respectively, to obtain a first sample grayscale difference and a second sample grayscale difference; if the first sample grayscale difference and the second sample grayscale difference satisfy the preset second condition, and the number of times the first sample grayscale difference and the second sample grayscale difference satisfy the preset second condition is greater than or equal to the preset second condition times, the pixel to be classified is determined as a key point; the preset second condition is: the first sample grayscale difference and / or the second sample grayscale difference is greater than the preset second threshold; if the first sample grayscale difference and the second sample grayscale difference meet the preset second condition, and the number of times the first sample grayscale difference and the second sample grayscale difference meet the preset second condition is less than the preset second number, then return to randomly determining the first sample neighboring pixel and the second sample neighboring pixel on the circle with the pixel to be classified as the center and the second preset distance as the radius, and the connecting line passes through the center of the circle; according to the judgment result of whether each pixel on each of the sample component images is a key point, the pixel points at each position are screened to obtain a target key point set; according to the patch feature data and the corresponding category of each key point in the target key point set, the data set is generated.

[0080] In this embodiment, a plurality of sample component images for the components to be identified may be obtained first. In implementation, they may be obtained by photographing with a camera. Alternatively, a sample component image may be directly used as a material to generate a sample set by an automatic synthesis method. Specifically, an affine transformation method may be used to process the sample component image, generate a new random view, and add white noise to obtain a sample set of size S.

[0081] After obtaining multiple sample parts images, key points and non-key points are classified for each pixel to be classified on each image. The specific classification process is similar to the classification process of any unclassified pixel on the image of the part to be identified, and will not be repeated here.

[0082] Then, according to the determination result of whether each pixel point on each sample component image is a key point, the pixel points at each position can be screened for keyness to obtain a target key point set. In a possible implementation, according to the determination result of whether each pixel point on each sample component image is a key point, the pixel points at each position can be screened for keyness to obtain a target key point set, including: according to the determination result of whether each pixel point on each sample component image is a key point, the number of times the pixel point at the same position is determined as a key point is calculated to obtain multiple target numbers; and the pixel points whose target numbers are ranked in front of a preset number of digits are selected to form the target key point set.

[0083] Here, the preset number can be determined according to actual needs. For example, suppose there are 5 sample parts images, each of which is 100×100 pixels in size. In the first image, the pixel at the upper left corner (0,0) is determined as a key point; in the second image, the pixel at the same position is not determined as a key point; in the third image, the pixel at the same position is determined as a key point; in the fourth image, the pixel at the same position is not determined as a key point, and in the fifth image, the pixel at the same position is determined as a key point. Then the number of times the pixel at the same position (i.e., the upper left corner (0,0) position) is determined as a key point is calculated to be 3 times. By analogy, the number of times the pixels at all positions are determined as key points is calculated to obtain multiple target times. Assuming the preset number is 4, sort the target times of all pixel points at all positions, and take the top 4 pixel points: position 1 (10, 20) (target times is 4 times), position 2 (30, 30), target times is 5 times, position 3 (50, 50), target times is 4 times, position 4 (80, 80), target times is 4 times. The pixel points at these positions constitute the target key point set.

[0084] In the training phase, this embodiment transforms the key point extraction problem in the sample image into a pixel binary classification problem. By means of repeated and random selection of adjacent pixels and accurate judgment of gray value differences, the robustness of key point recognition is significantly improved, ensuring accurate, stable and efficient screening of key pixels in sample component images in a high-noise environment, thereby significantly improving the quality of subsequent data sets.

[0085] In another possible implementation, the pixel points at each position are screened according to the determination result of whether each pixel point on each of the sample component images is a key point to obtain a target key point set, including: for the pixel point to be evaluated at any position, a target image is screened from the multiple sample component images; in the target image, the pixel point to be evaluated is a key point; a multi-scale Gaussian convolution operation and downsampling operation are performed on any image in the target image to obtain a second set of images to be processed with a structure of a Gaussian pyramid; and a second critical score of the pixel point to be evaluated is calculated using the following formula:

[0086]

[0087] Wherein, score(m1) represents the second critical score of the pixel m1 to be evaluated, n1 represents the number of elements in the second image set to be processed, represents the gray value of the pixel m1 to be evaluated in the second processed image at the jth layer of the Gaussian pyramid, represents the gray value of the third sample neighboring pixel corresponding to the pixel m1 to be evaluated in the second image to be processed at the jth layer of the Gaussian pyramid, represents the gray value of the fourth sample neighboring pixel corresponding to the pixel m1 to be evaluated in the second to-be-processed image of the j-th layer of the Gaussian pyramid, d is a unit vector, α1 represents the angle between the line between the third sample neighboring pixel and the fourth sample neighboring pixel and the horizontal diameter of a circle with the pixel m1 to be evaluated as the center and the second preset length R1 as the radius; each of the second critical scores is added to obtain the total score of the pixel to be evaluated; and each of the pixels to be evaluated is screened according to each of the total scores to obtain the target key point set.

[0088] In specific implementation, the total score of the pixel to be evaluated can be calculated according to the following formula:

[0089]

[0090] Where score(m1)′ represents the total score of the pixel m1 to be evaluated, S represents the number of sample component images, and r q Indicates whether the pixel m1 to be evaluated in the qth sample component image is a key point in the early pixel binary classification stage. If the pixel m1 to be evaluated is a key point, r qis 1, otherwise it is 0. During implementation, for example, the key point with a larger total score can be taken as the target key point. Exemplarily, there are 10 pixel points to be evaluated, then the total scores can be sorted in descending order, and the first 5 key points can be taken as the target key points. This embodiment first uses the idea of ​​binary classification to quickly narrow the search range of component key points, and then uses the Gaussian-Laplace-based scoring mechanism to further refine the selection of key points. The scoring mechanism can help identify feature points that remain stable and significant at different scales, further ensuring the quality of the extracted target key points, thereby significantly improving the quality of subsequent data sets.

[0091] Finally, a data set is generated based on the patch feature data of each key point in the target key point set and the corresponding category. In specific implementation, the generation process of patch feature data is similar to the generation process of patch feature data to be classified, which will not be repeated here.

[0092] After obtaining the data set, the data set can be used to train the initial classifier to obtain a pre-trained classifier.

[0093] In a possible implementation, the method further includes: calculating a target homography matrix based on coordinate data of key points of each component and virtual coordinate data of corresponding key points in the virtual component image; setting a model matrix; and flipping the target homography matrix; and merging the components in the virtual component image with the image information of the components in the component image to be identified according to the model matrix and the flipped target homography matrix to obtain an augmented reality image of the component to be identified.

[0094] This embodiment can be implemented based on tools such as OpenGL and DirectX. In this embodiment, the target homography matrix can be calculated based on the coordinate data of the key points of each component and the virtual coordinate data of the corresponding key points in the virtual component image. In specific implementation, for example, the target homography matrix can be calculated using the RANSAC (Random Sample Consensus) algorithm, the Direct Linear Transform (DLT) algorithm, the least squares method, etc. In a possible implementation of the present disclosure, the target homography matrix is ​​calculated based on the coordinate data of each of the component key points and the virtual coordinate data of the corresponding key points in the virtual component image, including: screening each of the component key points according to the category probability corresponding to the category prediction result of each of the component key points to obtain an initial matching set and a matching set to be judged; the matching data in the matching set includes the coordinate data of the component key points and the virtual coordinate data of the corresponding key points in the virtual component image; the overall level of the category probability of the component key points in the initial matching set is greater than the overall level of the category probability of the component key points in the matching set to be judged; using a preset number of groups of matching data in the initial matching set to calculate the initial homography matrix; based on the initial homography matrix, a support set is generated according to the matching set to be judged; if the number of matching data in the support set is greater than or equal to a preset threshold, the initial homography matrix is ​​updated using the support set to obtain the target homography matrix; if the number of matching data in the support set is less than the preset threshold, returning to the category probability corresponding to the category prediction result of each of the component key points, screening each of the component key points to obtain the initial matching set and the matching set to be judged.

[0095] In this embodiment, an initial matching set and a matching set to be judged can be generated first. The initial matching set is used to calculate the initial homography matrix, and the matching set to be judged is used to screen the matching data to form a support set. In specific implementation, the mean of the maximum and minimum values ​​in the category probability of each component key point can be used as a threshold, the arithmetic mean of the category probability of each component key point can be used as a threshold, and the median of the category probability of each component key point can be used as a threshold. It is judged one by one whether each category probability is greater than or equal to the threshold. If so, the matching data consisting of the coordinate data and virtual coordinate data corresponding to the category probability is divided into the initial matching set; otherwise, it is divided into the matching set to be judged.

[0096] After obtaining the initial matching set and the matching set to be judged, a preset number of groups of matching data can be randomly selected from the initial matching set, where the preset number is greater than or equal to 3, and can be set according to actual needs. Using these matching data, the initial homography matrix is ​​calculated. Then, all matching data that satisfy the initial homography matrix within a certain error tolerance range are found from the matching set to be judged to obtain the support set. It is determined whether the number of matching data in the support set is greater than or equal to a preset threshold, and the preset threshold can be set according to actual needs. If the number of matching data in the support set is greater than or equal to the preset threshold, the initial homography matrix is ​​updated using the support set to obtain the target homography matrix; if the number of matching data in the support set is less than the preset threshold, then return to the step of screening the key points of each component according to the category probability of the key points of each component to obtain the initial matching set and the matching set to be judged.

[0097] During implementation, the target homography matrix may also be updated using the Levenberg-Marquardt least squares estimation to obtain a more accurate, reliable and stable target homography matrix.

[0098] This embodiment improves the ARANSAC fitting algorithm by screening the key points of the parts according to their category probabilities to generate an initial matching set and a matching set to be judged. This processing method can more effectively utilize the key point information. The overall category probability of the key points in the initial matching set is high, and the proportion of internal points increases, providing a more reliable data basis for calculating the initial homography matrix, thereby improving the accuracy of the initial homography matrix estimation, and further improving the fitting performance. The improved algorithm has greatly improved performance in all aspects compared to the baseline algorithm, and can meet the real-time and reliability requirements of the augmented reality system.

[0099] After obtaining the target homography matrix, set the target homography matrix as the projection matrix. Since the target homography matrix and the rotation and translation matrix observed by the human eye are inverse to each other, the projection matrix is ​​flipped. Then the model matrix can be set so that the model is located in the center of the scene, the depth of field is -90, and the plane normal vector direction of the virtual component image coincides with the z-axis. Then set the scene depth of field z to range from 100 to 1000000. Fit the components in the virtual component image with the image of the component to be identified to obtain the corresponding augmented reality image. During implementation, the image of the component to be identified can also be obtained in real time, and the scene background can be set as a real-time input frame to achieve real-time fitting of the virtual component image.

[0100] This embodiment can merge the virtual component image with the image information of the component to be identified in the component image to generate a realistic and accurate augmented reality image of the component to be identified, so that the user can clearly and intuitively obtain the complete information of the component, which helps to improve the efficiency of component inspection, maintenance and design, and brings great convenience to work in related fields.

[0101] Figure 2 The schematic diagram of the process of the photovoltaic backlight side component fault detection method provided by the embodiment of the present disclosure is shown. Figure 1 It can be seen that the embodiment of the present disclosure at least includes steps S201-S203:

[0102] S201: Acquire the power supply status of the photovoltaic backlight side components; the power supply status is used to indicate whether the external power supply is normal;

[0103] S202: If the power supply status is that the external power supply is not supplying power normally, the whole machine energy storage module is started to supply power, and fault detection is performed based on a fault detection algorithm to determine the faulty parts of the photovoltaic backlight side components and the fault causes corresponding to the faulty parts;

[0104] S203: If the fault is caused by a line fault, starting the sub-energy storage module corresponding to the faulty component.

[0105] This embodiment can timely detect and determine the fault conditions of photovoltaic backlight side components, including faulty components and their causes. When the external power supply fails to supply power normally, the whole machine energy storage module can be started to supply power, and the fault detection algorithm can be used to perform fault diagnosis to ensure the normal operation of the photovoltaic system and rapid response to faults.

[0106] In a possible implementation, the method further includes: if the result of the fault detection is normal, charging the energy storage module of the photovoltaic backlight side component according to a preset charging order; the preset charging order includes: charging in descending order of priority of the energy storage module, and charging in ascending order of the current power of the energy storage module. In this embodiment, when the fault detection result of the photovoltaic backlight side component is normal, the energy storage module can be charged in a preset charging order to improve the charging efficiency and system stability. By considering the priority and current power of the energy storage module, the charging strategy is optimized to ensure that key components are charged first, thereby improving the reliability and efficiency of the entire photovoltaic system.

[0107] In a possible implementation, charging the energy storage module of the photovoltaic backlight side component according to a preset charging sequence includes: determining the highest priority charging sequence in the preset charging sequence, and charging the energy storage module of the photovoltaic backlight side component based on the highest priority charging sequence; wherein the priority of the charging sequence is related to a reference object corresponding to the charging sequence, and the reference object includes the priority of the energy storage module and the current power level of the energy storage module. This embodiment charges the energy storage module of the photovoltaic backlight side component in an orderly manner by determining the highest priority charging sequence in the preset charging sequence, and takes into account the relationship between the priority of the charging sequence and the reference object (such as the priority of the energy storage module and the current power level), making the charging process more flexible and efficient.

[0108] In one possible implementation, the priority of charging in ascending order of the current power of the energy storage modules is higher than that of charging in descending order of the priority of the energy storage modules. This embodiment clarifies that the priority of charging in ascending order of the current power of the energy storage modules is higher than that of charging in descending order of the priority of the energy storage modules. This priority setting helps to charge the modules with low power first, thereby avoiding system failures caused by power exhaustion.

[0109] In a possible implementation, the priority of the energy storage module is positively correlated with the importance of the part corresponding to the energy storage module. This embodiment associates the priority of the energy storage module with the importance of the part corresponding to it, ensuring that the energy storage module corresponding to the more important part can be charged first, thereby enhancing the system's protection of key components.

[0110] In a possible implementation, the method further includes: if the cause of the fault is a fault of the component itself, outputting fault alarm information. In this embodiment, when the cause of the fault is a fault of the component itself, the fault alarm information can be output to promptly notify maintenance personnel to handle the problem, thereby reducing system downtime and potential safety risks caused by component failure.

[0111] In a possible implementation, the outputting of the fault alarm information includes: determining the level of the fault of the part itself, and outputting the fault alarm information matching the level of the fault of the part itself. This embodiment determines the level of the fault of the part itself, and outputs the fault alarm information matching the level of the fault, so that the maintenance personnel can take corresponding maintenance measures according to the severity of the fault, thereby improving the efficiency and pertinence of fault handling.

[0112] In a possible implementation, the determining of the level of the component fault includes: determining the level of the component fault according to the level of the faulty component, or determining the level of the component fault according to whether the fault of the component affects the operation of the photovoltaic backlight side component. This embodiment provides a flexible method for determining the fault level, which can determine the fault level according to the level of the faulty component, or determine the fault level according to whether the fault affects the operation of the photovoltaic backlight side component, making fault handling more accurate and effective.

[0113] Those skilled in the art will appreciate that, in the above method of the specific implementation manner, the writing order of the steps does not mean a strict execution order, but does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.

[0114] It should be noted that, in practical applications, all possible implementations described above can be combined in any manner to form possible embodiments of the present disclosure, which will not be described one by one here.

[0115] Based on the same concept, the embodiment of the present disclosure also provides a photovoltaic backlight side component identification device, Figure 3 FIG. 1 is a schematic diagram showing a structure of a photovoltaic backlight side component identification device provided by an embodiment of the present disclosure, see Figure 3 As shown, the photovoltaic backlight side component identification device 300 provided in the embodiment of the present disclosure includes:

[0116] An image acquisition module 301 is used to acquire an image of a component to be identified on the backlight side of a photovoltaic panel;

[0117] An extraction module 302 is used to extract key points of components in the image of the component to be identified;

[0118] The classification module 303 is used to process the patch feature data of any key point to be processed among the key points of the component using a pre-trained classifier to obtain a category prediction result;

[0119] The recognition module 304 is used to recognize the components in the component image to be recognized according to the category prediction results of the key points of each component and the corresponding coordinate data.

[0120] In a possible implementation, in the above-mentioned device, the extraction module 302 is used to: for any unclassified pixel point on the image of the component to be identified, randomly determine a first neighboring pixel point and a second neighboring pixel point on a circle with the unclassified pixel point as the center and a first preset distance as the radius, and the connecting line passes through the center of the circle; calculate the grayscale value difference between the unclassified pixel point and the first neighboring pixel point and the second neighboring pixel point, respectively, to obtain a first grayscale difference and a second grayscale difference; if the first grayscale difference and the second grayscale difference meet the first preset condition, and the first grayscale difference, If the actual number of times that the second grayscale difference satisfies the first preset condition is greater than or equal to the first preset number, the unclassified pixel point is determined as the component key point; the first preset condition is: the first grayscale difference and / or the second grayscale difference is greater than a first preset threshold; if the first grayscale difference and the second grayscale difference satisfy the first preset condition, and the actual number of times is less than the first preset number, return to randomly determining the first neighboring pixel point and the second neighboring pixel point on the circumference of a circle with the unclassified pixel point as the center and the first preset distance as the radius, and the line passing through the center of the circle is randomly determined.

[0121] In a possible implementation manner, the device further includes a scoring module, which is used to: perform multi-scale Gaussian convolution operations and downsampling operations on the image of the component to be identified to obtain a first set of images to be processed having a Gaussian pyramid structure; and calculate a first criticality score of any candidate key point among the key points of the component using the following formula:

[0122]

[0123] Wherein, score(m) represents the first criticality score of the candidate key point m, and n represents the number of elements in the first set of images to be processed. represents the gray value of the candidate key point m in the first image to be processed at the i-th layer of the Gaussian pyramid, represents the gray value of the third neighboring pixel corresponding to the candidate key point m in the first image to be processed at the i-th layer of the Gaussian pyramid, represents the gray value of the fourth neighboring pixel point corresponding to the candidate key point m in the first to-be-processed image of the i-th layer of the Gaussian pyramid, d is a unit vector, α represents the angle between the line between the third neighboring pixel point and the fourth neighboring pixel point and the horizontal diameter of a circle with the candidate key point m as the center and the first preset length R as the radius; according to each of the first criticality scores, each of the component key points is screened, and the screened component key points are used as the component key points of the component image to be identified.

[0124] In a possible implementation, in the above-mentioned device, the classification module 303 is used to: obtain a patch to be classified centered on the key point to be processed, and based on the random Ferns algorithm, perform feature extraction on the patch to be classified to obtain feature data of the patch to be classified; use the pre-trained classifier to process the feature data of the patch to be classified to obtain a corresponding category prediction result; the pre-trained classifier is a semi-naive Bayes classifier.

[0125] In a possible implementation, the device further includes a training module for: acquiring a data set; the features of the samples in the data set are sample key point patch feature data, and the labels of the samples are corresponding key point categories; and using the data set to train an initial classifier to obtain the pre-trained classifier.

[0126] In a possible implementation manner, the device further includes a data set generation module, which is used to: obtain a plurality of sample component images; the components in each of the sample component images are the components to be identified; for any pixel to be classified on any image among the plurality of sample component images, randomly determine a first sample neighboring pixel point and a second sample neighboring pixel point whose connecting line passes through the center of a circle with the pixel to be classified as the center and a second preset distance as the radius; calculate the grayscale value difference between the pixel to be classified and the first sample neighboring pixel point and the second sample neighboring pixel point, respectively, to obtain a first sample grayscale difference and a second sample grayscale difference; if the first sample grayscale difference and the second sample grayscale difference satisfy a preset second condition, and the number of times the first sample grayscale difference and the second sample grayscale difference satisfy the preset second condition is greater than or equal to the preset second condition times, the pixel to be classified is determined as a key point; the preset second condition is: the first sample grayscale difference and / or the second sample grayscale difference is greater than the preset second threshold; if the first sample grayscale difference and the second sample grayscale difference meet the preset second condition, and the number of times the first sample grayscale difference and the second sample grayscale difference meet the preset second condition is less than the preset second number, then return to randomly determining the first sample neighboring pixel and the second sample neighboring pixel on the circle with the pixel to be classified as the center and the second preset distance as the radius, and the connecting line passes through the center of the circle; according to the judgment result of whether each pixel on each of the sample component images is a key point, the pixel points at each position are screened to obtain a target key point set; according to the patch feature data and the corresponding category of each key point in the target key point set, the data set is generated.

[0127] In one possible implementation, in the above-mentioned device, the data set generation module is used to: calculate the number of times the pixel point at the same position is determined as a key point based on the judgment result of whether each pixel point on each of the sample component images is a key point, and obtain multiple target numbers; take the pixel points whose number of times is ranked before a preset number of target numbers to constitute the target key point set.

[0128] In a possible implementation, in the above device, the data set generation module is used to: for a pixel point to be evaluated at any position, screen out a target image from the multiple sample component images; in the target image, the pixel point to be evaluated is a key point; perform a multi-scale Gaussian convolution operation and downsampling operation on any image in the target image to obtain a second set of images to be processed with a Gaussian pyramid structure; and calculate a second key score of the pixel point to be evaluated using the following formula:

[0129]

[0130] Wherein, score(m1) represents the second critical score of the pixel m1 to be evaluated, n1 represents the number of elements in the second image set to be processed, represents the gray value of the pixel m1 to be evaluated in the second processed image at the jth layer of the Gaussian pyramid, represents the gray value of the third sample neighboring pixel corresponding to the pixel m1 to be evaluated in the second image to be processed at the jth layer of the Gaussian pyramid, represents the gray value of the fourth sample neighboring pixel corresponding to the pixel m1 to be evaluated in the second to-be-processed image of the j-th layer of the Gaussian pyramid, d is a unit vector, α1 represents the angle between the line between the third sample neighboring pixel and the fourth sample neighboring pixel and the horizontal diameter of a circle with the pixel m1 to be evaluated as the center and the second preset length R1 as the radius; each of the second critical scores is added to obtain the total score of the pixel to be evaluated; and each of the pixels to be evaluated is screened according to each of the total scores to obtain the target key point set.

[0131] In a possible implementation, the device further includes an augmented reality module, which is used to: calculate a target homography matrix based on the coordinate data of the key points of each component and the virtual coordinate data of the corresponding key points in the virtual component image; set a model matrix; and flip the target homography matrix; and merge the components in the virtual component image with the image information of the components in the component image to be identified according to the model matrix and the flipped target homography matrix to obtain an augmented reality image of the component to be identified.

[0132] In a possible implementation manner, in the above-mentioned device, the augmented reality module is used to: screen each of the component key points according to the category probability corresponding to the category prediction result of each of the component key points to obtain an initial matching set and a matching set to be judged; the matching data in the matching set includes the coordinate data of the component key points and the virtual coordinate data of the corresponding key points in the virtual component image; the overall level of the category probability of the component key points in the initial matching set is greater than the overall level of the category probability of the component key points in the matching set to be judged; calculate the initial homography matrix using a preset number of groups of matching data in the initial matching set; generate a support set based on the initial homography matrix according to the matching set to be judged; if the number of matching data in the support set is greater than or equal to a preset threshold, update the initial homography matrix using the support set to obtain the target homography matrix; if the number of matching data in the support set is less than the preset threshold, return to the category probability corresponding to the category prediction result of each of the component key points, and screen each of the component key points to obtain an initial matching set and a matching set to be judged.

[0133] It should be noted that any of the above-mentioned photovoltaic backlight side component identification devices can implement the above-mentioned photovoltaic backlight side component identification method in a one-to-one correspondence, which will not be described in detail here.

[0134] Based on the same concept, the embodiment of the present disclosure also provides a fault detection device for photovoltaic backlight side components. Figure 4 FIG. 1 shows a schematic diagram of a structure of a fault detection method for photovoltaic backlight side components provided by an embodiment of the present disclosure, see FIG. Figure 4 As shown, the photovoltaic backlight side component fault detection 400 provided in the embodiment of the present disclosure includes:

[0135] The acquisition module 401 is used to acquire the power supply status of the photovoltaic backlight side components; the power supply status is used to indicate whether the external power supply is normal;

[0136] The first processing module 402 is used to start the whole machine energy storage module to supply power if the power supply status is that the external power supply is not supplying power normally, and perform fault detection based on the fault detection algorithm to determine the faulty parts of the photovoltaic backlight side components and the fault causes corresponding to the faulty parts;

[0137] The second processing module 403 is configured to start the sub-energy storage module corresponding to the faulty component if the fault cause is a line fault.

[0138] In a possible implementation, the device further includes a charging module, which is used to: if the result of the fault detection is normal, charge the energy storage module of the photovoltaic backlight side component according to a preset charging sequence; the preset charging sequence includes: charging in descending order of priority of the energy storage modules, and charging in ascending order of current power levels of the energy storage modules.

[0139] In a possible implementation, in the above-mentioned device, the charging module is used to: determine the charging sequence with the highest priority in the preset charging sequence, and charge the energy storage module of the photovoltaic backlight side component based on the charging sequence with the highest priority; wherein the priority of the charging sequence is related to the reference object corresponding to the charging sequence, and the reference object includes the priority of the energy storage module and the current power level of the energy storage module.

[0140] In a possible implementation, in the above device, charging in ascending order of current power levels of the energy storage modules has a higher priority than charging in descending order of priorities of the energy storage modules.

[0141] In a possible implementation, in the above device, the priority of the energy storage module is positively correlated with the importance of the parts corresponding to the energy storage module.

[0142] In a possible implementation manner, the device further includes an alarm module, which is used to output fault alarm information if the fault is caused by a fault of the component itself.

[0143] In a possible implementation manner, the alarm module is used to determine the level of the component's own fault and output fault alarm information matching the level of the component's own fault.

[0144] In a possible implementation, the alarm module is used to determine the level of the part's own fault according to the level of the faulty part, or to determine the level of the part's own fault according to whether the part's own fault affects the operation of the photovoltaic backlight side components.

[0145] It should be noted that any of the above-mentioned photovoltaic backlight side component fault detection devices can implement the above-mentioned photovoltaic backlight side component fault detection method in a one-to-one correspondence manner, which will not be described in detail here.

[0146] Figure 5 FIG. 1 shows a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure. Figure 5As shown, at the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. Among them, the memory may include a memory, such as a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage, etc. Of course, the electronic device may also include hardware required for other services.

[0147] The processor, network interface and memory can be interconnected through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0148] The memory is used to store programs. Specifically, the program may include program code, and the program code includes computer operation instructions. The memory may include internal memory and non-volatile memory, and provides instructions and data to the processor.

[0149] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming a photovoltaic backlight side component identification device or a photovoltaic backlight side component fault detection device at the logical level. The processor executes the program stored in the memory and is specifically used to execute the above method.

[0150] The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The disclosed methods, steps and logic block diagrams in the embodiments of the present disclosure may be implemented or executed. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present disclosure may be directly embodied as being executed by a hardware decoding processor, or may be executed by a combination of hardware and software modules in a decoding processor. The software module may be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0151] The electronic device can execute the photovoltaic backlight side component identification method provided by the multiple embodiments of the present disclosure, and realize the photovoltaic backlight side component identification device in Figure 3 The functions of the embodiments shown in the figure, or the photovoltaic backlight side component fault detection method provided by the multiple embodiments of the present disclosure, and implemented as a photovoltaic backlight side component fault detection device in Figure 4 The functions of the illustrated embodiments will not be described in detail in the embodiments of the present disclosure.

[0152] The embodiments of the present disclosure also propose a computer-readable storage medium, which stores one or more programs, and the one or more programs include instructions. When the instructions are executed by an electronic device including multiple application programs, the electronic device can execute the photovoltaic backlight side component identification method or photovoltaic backlight side component fault detection method provided by multiple embodiments of the present disclosure.

[0153] Those skilled in the art will appreciate that the embodiments of the present disclosure may be provided as methods, systems, or computer program products. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0154] The present disclosure is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present disclosure. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0155] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0156] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0157] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0158] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0159] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape, disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0160] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity or device including the element.

[0161] It will be appreciated by those skilled in the art that the embodiments of the present disclosure may be provided as methods, systems or computer program products. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0162] The above are only embodiments of the present disclosure and are not intended to limit the present disclosure. For those skilled in the art, the present disclosure may have various modifications and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present disclosure shall be included in the scope of the claims of the present disclosure.

Claims

1. A method for identifying photovoltaic backlight side components, characterized in that: The method comprises: Acquire the image of the component to be identified on the backlight side of the photovoltaic panel; Extracting key points of components in the image of the component to be identified; Using a pre-trained classifier, processing the patch feature data of any key point to be processed among the key points of the component to obtain a category prediction result; The components in the to-be-identified component image are identified according to the category prediction results of the key points of each component and the corresponding coordinate data.

2. The method according to claim 1, characterized in that The step of extracting key points of components in the image of the component to be identified comprises: For any unclassified pixel point on the image of the component to be identified, randomly determine a first neighboring pixel point and a second neighboring pixel point on a circle with the unclassified pixel point as the center and a first preset distance as the radius, and a connecting line passing through the center of the circle; Calculating the grayscale value differences between the unclassified pixel and the first neighboring pixel and the second neighboring pixel, respectively, to obtain a first grayscale difference and a second grayscale difference; If the first grayscale difference and the second grayscale difference meet a first preset condition, and the actual number of times that the first grayscale difference and the second grayscale difference meet the first preset condition is greater than or equal to the first preset number, the unclassified pixel point is determined as the component key point; the first preset condition is: the first grayscale difference and / or the second grayscale difference is greater than a first preset threshold; If the first grayscale difference and the second grayscale difference meet the first preset condition, and the actual number of times is less than the first preset number of times, return to randomly determining a first neighboring pixel point and a second neighboring pixel point on the circumference of a circle with the unclassified pixel point as the center and the first preset distance as the radius, and a line passing through the center of the circle.

3. The method according to claim 2, characterized in that The method further comprises: Performing multi-scale Gaussian convolution operations and downsampling operations on the image of the component to be identified to obtain a first set of images to be processed having a Gaussian pyramid structure; The first criticality score of any candidate key point among the key points of the component is calculated using the following formula: Wherein, score(m) represents the first criticality score of the candidate key point m, and n represents the number of elements in the first set of images to be processed. represents the gray value of the candidate key point m in the first image to be processed at the i-th layer of the Gaussian pyramid, represents the gray value of the third neighboring pixel corresponding to the candidate key point m in the first image to be processed at the i-th layer of the Gaussian pyramid, represents the gray value of the fourth neighboring pixel corresponding to the candidate key point m in the first to-be-processed image of the i-th layer of the Gaussian pyramid, d is a unit vector, and α represents the angle between the line between the third neighboring pixel and the fourth neighboring pixel and the horizontal diameter of a circle with the candidate key point m as the center and the first preset length R as the radius; According to each of the first criticality scores, each of the component key points is screened, and the screened component key points are used as the component key points of the component image to be identified.

4. The method according to claim 1, characterized in that: The method of using a pre-trained classifier to process the patch feature data of any key point to be processed among the key points of the component to obtain a category prediction result includes: Acquire a face patch to be classified with the key point to be processed as the center, and perform feature extraction on the face patch to be classified based on a random Ferns algorithm to obtain feature data of the face patch to be classified; The pre-trained classifier is used to process the face feature data to be classified to obtain a corresponding category prediction result; the pre-trained classifier is a semi-naive Bayes classifier.

5. The method according to claim 1, characterized in that: The pre-trained classifier is obtained according to the following method: Obtain a data set; the features of the samples in the data set are the sample key point patch feature data, and the labels of the samples are the corresponding key point categories; The initial classifier is trained using the data set to obtain the pre-trained classifier.

6. The method according to claim 5, characterized in that The method further comprises: Acquire multiple sample component images; the components in each of the sample component images are the components to be identified; For any pixel to be classified on any image of the plurality of sample component images, randomly determine a first sample neighboring pixel and a second sample neighboring pixel whose connecting line passes through the center of a circle with the pixel to be classified as the center and a second preset distance as the radius; Calculate the grayscale value differences between the pixel to be classified and the first sample neighboring pixel points and the second sample neighboring pixel points to obtain a first sample grayscale difference value and a second sample grayscale difference value; If the first sample grayscale difference and the second sample grayscale difference meet the preset second condition, and the number of times the first sample grayscale difference and the second sample grayscale difference meet the preset second condition is greater than or equal to the preset second number, the pixel to be classified is determined as a key point; the preset second condition is: the first sample grayscale difference and / or the second sample grayscale difference is greater than a preset second threshold; If the first sample grayscale difference and the second sample grayscale difference satisfy the preset second condition, and the number of times the first sample grayscale difference and the second sample grayscale difference satisfy the preset second condition is less than the preset second number, return to the first sample neighboring pixel point and the second sample neighboring pixel point randomly determined on the circumference of a circle with the pixel point to be classified as the center and the second preset distance as the radius, and the line passing through the center of the circle; According to the determination result of whether each pixel point on each sample component image is a key point, the pixel points at each position are screened to obtain a target key point set; The data set is generated according to the patch feature data of each key point in the target key point set and the corresponding category.

7. The method according to claim 6, characterized in that The step of screening the pixel points at each position according to the determination result of whether each pixel point on each sample component image is a key point to obtain a target key point set includes: According to the determination result of whether each pixel point on each sample component image is a key point, the number of times the pixel point at the same position is determined as a key point is calculated to obtain multiple target numbers; The pixel points whose target times are ranked before a preset number are selected to form the target key point set.

8. The method according to claim 6, characterized in that The step of screening the pixel points at each position according to the determination result of whether each pixel point on each sample component image is a key point to obtain a target key point set includes: For a pixel point to be evaluated at any position, a target image is screened out from the plurality of sample component images; in the target image, the pixel point to be evaluated is a key point; Performing a multi-scale Gaussian convolution operation and a downsampling operation on any image in the target images to obtain a second set of images to be processed having a Gaussian pyramid structure; The second critical score of the pixel to be evaluated is calculated using the following formula: Wherein, score(m1) represents the second critical score of the pixel m1 to be evaluated, n1 represents the number of elements in the second image set to be processed, represents the gray value of the pixel m1 to be evaluated in the second processed image at the jth layer of the Gaussian pyramid, represents the gray value of the third sample neighboring pixel corresponding to the pixel m1 to be evaluated in the second image to be processed at the jth layer of the Gaussian pyramid, represents the gray value of the fourth sample neighboring pixel corresponding to the pixel m1 to be evaluated in the second to-be-processed image of the j-th layer of the Gaussian pyramid, d is a unit vector, α1 represents the angle between the connecting line between the third sample neighboring pixel and the fourth sample neighboring pixel and the horizontal diameter of a circle with the pixel m1 to be evaluated as the center and the second preset length R1 as the radius; Adding the second criticality scores to obtain a total score of the pixel to be evaluated; According to each of the total scores, each of the pixels to be evaluated is screened to obtain the target key point set.

9. An electronic device, comprising: processor; as well as A memory arranged to store computer executable instructions, wherein when the executable instructions are executed, the processor is caused to perform the steps of the method according to any one of claims 1 to 8.

10. A computer-readable storage medium storing one or more programs, characterized in that: When the one or more programs are executed by an electronic device including a plurality of application programs, the electronic device executes the steps of the method as claimed in any one of claims 1 to 8.

Images