A photovoltaic industry visual detection method and system based on a 5G edge cloud

Abstract

This invention relates to the field of industrial visual inspection technology, and more particularly to a photovoltaic industrial visual inspection method and system based on 5G edge cloud. This method forms a closed loop between the client and server. Monitoring is performed on the server side. If the number of NG (non-compliant) products exceeds a set threshold within a preset time period, the data returned from the client is processed, and the detection model is retrained. The retrained detection model is then sent back to the client to update the detection model in the client. This reduces manual intervention, making implementation and maintenance personnel more efficient. The client only needs to use the model inference service, reducing the client's load and simplifying its functionality.

Description

Technical Field

[0001] This invention relates to the field of industrial visual inspection technology, and in particular to a photovoltaic industrial visual inspection method and system based on 5G edge cloud. Background Technology

[0002] In industrial photovoltaic product manufacturing lines, quality inspections are required at multiple stages of production, including but not limited to photovoltaic panel inspection and glue filling inspection. Defect detection at each stage is crucial. Currently, there are two main visual inspection solutions for the entire photovoltaic industry: manual inspection and AI-based inspection.

[0003] In the photovoltaic product manufacturing production line, there are many workstations where manual inspection is required. Defect detection is needed at each workstation to assess the condition of the product upon arrival.

[0004] For AI-based inspection, visual algorithms are used to detect product defects at every stage of the process. AI inspection includes a manual data collection stage, a model training stage, a product manufacturing stage, and a manual data retrieval stage. Because the product conditions at each workstation are different, there are numerous inspection items to be checked. For example, the photovoltaic panel inspection station needs to check for defects such as short circuits, broken grids, black spots, scratches, gaps, poor soldering, foreign objects, and microcracks; similarly, it needs to check for defects such as insufficient glue, inadequate glue quantity, glue overflow, and overfilling during glue application.

[0005] Manual inspection is inherently subjective, influenced by individual differences in experience, resulting in a high false positive rate and slow cycle time. Furthermore, the inspection items vary at each workstation, with differences in product size and appearance, making manual inspection even more difficult and prone to errors or omissions. In terms of cycle time, manual sampling increases the likelihood of a large number of defective parts being shipped out. AI-based inspection can effectively address the shortcomings of manual inspection. However, the entire AI inspection process is cumbersome, requiring manual intervention for data labeling and cleaning, model training and deployment, and data mining and recycling, leading to extremely complex and difficult maintenance. Therefore, existing industrial photovoltaic product production lines suffer from cumbersome inspection processes and low inspection accuracy. Summary of the Invention

[0006] This invention provides a 5G edge cloud-based visual inspection method and system for the photovoltaic industry, which solves the problems of cumbersome inspection processes and low inspection accuracy in existing industrial photovoltaic product production lines.

[0007] To achieve the above objectives, the present invention employs the following technical solution:

[0008] In a first aspect, the present invention provides a photovoltaic industrial visual inspection method based on 5G edge cloud, comprising:

[0009] The client acquires an image of the product to be inspected, calls the model inference interface to obtain the inspection model, inputs the product image into the inspection model, obtains the output result of the inspection model, and sends the model's input data as feedback data to the server.

[0010] The server receives feedback data from the client, uses a detection model to determine NG data based on the feedback data, and if the number of NG products within a preset time period is higher than a set threshold, the feedback data is processed and the detection model is retrained. The trained detection model is then sent to the client.

[0011] The client updates its own detection model based on the model from the server.

[0012] Optionally, the reflowed data is processed and the detection model is retrained, including:

[0013] Simple sample data in the returned data is filtered out, and hard sample data is obtained by mining hard samples in the returned data based on the amount of information in the image.

[0014] For difficult sample data, data cleaning is performed, and the cleaned data is input into the current detection model. Based on the output of the current detection model, the corresponding annotations are automatically generated to obtain the labeled data.

[0015] The automatically generated annotations are fine-tuned and fitted to the edges of the annotated objects.

[0016] Input the labeled data into the current detection model, calculate the index of the current detection model, and if the index is less than the set threshold, retrain the current detection model based on the labeled data.

[0017] Optionally, hard sample data is obtained by performing hard sample mining on the reflowed data based on the amount of information in the image, including:

[0018] The uncertainty of the sample is calculated based on the target information content. Samples with uncertainty higher than the threshold are filtered out. Then, the number of recovered samples is compressed by perceptual hashing and clustering to mine hard sample data from the returned data.

[0019] Optionally, before inputting the labeled data into the current detection model, the method further includes:

[0020] Review and verify the labeled data, and adjust the automatically generated labels if the labeled data does not meet the preset requirements.

[0021] Optionally, the trained detection model is sent to the client, including:

[0022] The retrained model is converted into a deployment model, and the deployment model is packaged into a service image. The server cloud then automatically distributes the image to the client via 5G to update the detection model in the client.

[0023] Optionally, the results output by the detection model include NG data and OK data. NG data is used to indicate that the product corresponding to the image is a non-conforming product, while OK data is used to indicate that the product corresponding to the image is a conforming product.

[0024] Optionally, when the client sends the model's input data as feedback data to the server, it can upload it via 5G.

[0025] In a second aspect, the present invention provides a photovoltaic industrial visual inspection system based on 5G edge cloud, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described in the first aspect.

[0026] Beneficial effects:

[0027] The photovoltaic industry visual inspection method based on 5G edge cloud provided by this invention forms a closed loop between the client and the server. Monitoring is performed on the server side. If the number of NG (non-compliant) products exceeds a set threshold within a preset time period, the data returned from the client is processed, and the detection model is retrained. The retrained detection model is then sent back to the client to update the detection model in the client. This reduces manual intervention, making implementation and maintenance personnel more efficient. The client only needs to use the model inference service, reducing the client's load and simplifying its functionality. Attached Figure Description

[0028] Figure 1 This is one of the flowcharts of a photovoltaic industrial visual inspection method based on 5G edge cloud, according to a preferred embodiment of the present invention.

[0029] Figure 2 This is a data processing flowchart of a preferred embodiment of the present invention;

[0030] Figure 3 This is a flowchart of the NG monitoring process according to a preferred embodiment of the present invention;

[0031] Figure 4 This is a flowchart of the indicator monitoring process according to a preferred embodiment of the present invention;

[0032] Figure 5 This is a flowchart illustrating the model iteration process of a preferred embodiment of the present invention.

[0033] Figure 6 This is a second flowchart of a photovoltaic industrial visual inspection method based on 5G edge cloud, according to a preferred embodiment of the present invention.

[0034] Figure 7 This is a schematic diagram of cloud-based annotation according to a preferred embodiment of the present invention;

[0035] Figure 8 This is a schematic diagram of cloud-based review according to a preferred embodiment of the present invention;

[0036] Figure 9 This is a preferred embodiment of the photovoltaic panel black spot identification example of the present invention;

[0037] Figure 10 This is a preferred embodiment of the photovoltaic panel black spot identification method of the present invention;

[0038] Figure 11 This is a preferred embodiment of the photovoltaic panel grid breakage identification method of the present invention;

[0039] Figure 12 This is a photovoltaic panel scratch recognition example of a preferred embodiment of the present invention;

[0040] Figure 13 This is a normal example of a photovoltaic panel according to a preferred embodiment of the present invention;

[0041] Figure 14 This is a preferred embodiment of the present invention for identifying glue overflow during glue pouring;

[0042] Figure 15 This is a preferred embodiment of the glue-filling deficiency identification case of the present invention;

[0043] Figure 16 This is an identification example on the glue-filling cap according to a preferred embodiment of the present invention;

[0044] Figure 17 This is a schematic diagram of a client-side single-site mode according to a preferred embodiment of the present invention;

[0045] Figure 18 This is a schematic diagram of the client-cloud interaction mode according to a preferred embodiment of the present invention;

[0046] Figure 19 This is the third flowchart of a photovoltaic industrial visual inspection method based on 5G edge cloud, which is a preferred embodiment of the present invention. Detailed Implementation

[0047] The technical solution of the present invention will be clearly and completely described below. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0048] Unless otherwise defined, the technical or scientific terms used in this invention shall have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Similarly, the terms "an" or "a" and similar terms do not indicate a quantity limitation, but rather indicate the presence of at least one. The terms "connected" or "linked" and similar terms are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up," "down," "left," "right," etc., are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship also changes accordingly.

[0049] Please see Figure 1 This application provides a photovoltaic industrial visual inspection method based on 5G edge cloud, comprising:

[0050] The client acquires an image of the product to be inspected, calls the model inference interface to obtain the inspection model, inputs the product image into the inspection model, obtains the output result of the inspection model, and sends the model's input data as feedback data to the server.

[0051] The server receives feedback data from the client, uses a detection model to determine NG data based on the feedback data, and if the number of NG products within a preset time period is higher than a set threshold, the feedback data is processed and the detection model is retrained. The trained detection model is then sent to the client.

[0052] The client updates its own detection model based on the model from the server.

[0053] In this implementation, the server receives model output data from the client via a 5G communication module, resulting in high transmission speed. The output of the detection model includes NG (Not Acceptable) data and OK (Acceptable) data. NG data indicates that the product corresponding to the image is a non-conforming product, while OK data indicates that the product corresponding to the image is a conforming product.

[0054] The aforementioned 5G edge cloud-based visual inspection method for the photovoltaic industry forms a closed loop between the client and server. Monitoring is performed on the server side. If the number of NG (non-compliant) products exceeds a set threshold within a preset time period, the data returned from the client is processed, and the detection model is retrained. The retrained detection model is then sent back to the client to update the detection model in the client. This reduces manual intervention, making implementation and maintenance personnel more efficient. The client only needs to use the model inference service, reducing the client's load and simplifying its functionality.

[0055] Furthermore, the server processes the received data in the cloud. In one example, the cloud functions include data processing, model monitoring, model iteration, and model service delivery to the client. Data processing is semi-automatic, requiring manual intervention; model monitoring, model iteration, and model service delivery are fully automated. Data processing includes hard sample mining, data cleaning, data label generation, data label fine-tuning, and data review. The data processing flow is as follows: Figure 2 As shown.

[0056] Optionally, the reflowed data is processed and the detection model is retrained, including:

[0057] Simple sample data in the returned data is filtered out, and hard sample data is obtained by mining hard samples in the returned data based on the amount of information in the image.

[0058] For difficult sample data, data cleaning is performed, and the cleaned data is input into the current detection model. Based on the output of the current detection model, the corresponding annotations are automatically generated to obtain the labeled data.

[0059] The automatically generated annotations are fine-tuned and fitted to the edges of the annotated objects.

[0060] Input the labeled data into the current detection model, calculate the index of the current detection model, and if the index is less than the set threshold, retrain the current detection model based on the labeled data.

[0061] In this optional implementation, filtering out duplicate data in the model output data is to filter out simple data in the model output data. In practice, the amount of model output data is very large, so it is necessary to filter out the simple data first. Then, the filtered samples are cleaned to maintain a certain sample balance.

[0062] Model monitoring includes NG monitoring and indicator monitoring. The NG monitoring process is as follows: Figure 3 As shown, the indicator monitoring process is as follows: Figure 4 As shown.

[0063] Optionally, hard sample data is obtained by performing hard sample mining on the reflowed data based on the amount of information in the image, including:

[0064] The uncertainty of the sample is calculated based on the target information content. Samples with uncertainty higher than the threshold are filtered out. Then, the number of recovered samples is compressed by perceptual hashing and clustering to mine hard sample data from the returned data.

[0065] Optionally, before inputting the labeled data into the current detection model, the method further includes:

[0066] Review and verify the labeled data, and adjust the automatically generated labels if the labeled data does not meet the preset requirements.

[0067] In this optional implementation, by reviewing and verifying the labeled data, the labels can be reviewed more meticulously to ensure that there are no problems. The review and verification can be carried out manually.

[0068] Optionally, the trained detection model is sent to the client, including:

[0069] The retrained model is converted into a deployment model, and the deployment model is packaged into a service image. The server cloud then automatically distributes the image to the client via 5G to update the detection model in the client.

[0070] Model training refers to continuously learning and optimizing the trained model based on new data. The trained model is then converted into a deployment model, such as ONNX or TensorRT. After all the above processes are completed, the model can be packaged into a service image, and automatically distributed from the cloud to clients via 5G, completing the model service update. The model iteration process is as follows: Figure 5 As shown.

[0071] Please see Figure 6 The client-side process includes:

[0072] Acquire images of the product to be inspected;

[0073] Call the model inference interface to obtain the detection model, input the product image into the detection model, and obtain the model output data output by the detection model;

[0074] The model input data is uploaded to the server via 5G;

[0075] Receive the updated detection model from the server.

[0076] In a specific example, the client performs model inference and data recycling. The model inference interface is integrated into the already developed software, and the data recycling enables fully automated 5G data upload to the cloud.

[0077] It is worth emphasizing that the drawback of AI detection is that each step requires human intervention, resulting in huge maintenance costs and low efficiency. Compared with AI detection solutions, the method in this application can solve the problems of fully automatic data collection to the cloud, fully automatic model iteration in the cloud, and fully automatic delivery of model services to the client. Furthermore, it utilizes 5G communication, transforming the client-side single-site mode into a client-cloud interactive mode.

[0078] In one feasible implementation, please refer to Figure 7 When annotating in the cloud, you can click the "Submit for Review" button in the upper right corner after completing the annotation. Please see below. Figure 8When conducting cloud-based data review, you can select the "Approved" or "Rejected" button at the top.

[0079] In another feasible implementation, taking a photovoltaic panel defect as an example, the method of this application is further verified. Among them, Figure 9 This is a case study on identifying black spots on photovoltaic panels. Figure 10 This is a case study on black spot identification on photovoltaic panels. Figure 11 This is a case study on photovoltaic panel grid breakage identification. Figure 12 This is a case study on scratch identification of photovoltaic panels. Figure 13 This is a normal case for photovoltaic panels. Figures 9 to 13 In the image, the left side is the original image, the middle side is the visualized heatmap output by the model, and the right side is the category label map output by the model.

[0080] In another feasible implementation, the method of this application is further verified using a glue-pouring defect as an example. Figure 14 This is a case study for identifying glue overflow during glue dispensing. Figure 15 This is a case study for identifying insufficient glue during glue application. Figure 16 For identifying cases on the glue-filling cap, since only detected defects are visualized in the original image, normal cases cannot be displayed. Figures 14 to 16 In the image, the left side shows the original image, and the right side shows the visualization output from the model.

[0081] Please see Figure 17-18 This application proposes a closed-loop visual inspection process for the photovoltaic industry using 5G+edge cloud technology, transforming the process from a single-site client mode to an interactive mode between the client and the cloud. Fully automated data feedback and model service delivery both utilize 5G communication, while semi-automatic data processing and fully automated model iteration are performed in the cloud. This significantly reduces manual intervention, making implementation and maintenance personnel more efficient; it also greatly reduces the client's workload, simplifying client functionality, requiring only the use of model inference services.

[0082] In summary, please refer to Figure 19 This application presents a 5G edge cloud-based visual inspection method for the photovoltaic industry, forming a closed loop between the client and server. Monitoring is performed on the server side. If the number of NG (non-compliant) products exceeds a set threshold within a preset time period, the returned data from the client is processed, and the detection model is retrained. The retrained detection model is then sent back to the client to update the detection model in the client. This reduces manual intervention, making implementation and maintenance more efficient. The client only needs to use the model inference service, reducing the client's load and simplifying its functionality.

[0083] This application also provides a photovoltaic industrial visual inspection system based on 5G edge cloud, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the above-described method. This photovoltaic industrial visual inspection system based on 5G edge cloud can implement various embodiments of the photovoltaic industrial visual inspection method based on 5G edge cloud and achieve the same beneficial effects, which will not be elaborated here.

[0084] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A photovoltaic industrial visual inspection method based on 5G edge cloud, characterized in that, include: The client acquires an image of the product to be detected, calls the model inference interface to obtain a detection model, inputs the product image into the detection model, obtains the output result of the detection model, and sends the input data of the model as feedback data to the server. The server receives feedback data from the client, uses a detection model to determine NG data based on the feedback data, and if the amount of NG products within a preset time period is higher than a set threshold, the feedback data is processed and the detection model is retrained, and the trained detection model is sent to the client. The client updates its own detection model based on the model from the server. The process of processing the returned data and retraining the detection model includes: Simple sample data in the returned data is filtered out, and hard sample data is obtained by mining hard samples from the returned data based on the amount of information in the image. The difficult sample data is cleaned, and the cleaned data is input into the current detection model. Based on the output of the current detection model, the corresponding annotations are automatically generated to obtain the labeled data. The automatically generated annotations are fine-tuned and fitted to the edges of the annotated objects. The labeled data is input into the current detection model to calculate the index of the current detection model. If the index is less than a set threshold, the current detection model is retrained based on the labeled data.

2. The photovoltaic industrial visual inspection method based on 5G edge cloud according to claim 1, characterized in that, The process of obtaining hard sample data by mining the reflowed data based on the amount of information in the image includes: The uncertainty of the sample is calculated based on the target information content. Samples with uncertainty higher than the threshold are filtered out. Then, the number of recovered samples is compressed by perceptual hashing and clustering to mine hard sample data from the returned data.

3. The photovoltaic industrial visual inspection method based on 5G edge cloud according to claim 1, characterized in that, Before inputting the labeled data into the current detection model, the method further includes: The annotation data is reviewed and verified. If the annotation data does not meet the preset requirements, the automatically generated annotations are adjusted.

4. The photovoltaic industrial visual inspection method based on 5G edge cloud according to claim 1, characterized in that, Sending the trained detection model to the client includes: The retrained model is converted into a deployment model, which is then packaged into a service image and automatically distributed to the client via 5G from the server cloud to update the detection model in the client.

5. The photovoltaic industrial visual inspection method based on 5G edge cloud according to claim 1, characterized in that, The detection model outputs NG data and OK data. NG data indicates that the product corresponding to the image is a defective product, and OK data indicates that the product corresponding to the image is a qualified product.

6. The photovoltaic industrial visual inspection method based on 5G edge cloud according to claim 1, characterized in that, When the client transmits the input data of the model as feedback data to the server, it does so via 5G.

7. A photovoltaic industrial visual inspection system based on 5G edge cloud, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method described in any one of claims 1 to 6.

Images