Defect detection method and system based on local and edge cloud joint detection

Abstract

This invention provides a defect detection method and system based on joint detection by local and edge cloud, belonging to the field of defect detection technology. The method includes: a local terminal performing preliminary detection on a sample image; if the confidence level of all preliminary detection results is greater than or equal to a first threshold, the preliminary detection results are reported; otherwise, the sample is uploaded to an edge cloud server. The edge cloud server uses multiple detection schemes to jointly detect the sample, and determines the joint decision result and its confidence level based on the majority rule principle. If the confidence level of the joint decision result meets the requirements, it is reported; otherwise, it is pushed to a manual decision-making module for re-inspection, and finally, the manual re-inspection result is reported. This invention effectively improves the accuracy and efficiency of defect detection through the collaborative work of the local terminal, edge cloud server, and manual decision-making module, and is suitable for image defect detection scenarios of target products.

Description

Technical Field

[0001] This invention relates to the field of defect detection technology, and in particular to a defect detection method and system based on joint detection of local and edge cloud. Background Technology

[0002] As industrial products become increasingly sophisticated, the demands on the precision of visual inspection are also rising, which to some extent reduces inspection efficiency and accuracy. The most direct and rapid way to improve inspection efficiency is to increase the hardware configuration and quantity of inspection equipment. However, this approach not only increases costs for enterprises but also occupies a significant amount of space. Even so, this solution cannot solve the fundamental problem and cannot increase costs and product size indefinitely.

[0003] To solve the above problems, the following aspects are typically addressed from the software side:

[0004] The approach combines traditional and AI algorithms. Traditional algorithms, using predefined defect features and rules, are effective for handling defects with clear structures and well-defined boundaries, but they have limitations and lack control over unknown defects. AI algorithms, on the other hand, handle defects such as cracks, bubbles, and impurities that are difficult to identify using simple features. Furthermore, AI learns defect characteristics during training, allowing the model to adapt to a certain degree of unfamiliar defects, but its generalization ability is easily limited by the diversity of defect samples. Therefore, even with this combination, it is impossible to effectively intercept new defects, and the risk of missed detections still exists.

[0005] Manual iterative models reduce missed detections. AI algorithms, compared to traditional algorithms, possess stronger adaptability and robustness when facing new defects. However, this advantage relies on a large, comprehensive, and diverse set of defect samples. In the early stages of a project or during periods of process instability, this condition is often difficult to meet, with the unpredictable emergence of new defect types posing the biggest obstacle. In such situations, teams must allocate manpower for multiple rounds of sample collection and cleaning, and invest significant computational resources in training and inference. However, this process is typically time-consuming, leading to a significant problem: if a product switch occurs before the model is fully optimized, previous iterations may become meaningless and must be restarted. This makes iterative progress difficult to keep pace with product switchovers, resulting in inefficiency and wasted resources.

[0006] Anomaly detection reduces reliance on defect samples. It relies less on manual inspection and defect sample collection, effectively identifying potential defects early, improving detection efficiency, and reducing the risk of human error. Furthermore, anomaly detection demonstrates good practicality in intercepting novel defects. However, real-time anomaly detection algorithms typically require significant computing resources. Especially in industrial environments, this resource requirement can significantly increase operating costs when dealing with high-precision, high-resolution image processing. Therefore, when implementing an anomaly detection system, companies need to comprehensively consider its technological advantages and cost-effectiveness to achieve optimal detection results.

[0007] While existing technologies each have their advantages, they still fall short of meeting the practical requirements of high efficiency, high precision, and high accuracy in industrial testing scenarios. This highlights the need for further innovation and optimization within the current technological framework to better adapt to evolving industrial demands. Summary of the Invention

[0008] This invention provides a defect detection method and system based on joint detection of local and edge cloud, to solve at least one defect in the prior art.

[0009] In a first aspect, the present invention provides a defect detection method based on joint detection of local and edge cloud, comprising:

[0010] The local terminal acquires the sample to be detected; the sample to be detected includes at least one image of the target product;

[0011] The local terminal performs defect detection on each image in the sample, and obtains all preliminary detection results and the confidence level of each preliminary detection result;

[0012] If the confidence level of all preliminary detection results is greater than or equal to the first threshold, the local terminal sends the preliminary detection results of the sample to the result reporting module of the local terminal; otherwise, the sample is sent to the edge cloud server.

[0013] The edge cloud server uses multiple detection schemes to jointly detect each image in the sample, and obtains all joint decision results and the confidence level of each joint decision result based on the principle of majority rule;

[0014] If the edge cloud server determines that the confidence level of all joint decision results is greater than or equal to the second threshold, it will send the joint decision results of the sample to the result reporting module of the local terminal; otherwise, it will push the sample to the manual decision-making module.

[0015] The manual decision-making module is used to manually re-examine the samples, obtain the manual re-examination results, and send the manual re-examination results to the result reporting module.

[0016] According to the defect detection method based on joint detection of local and edge cloud provided by the present invention, the defect detection method adopts a hierarchical detection strategy, specifically as follows:

[0017] The first detection level, executed by the local terminal, is used to perform preliminary detection on the samples, screen samples that meet the condition that the confidence level of all preliminary detection results is greater than or equal to the first threshold, and send the preliminary detection results of samples that meet the condition to the result reporting module, while uploading samples that do not meet the condition to the edge cloud server.

[0018] The second detection layer, executed by the edge cloud server, is used to perform multi-algorithm joint detection on samples from the first detection layer, filter samples that meet the condition that the confidence of all joint decision results is greater than or equal to the second threshold, and send the joint decision results of samples that meet the condition to the result reporting module, and send samples that do not meet the condition to the manual decision module.

[0019] The third detection level, executed by the manual decision-making module, is used to manually re-examine the samples from the second detection level and send the manual re-examination results to the result reporting module.

[0020] The defect detection method based on joint detection of local and edge cloud provided by the present invention further includes: collecting samples for model training in defect detection, specifically:

[0021] First, collect all samples from manual re-examinations;

[0022] Second, for samples that have undergone the second detection level, the detection results of the first detection level are compared with the detection results of each detection scheme in the second detection level, and the result in which most detection results are consistent is determined as the final judgment result; the final judgment result is compared with the detection results of the first detection level, and the sample is retained if the results are different.

[0023] The defect detection method based on joint detection of local and edge cloud provided by the present invention further includes, after collecting samples for model training for defect detection, using the collected samples to train and test the model for defect detection on the edge cloud server; and sending the trained and tested model to the local terminal.

[0024] The defect detection method based on joint detection of local and edge cloud provided by the present invention utilizes multiple detection schemes to jointly detect each image in the sample, and obtains all joint decision results and the confidence level of each joint decision result based on the principle of majority rule. Specifically, it includes: using multiple detection schemes to detect the uploaded sample and obtaining detection results; determining the joint decision result based on the detection results of multiple detection schemes according to the principle of majority rule; and determining the confidence level of the joint decision result based on the confidence level of the detection results that are consistent with the majority of detection results.

[0025] According to the defect detection method based on joint detection of local and edge cloud provided by the present invention, the confidence level of the joint decision result is determined based on the confidence level of the detection results with consistent majority detection results, including: determining the weighting weight based on the historical detection accuracy of the detection scheme with consistent majority detection results; and determining the confidence level of the joint decision result based on the confidence level of the detection results with consistent majority detection results and the weighting weight.

[0026] The defect detection method based on joint detection of local and edge cloud provided by the present invention further includes: uploading the received detection results to a preset client through a result reporting module.

[0027] Secondly, the present invention also provides a defect detection system based on joint detection of local and edge cloud, comprising:

[0028] An image acquisition module installed on a local terminal is used to acquire a sample to be detected; the sample to be detected includes at least one image of the target product.

[0029] A basic general-purpose defect detection module is set up on the local terminal to perform defect detection on each image in the sample, and obtain all preliminary detection results and the confidence level of each preliminary detection result;

[0030] A first result filtering module is set up on the local terminal to send the preliminary detection results of the sample to the result reporting module of the local terminal if the confidence level of all preliminary detection results is greater than or equal to the first threshold; otherwise, the sample is sent to the edge cloud server.

[0031] The multi-detection scheme joint decision module, set up on the edge cloud server, is used to perform joint detection on each image in the sample using multiple detection schemes, and obtain all joint decision results and the confidence level of each joint decision result based on the principle of majority rule.

[0032] The second result filtering module, set up on the edge cloud server, is used to send the joint decision results of the sample to the result reporting module of the local terminal if the confidence level of all joint decision results is greater than or equal to the second threshold; otherwise, the sample is pushed to the manual decision module.

[0033] The manual decision-making module is used to manually re-examine the samples, obtain the manual re-examination results, and send the manual re-examination results to the result reporting module.

[0034] According to the present invention, a defect detection system based on joint detection of local and edge cloud is provided. The defect detection system adopts a hierarchical detection strategy, specifically as follows:

[0035] The first detection level, executed by the local terminal, is used to perform preliminary detection on the samples, screen samples that meet the condition that the confidence level of all preliminary detection results is greater than or equal to the first threshold, and send the preliminary detection results of samples that meet the condition to the result reporting module, while uploading samples that do not meet the condition to the edge cloud server.

[0036] The second detection layer, executed by the edge cloud server, is used to perform multi-algorithm joint detection on samples from the first detection layer, filter samples that meet the condition that the confidence of all joint decision results is greater than or equal to the second threshold, and send the joint decision results of samples that meet the condition to the result reporting module, and send samples that do not meet the condition to the manual decision module.

[0037] The third detection level, executed by the manual decision-making module, is used to manually re-examine the samples from the second detection level and send the manual re-examination results to the result reporting module.

[0038] A defect detection system based on joint detection of local and edge cloud provided by the present invention further includes: a sample collection module; the sample collection module is used to collect samples for training the defect detection model, specifically including:

[0039] First, collect all samples from manual re-examinations;

[0040] Second, for samples that have undergone the second detection level, the detection results of the first detection level are compared with the detection results of each detection scheme in the second detection level, and the result in which most detection results are consistent is determined as the final judgment result; the final judgment result is compared with the detection results of the first detection level, and the sample is retained if the results are different.

[0041] This invention provides a defect detection method and system based on joint local and edge cloud detection. It proposes a distributed architecture for layered edge cloud detection, which distributes different functions and processing layers across different terminal devices and servers to improve system flexibility and scalability. This joint detection deployment method effectively utilizes existing resources and improves overall detection efficiency, making it highly suitable for industrial environments. It offers the following advantages:

[0042] (1) Improve detection efficiency: By dividing the detection task into different levels, high-confidence samples can be quickly screened out, thereby reducing the workload of subsequent processing.

[0043] (2) Enhanced accuracy: Each level adopts a detection method suitable for its complexity, thereby improving the detection accuracy in each level, especially for samples that are difficult to identify.

[0044] (3) Effective resource utilization: Lightweight algorithms can quickly handle most common problems, while complex algorithms focus on solving more challenging detection tasks, thereby optimizing the use of computing resources.

[0045] (4) Reduced false positive rate: The layered strategy ensures that in-depth detection is only performed when necessary, reducing false positives caused by over-checking.

[0046] (5) Flexible response to changes: Based on the test results at different levels, the subsequent processing methods can be flexibly adjusted to adapt to different scenarios and product requirements.

[0047] (6) Facilitates improvement and feedback: By collecting detection data at different levels, the model and detection strategy can be continuously optimized to improve the overall performance of the system.

[0048] (7) Promote the rationality of human intervention: manual re-examination is only for the most controversial or uncertain samples to ensure that human resources are used efficiently.

[0049] (8) High adaptability: Layered detection method can be applied to a variety of industrial detection scenarios, especially in fields with high requirements for real-time performance and accuracy. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0051] Figure 1 This is a schematic diagram of the deployment architecture based on cloud-edge layered detection provided by the present invention;

[0052] Figure 2 This is a schematic diagram of the functional architecture of cloud-edge layered detection provided by the present invention;

[0053] Figure 3 This is a flowchart illustrating the defect detection method based on joint detection of local and edge cloud provided by the present invention.

[0054] Figure 4 This is a flowchart illustrating the hierarchical detection strategy provided by the present invention;

[0055] Figure 5 This is a flowchart of the sample collection strategy provided by the present invention. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0057] It should be noted that, in the description of the embodiments of the present invention, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0058] The terms "first," "second," etc., used in this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more.

[0059] Figure 1 This is a schematic diagram of the deployment architecture based on cloud-edge layered detection provided by the present invention. One computing server can connect to N local terminals. One edge cloud can connect to multiple computing servers.

[0060] The server is a high-performance computer or computer cluster that can receive decision requests from local terminals (such as computers), perform further verification using more complex and resource-intensive detection methods, and send the decision results back to the local terminal. It also includes comprehensive model-related tasks (model training, model validation, model inference), comprehensive data-related tasks (storage of outlier data, data preview, data annotation), comprehensive algorithm scheme management tasks (scheme editing and management), and load balancing tasks (concurrent training of multiple models, concurrent processing of multiple decision tasks).

[0061] Network transmission can be achieved through switches, routers, or the cloud.

[0062] The local terminal only needs to meet the performance requirements for model inference, using a low-end graphics card or an embedded terminal. The local terminal performs preliminary inferences on the input samples. When there is doubt or uncertainty in the preliminary detection results, the terminal computer can upload samples to the edge cloud computing server and push a decision request, while simultaneously receiving the decision results returned from the edge cloud computing server. The terminal device also supports the function of actively acquiring images to be detected from a camera or a specific location.

[0063] For a specific terminal and edge computing server, the corresponding functional flowchart architecture is as follows: Figure 2 As shown, Figure 2 This is a schematic diagram of the functional architecture of cloud-edge layered detection provided by the present invention, which includes the following modules:

[0064] (1) Image acquisition module set on the local terminal (i.e., detection terminal)

[0065] The image acquisition module is used to acquire the sample to be inspected; the sample to be inspected includes at least one image of the target product; wherein, the target product can be an AOI display panel, 3C products, etc.

[0066] Specifically, this module is responsible for actively acquiring images of samples with defects to be detected from a camera or a specific location and storing them on a local terminal.

[0067] (2) Basic general defect detection module set on local terminal

[0068] When performing defect detection, an image must have at least one detection result. Each detection result can include information such as the specific defect type and the corresponding defect location.

[0069] The basic general defect detection module is used to perform defect detection on each image in the sample, and obtain all preliminary detection results and the confidence level of each preliminary detection result.

[0070] The basic general-purpose defect detection module employs a lightweight detection algorithm designed to quickly obtain preliminary detection results, thereby saving time and improving efficiency. It can identify all anomalies and defects and provide confidence scores for the detection results.

[0071] (3) Set the first result filtering module on the local terminal

[0072] The first result filtering module is used to send the preliminary detection results of the sample to the result reporting module of the local terminal when it is determined that the confidence level of all preliminary detection results is greater than or equal to the first threshold; otherwise (when it is determined that the confidence level of a certain preliminary detection result is less than the first threshold), the sample is sent to the edge cloud server.

[0073] (4) Multi-detection scheme joint decision-making module set on edge cloud server (i.e., server)

[0074] The multi-detection scheme joint decision module is used to perform joint detection on each image in the sample using multiple detection schemes, and obtain all joint decision results and the confidence level of each joint decision result based on the principle of majority rule;

[0075] (5) The second result filtering module set on the edge cloud server

[0076] The second result filtering module is used to send the joint decision results of the sample to the result reporting module of the local terminal when it is determined that the confidence level of all joint decision results is greater than or equal to the second threshold; otherwise (when it is determined that the confidence level of a certain joint decision result is less than the second threshold), the sample is pushed to the manual decision-making module.

[0077] (6) Manual decision-making module

[0078] The manual decision-making module is used to manually re-examine the samples, obtain the manual re-examination results, and send the manual re-examination results to the result reporting module.

[0079] The human decision-making module can be a human-computer interaction client, which can realize data transmission with the edge cloud server and can also perform tasks such as labeling sample data.

[0080] (7) Result reporting module set on local terminal

[0081] The received detection results are uploaded to the preset client through the result reporting module.

[0082] Specifically, this module is located at the terminal and is responsible for receiving the result data from the basic general defect detection module, the multi-detection scheme joint decision-making module, and the human decision-making module, and reporting it to the customer's MES system for subsequent processing and analysis.

[0083] Based on the above embodiments, the defect detection system adopts a hierarchical detection strategy, specifically as follows:

[0084] The first detection level, executed by the local terminal, is used to perform preliminary detection on the samples, screen samples that meet the condition that the confidence level of all preliminary detection results is greater than or equal to the first threshold, and send the preliminary detection results of samples that meet the condition to the result reporting module, while uploading samples that do not meet the condition to the edge cloud server.

[0085] The second detection layer, executed by the edge cloud server, is used to perform multi-algorithm joint detection on samples from the first detection layer, filter samples that meet the condition that the confidence of all joint decision results is greater than or equal to the second threshold, and send the joint decision results of samples that meet the condition to the result reporting module, and send samples that do not meet the condition to the manual decision module.

[0086] The third detection level, executed by the manual decision-making module, is used to manually re-examine the samples from the second detection level and send the manual re-examination results to the result reporting module.

[0087] Optionally, the system provided by the present invention further includes: a sample collection module;

[0088] The sample collection module is used to collect samples for training the defect detection model, specifically including:

[0089] First, collect all samples from manual re-examinations;

[0090] Second, for samples that have undergone the second detection level, the detection results of the first detection level are compared with the detection results of each detection scheme in the second detection level, and the result in which most detection results are consistent is determined as the final judgment result; the final judgment result is compared with the detection results of the first detection level, and the sample is retained if the results are different.

[0091] On the other hand, Figure 3 This is a flowchart illustrating the defect detection method based on joint local and edge cloud detection provided by the present invention, as shown below. Figure 3 As shown, including but not limited to the following steps:

[0092] Step 301: The local terminal acquires the sample to be detected; the sample to be detected includes at least one image of the target product.

[0093] This step is the beginning of the entire testing process, ensuring that suitable samples are available for subsequent testing.

[0094] Step 302: The local terminal performs defect detection on each image in the sample, and obtains all preliminary detection results and the confidence level of each preliminary detection result.

[0095] The initial detection here may be based on a lightweight detection algorithm / model pre-configured on the local terminal to identify obvious defect features, and the confidence level is used to measure the reliability of the detection results.

[0096] Step 303: If the confidence level of all preliminary detection results is greater than or equal to the first threshold, the local terminal sends the preliminary detection results of the sample to the result reporting module of the local terminal; otherwise, the sample is sent to the edge cloud server.

[0097] The local terminal determines whether the confidence level of all preliminary detection results is greater than or equal to the first threshold.

[0098] If the conditions are met, the preliminary detection results of the sample are sent to the result reporting module of the local terminal. This means that the local terminal is capable of processing and reporting some high-confidence detection results, reducing reliance on edge cloud servers and improving detection efficiency.

[0099] If the conditions are not met, the sample is sent to the edge cloud server. This is to leverage the more powerful computing capabilities and more sophisticated detection schemes of the edge cloud server to process samples that cannot be accurately judged by the local terminal.

[0100] Step 304: The edge cloud server uses a multi-detection scheme to perform joint detection on each image in the sample, and obtains all joint decision results and the confidence level of each joint decision result based on the principle of majority rule.

[0101] Edge cloud servers utilize multiple detection schemes to jointly detect each image in the sample. These multiple detection schemes may include different types of detection algorithms, such as deep learning-based defect detection models and detection methods based on traditional image feature extraction. By combining multiple schemes, potential defects can be discovered more comprehensively.

[0102] Based on the principle of majority rule, all joint decision-making results and the confidence level of each joint decision-making result are obtained. This voting mechanism helps to synthesize the results of multiple alternative tests, improving the accuracy and reliability of decision-making.

[0103] Step 305: If the edge cloud server determines that the confidence level of all joint decision results is greater than or equal to the second threshold, it sends the joint decision results of the sample to the result reporting module of the local terminal; otherwise, it pushes the sample to the manual decision-making module.

[0104] The edge cloud server determines whether the confidence level of all joint decision results is greater than or equal to the second threshold.

[0105] If the conditions are met, the joint decision results of the samples are sent to the result reporting module on the local terminal. This ensures that most samples processed by the edge cloud server and whose results are relatively reliable can be reported appropriately.

[0106] If the conditions are not met, the sample will be pushed to the human decision-making module. This is because samples with still low confidence levels may require human intervention for further evaluation to ensure the final accuracy and quality of the test.

[0107] Step 306: Use the manual decision-making module to manually re-examine the sample, obtain the manual re-examination results, and send the manual re-examination results to the result reporting module.

[0108] The human decision-making module is used to manually re-examine the samples. Human re-examination can give full play to human observation and judgment, and provide final confirmation for situations that are difficult for machine testing to determine.

[0109] Obtain the results of manual re-inspection and send them to the result reporting module. This step ensures that all samples that have undergone the testing process have final test results, which can be aggregated and reported to provide a basis for subsequent quality control, data analysis, etc.

[0110] The defect detection method based on joint detection of local and edge cloud provided by this invention combines the advantages of local terminals and edge cloud servers. Through reasonable traffic distribution and multi-level detection, it improves the accuracy, efficiency and reliability of defect detection.

[0111] Figure 4 This is a flowchart illustrating the hierarchical detection strategy provided by the present invention. (Refer to...) Figure 4 As shown, the defect detection method provided by this invention adopts a hierarchical detection strategy, specifically including:

[0112] The first detection level, executed by the local terminal, is used to perform preliminary detection on the samples, screen samples that meet the condition that the confidence level of all preliminary detection results is greater than or equal to the first threshold, and send the preliminary detection results of samples that meet the condition to the result reporting module, while uploading samples that do not meet the condition to the edge cloud server.

[0113] The second detection layer, executed by the edge cloud server, is used to perform multi-algorithm joint detection on samples from the first detection layer, filter samples that meet the condition that the confidence of all joint decision results is greater than or equal to the second threshold, and send the joint decision results of samples that meet the condition to the result reporting module, and send samples that do not meet the condition to the manual decision module.

[0114] The third detection level, executed by the manual decision-making module, is used to manually re-examine the samples from the second detection level and send the manual re-examination results to the result reporting module.

[0115] This hierarchical inspection strategy effectively balances inspection efficiency and accuracy, ensuring the accuracy and reliability of defect detection through collaboration among different levels.

[0116] To enhance the detection capability of the terminal's general basic detection scheme, the system will combine the joint decision-making results of multiple detection schemes with the results of manual re-inspection to collect abnormal or uncertain sample data from the general basic detection scheme. This data will be used for subsequent model training, thereby further enhancing the detection capability of the detection terminal.

[0117] Specifically, Figure 5 This is a flowchart of the sample collection strategy provided by the present invention, such as... Figure 5 As shown, the sample collection rules include:

[0118] (1) Samples that have only undergone the first layer of general basic testing will not be collected;

[0119] (2) For samples that have been tested by the second detection level, the detection results of the first detection level are compared with the detection results of each detection scheme of the second detection level, and the result with the most consistent detection results is determined as the final judgment result.

[0120] (3) Compare the final judgment result with the detection result of the first detection level, and retain the sample if the results are different.

[0121] (4) All samples that have undergone manual re-examination must be collected.

[0122] Based on the above embodiments, as an optional embodiment, the defect detection method based on joint local and edge cloud detection further includes, after collecting samples for model training for defect detection, using the collected samples to train and test the defect detection model on an edge cloud server; and sending the trained and tested model to the local terminal. Users can manually update the model on the terminal to enhance its detection capabilities.

[0123] Based on the above embodiments, the following describes an optional method of "jointly detecting each image in the sample using multiple detection schemes, and obtaining all joint decision results and the confidence level of each joint decision result based on the principle of majority rule," which specifically includes the following steps:

[0124] (1) Use multiple detection methods to detect the uploaded samples and obtain the detection results;

[0125] This step requires calling multiple pre-built detection schemes on the edge cloud server. A detection scheme can be understood as a detection model (each detection model may include multiple sub-models to achieve the function of defect detection).

[0126] These detection models (detection schemes) can have different algorithmic foundations. For example, one model might be a deep learning-based object detection model that can learn from large amounts of image data to identify complex defect features in product images, such as tiny cracks or irregular shapes. Another model might be a template matching model based on traditional image processing techniques, which identifies differences by matching the image with known defect-free product images to determine the presence of defects. These different schemes / models are used to detect uploaded sample images, and each scheme / model outputs its own detection results, including information such as the presence, location, and type of defects.

[0127] (2) Based on the principle of majority rule, the joint decision result is determined by using the detection results of multiple detection schemes.

[0128] For example, an edge cloud-based joint detection system can be used to determine if a mobile phone screen has scratches or other defects. When the confidence level of the local terminal detection is insufficient, a multi-scheme joint detection method from the edge cloud is triggered (assuming three schemes are set up):

[0129] The sample includes image A of a mobile phone screen (excluding other images). For the same image A, three detection methods were used respectively, and the detection results and confidence scores were obtained:

[0130] YOLOv5-based defect detection model: with scratches, confidence level 0.88;

[0131] ResNet-50-based classification model: with scratches, confidence score 0.82;

[0132] Based on traditional image processing: no scratches, confidence level 0.75.

[0133] The majority decision result is a scratch; if two options support the decision and one opposes it, the joint decision result is a scratch, with a corresponding confidence level of 0.85.

[0134] As another optional embodiment, this step may also include the following processes:

[0135] 1) Based on the defect area in the detection result of the preset detection scheme, calculate the crossover ratio of defect areas between the preset detection scheme and other detection schemes respectively.

[0136] Specifically, from N detection schemes, one scheme is randomly selected or selected according to priority (which can be set manually) as the benchmark, and the coordinates of the detected defect regions (such as bounding boxes or segmentation masks) are extracted. The other N-1 schemes are traversed, and the intersection-union ratio of the defect regions with the benchmark scheme is calculated for each scheme.

[0137] 2) Count the number of other detection scheme pairs whose defect area crossover ratio with the preset detection scheme is greater than the preset crossover ratio threshold;

[0138] Specifically, the number of other schemes whose IoU with the baseline scheme exceeds a preset intersection-union ratio threshold (e.g., 0.5) is denoted as K.

[0139] 3) If the number of other detection scheme pairs is greater than or equal to the preset number threshold, the detection results corresponding to the crossover ratio of the defect area being greater than the preset crossover ratio threshold shall be regarded as valid detection results.

[0140] Assuming N = 5 detection schemes, the baseline scheme is A, the IoU threshold is 0.6, and the number threshold is 2, the following results are obtained after calculation:

[0141] The IoU values ​​of schemes B and C are 0.7 and 0.8 respectively, while those of schemes D and E are 0.4 and 0.3 respectively. Therefore, K = 2 (the detection results of B and C are valid), which satisfies the condition.

[0142] 4) Based on the valid detection results, and following the principle of majority rule, determine the joint decision-making result.

[0143] The valid test results are the test results of test schemes A, B, and C. If the test result of A is "scratch" and the test results of B and C are "contamination", then the joint decision result is "contamination".

[0144] (3) Determine the confidence level of the joint decision result based on the confidence level of the test results (or valid test results) that are consistent with the majority of test results.

[0145] The confidence level of the joint decision result is determined based on the confidence level of the majority of consistent detection results. Each detection model has a confidence level assessment for its own detection results, reflecting the model's degree of confidence in its own results. For example, if two of the three models detect defects, and their confidence levels are 0.8 and 0.7 respectively, the confidence level of the joint decision result can be calculated according to certain rules. The average of these two confidence levels (0.75) can be taken as the confidence level of the joint decision result, or other more complex weighting methods can be used, such as allocating weights according to the model's accuracy in previous detections, to obtain a comprehensive confidence level. This confidence level is used to measure the credibility of the joint decision result, providing a basis for subsequent operations such as whether manual re-examination is needed.

[0146] It should be noted that the number or weight of the detection schemes can be specially set or dynamically adjusted in this invention so as to output a detection result with consistent results from most detections as a joint decision result.

[0147] Optionally, the confidence level of the joint decision result is determined based on the confidence level of the detection results that are consistent with the majority of detection results, including:

[0148] (1) Determine the weighting based on the historical detection accuracy of the detection scheme with consistent detection results.

[0149] Historical detection accuracy refers to the accuracy rate of each detection scheme in past detection tasks, which can be obtained through statistical analysis of past detection results. For example, for detection schemes A, B, and C, suppose their historical detection accuracies are 90%, 80%, and 70%, respectively. If the detection results of detection schemes A and B are consistent in this joint detection, then their historical detection accuracy can be used as the basis for weighting. Their accuracy values ​​can be used as the numerators of the weights, and the total accuracy (90 + 80 = 170) as the denominator. Thus, the weighted weight of detection scheme A is 90 / 170, and the weighted weight of detection scheme B is 80 / 170. This weighting method based on historical performance can more objectively reflect the reliability of each detection scheme, making the determination of the confidence level of the joint decision result more reasonable.

[0150] (2) Determine the confidence level of the joint decision result based on the confidence level and weighting of the test results that are consistent with the majority of test results.

[0151] The confidence level of the joint decision-making result is determined by combining the confidence levels of the majority of consistent test results with a weighted average. Assume that test plan A has a confidence level of 0.8 for the current test result, and test plan B has a confidence level of 0.7. Based on the weighted average calculated above, the confidence level of the joint decision-making result can be calculated using a weighted average: (0.8 × 90 / 170) + (0.7 × 80 / 170). This confidence level combines the confidence levels of the majority of test plans with their historical reliability, more accurately reflecting the credibility of the joint decision-making result. This method effectively utilizes historical data and current test results, providing a more reliable basis for subsequent decisions (such as whether manual retesting is needed).

[0152] In summary, the defect detection method and system based on joint local and edge cloud detection provided by this invention proposes a distributed architecture for layered edge cloud detection. This architecture distributes different functions and processing layers across different terminal devices and servers to improve system flexibility and scalability. This joint detection deployment method can effectively utilize existing resources and improve overall detection efficiency, making it very suitable for industrial environments. It has the following beneficial effects:

[0153] (1) Improve detection efficiency: By dividing the detection task into different levels, high-confidence samples can be quickly screened out, thereby reducing the workload of subsequent processing.

[0154] (2) Enhanced accuracy: Each level adopts a detection method suitable for its complexity, thereby improving the detection accuracy in each level, especially for samples that are difficult to identify.

[0155] (3) Effective resource utilization: Lightweight algorithms can quickly handle most common problems, while complex algorithms focus on solving more challenging detection tasks, thereby optimizing the use of computing resources.

[0156] (4) Reduced false positive rate: The layered strategy ensures that in-depth detection is only performed when necessary, reducing false positives caused by over-checking.

[0157] (5) Flexible response to changes: Based on the test results at different levels, the subsequent processing methods can be flexibly adjusted to adapt to different scenarios and product requirements.

[0158] (6) Facilitates improvement and feedback: By collecting detection data at different levels, the model and detection strategy can be continuously optimized to improve the overall performance of the system.

[0159] (7) Promote the rationality of human intervention: manual re-examination is only for the most controversial or uncertain samples to ensure that human resources are used efficiently.

[0160] (8) High adaptability: Layered detection method can be applied to a variety of industrial detection scenarios, especially in fields with high requirements for real-time performance and accuracy.

[0161] The embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A defect detection method based on local and edge cloud joint detection, characterized in that, include: The local terminal acquires the sample to be tested; The sample to be tested includes at least one image of the target product; The local terminal performs defect detection on each image in the sample, and obtains all preliminary detection results and the confidence level of each preliminary detection result; Once the local terminal determines that the confidence level of all preliminary detection results is greater than or equal to the first threshold, it sends the preliminary detection results of the sample to the result reporting module of the local terminal. Otherwise, send the sample to the edge cloud server; The edge cloud server uses multiple detection schemes to jointly detect each image in the sample, and obtains all joint decision results and the confidence level of each joint decision result based on the principle of majority rule; Once the edge cloud server determines that the confidence level of all joint decision results is greater than or equal to the second threshold, it sends the joint decision results of the sample to the result reporting module of the local terminal. Otherwise, the sample will be sent to the human decision-making module; The manual decision-making module is used to manually re-examine the samples, obtain the manual re-examination results, and send the manual re-examination results to the result reporting module.

2. The defect detection method based on joint local and edge cloud detection according to claim 1, characterized in that, The defect detection method employs a hierarchical detection strategy, specifically: The first detection level, executed by the local terminal, is used to perform preliminary detection on the samples, screen samples that meet the condition that the confidence level of all preliminary detection results is greater than or equal to the first threshold, and send the preliminary detection results of samples that meet the condition to the result reporting module, while uploading samples that do not meet the condition to the edge cloud server. The second detection layer, executed by the edge cloud server, is used to perform multi-algorithm joint detection on samples from the first detection layer, filter samples that meet the condition that the confidence of all joint decision results is greater than or equal to the second threshold, and send the joint decision results of samples that meet the condition to the result reporting module, and send samples that do not meet the condition to the manual decision module. The third detection level, executed by the manual decision-making module, is used to manually re-examine the samples from the second detection level and send the manual re-examination results to the result reporting module.

3. The defect detection method based on joint local and edge cloud detection according to claim 2, characterized in that, Also includes: Collect samples for model training in defect detection, specifically: First, collect all samples from manual re-examinations; Second, for samples that have undergone the second detection level, the detection results of the first detection level are compared with the detection results of each detection scheme in the second detection level, and the result in which most detection results are consistent is taken as the final judgment result. The final judgment result is compared with the detection result of the first detection level, and the sample is retained if the results are different.

4. The defect detection method based on joint local and edge cloud detection according to claim 3, characterized in that, After collecting samples for training the model used for defect detection, the following is also included: The collected samples are used to train and test the defect detection model on an edge cloud server; Send the trained and tested model to the local terminal.

5. The defect detection method based on joint local and edge cloud detection according to claim 1, characterized in that, A multi-detection scheme is used to jointly detect each image in the sample, and based on the principle of majority rule, all joint decision results and the confidence level of each joint decision result are obtained, specifically including: Multiple detection methods are used to test the uploaded samples and obtain the test results; Based on the principle of majority rule, the joint decision-making result is determined by utilizing the detection results of multiple detection schemes; The confidence level of the joint decision result is determined based on the confidence level of the test results that are consistent with the majority of test results.

6. The defect detection method based on joint local and edge cloud detection according to claim 5, characterized in that, The confidence level of the joint decision result is determined based on the confidence level of the test results that are consistent with the majority of test results, including: The weighting is determined based on the historical detection accuracy of detection schemes with consistent majority of detection results. The confidence level of the joint decision result is determined based on the confidence level and weighting of the test results that are consistent with the majority of test results.

7. The defect detection method based on joint local and edge cloud detection according to claim 1, characterized in that, Also includes: The received detection results are uploaded to the preset client through the result reporting module.

8. A defect detection system based on joint local and edge cloud detection, characterized in that, include: An image acquisition module installed on a local terminal is used to acquire a sample to be detected; the sample to be detected includes at least one image of the target product. A basic general-purpose defect detection module is set up on the local terminal to perform defect detection on each image in the sample, and obtain all preliminary detection results and the confidence level of each preliminary detection result; A first result filtering module is set up on the local terminal to send the preliminary test results of the sample to the result reporting module of the local terminal when it is determined that the confidence level of all preliminary test results is greater than or equal to the first threshold. Otherwise, send the sample to the edge cloud server; The multi-detection scheme joint decision module, set up on the edge cloud server, is used to perform joint detection on each image in the sample using multiple detection schemes, and obtain all joint decision results and the confidence level of each joint decision result based on the principle of majority rule. The second result filtering module, set on the edge cloud server, is used to send the joint decision results of the sample to the result reporting module of the local terminal when the confidence level of all joint decision results is greater than or equal to the second threshold. Otherwise, the sample will be sent to the human decision-making module; The manual decision-making module is used to manually re-examine the samples, obtain the manual re-examination results, and send the manual re-examination results to the result reporting module.

9. The defect detection system based on joint local and edge cloud detection according to claim 8, characterized in that, The defect detection system employs a hierarchical detection strategy, specifically: The first detection level, executed by the local terminal, is used to perform preliminary detection on the samples, screen samples that meet the condition that the confidence level of all preliminary detection results is greater than or equal to the first threshold, and send the preliminary detection results of samples that meet the condition to the result reporting module, while uploading samples that do not meet the condition to the edge cloud server. The second detection layer, executed by the edge cloud server, is used to perform multi-algorithm joint detection on samples from the first detection layer, filter samples that meet the condition that the confidence of all joint decision results is greater than or equal to the second threshold, and send the joint decision results of samples that meet the condition to the result reporting module, and send samples that do not meet the condition to the manual decision module. The third detection level, executed by the manual decision-making module, is used to manually re-examine the samples from the second detection level and send the manual re-examination results to the result reporting module.

10. The defect detection system based on joint local and edge cloud detection according to claim 9, characterized in that, Also includes: Sample collection module; The sample collection module is used to collect samples for training the defect detection model, specifically including: First, collect all samples from manual re-examinations; Second, for samples that have undergone the second detection level, the detection results of the first detection level are compared with the detection results of each detection scheme in the second detection level, and the result in which most detection results are consistent is determined as the final judgment result; the final judgment result is compared with the detection results of the first detection level, and the sample is retained if the results are different.

Images