Image detection method, device, equipment and storage medium

Abstract

The application discloses an image detection method and device, equipment and a storage medium, and relates to the technical field of industrial visual detection. The method comprises the following steps: obtaining a region of interest of a to-be-detected image, performing local inference on the region of interest through an edge-end lightweight model, and obtaining an edge-end probability vector and an edge-end prediction category; calculating comprehensive edge-end reliability according to the edge-end probability vector, the matching quality of the region of interest and the running state parameters of the edge-end device; in the case of determining that cloud-end review needs to be triggered, uploading the region of interest to a cloud server, so that the cloud server performs cloud-end inference on the region of interest and returns a cloud-end inference result; and determining a defect detection result of the to-be-detected image based on the cloud-end inference result, the edge-end probability vector, the edge-end prediction category and the comprehensive edge-end reliability. The application effectively improves the accuracy of laser printing defect detection under the premise of ensuring the real-time performance of a production line and the safety of data, so that the low time delay of an edge-end and the high precision of a cloud-end are taken into account.

Description

Technical Field

[0001] This application relates to the field of industrial visual inspection technology, and in particular to an image inspection method, apparatus, device and storage medium. Background Technology

[0002] In industrial scenarios such as laser printing and inkjet marking, it is usually necessary to perform real-time defect detection on the printed area of ​​the product to be inspected in order to avoid quality defects such as missing prints, ghosting, ink spots, and blurred character edges, thereby ensuring the readability and traceability of product markings.

[0003] Currently, industrial vision inspection systems mainly face the choice of two implementation schemes. The first is to deploy lightweight models at the edge to complete all inference. This type of scheme has the advantage of low latency, but it is prone to false positives and false negatives under conditions such as changes in lighting, printing speed, equipment aging, and weak defect features. The second type is to upload images to the cloud and use high-precision models for inference. This type of scheme has higher accuracy, but it has the disadvantages of large network transmission volume, high cloud response latency, insufficient real-time performance on the production line, and the risk of privacy data leakage.

[0004] It is evident that existing solutions either sacrifice accuracy for real-time performance or sacrifice real-time performance and data security for high accuracy, making it difficult to achieve both simultaneously. Therefore, in industrial quality inspection scenarios, overcoming the contradiction between limited computing power at the edge and cloud transmission latency and data security, thereby effectively improving the accuracy of defect detection while ensuring the real-time performance of the production line, is a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] The main purpose of this application is to provide an image detection method, apparatus, device, and storage medium, which aims to solve the technical problem of how to overcome the contradiction between limited computing power at the edge and transmission latency and data security in the cloud in industrial quality inspection scenarios, thereby effectively improving the accuracy of defect detection while ensuring the real-time performance of the production line.

[0006] To achieve the above objectives, this application provides an image detection method applied to an edge device, the method comprising the following steps: The region of interest in the image to be detected is obtained, and local inference is performed on the region of interest through the edge lightweight model to obtain the edge probability vector and the edge prediction category. The overall edge credibility is calculated based on the edge probability vector, the matching quality of the region of interest, and the operating status parameters of the edge device. If it is determined that cloud review needs to be triggered, the region of interest is uploaded to the cloud server so that the cloud server can perform cloud inference on the region of interest and return the cloud inference result; Based on the cloud inference results, the edge probability vector, the edge prediction category, and the comprehensive edge credibility, the defect detection result of the image to be detected is determined.

[0007] In one embodiment, the step of calculating the comprehensive edge confidence based on the edge probability vector, the matching quality of the region of interest, and the operating state parameters of the edge device includes: The maximum class probability, probability margin, and prediction entropy are determined based on the edge probability vector. The probability margin is the difference between the maximum class probability and the second largest class probability. The prediction entropy is used to characterize the degree of disorder in the probability distribution of the edge probability vector. The operating status parameters of the edge device are normalized to obtain the normalized edge load. The overall edge reliability is calculated based on the maximum class probability, the probability margin, the prediction entropy, the matching quality of the region of interest, and the normalized runtime load. ; in, To comprehensively assess the credibility of the edge, Let H be the maximum class probability, Margin be the probability margin, and H be the prediction entropy. To predict the maximum entropy, For the matching quality of the region of interest, To normalize the operating load, to These are the weighting coefficients.

[0008] In one embodiment, after the step of calculating the comprehensive edge confidence based on the edge probability vector, the matching quality of the region of interest, and the operating state parameters of the edge device, the method further includes: If the overall edge credibility is lower than the preset high credibility threshold, it is determined that cloud-based review needs to be triggered. And / or, if the edge device load in the operating status parameters exceeds a preset load threshold, it is determined that a cloud review needs to be triggered; And / or, if the matching quality of the region of interest is lower than a preset quality threshold, it is determined that a cloud-based review needs to be triggered; And / or, if the image to be detected has abnormal image quality, it is determined that a cloud-based review needs to be triggered; And / or, if the fluctuation of the edge prediction category, the comprehensive edge confidence, or the maximum category probability exceeds a preset fluctuation threshold, it is determined that cloud review needs to be triggered; And / or, if it is determined that cloud-based verification needs to be triggered when the current sampling period for high-confidence samples is underway.

[0009] In one embodiment, the cloud inference result includes a cloud probability vector, a cloud prediction category, and a cloud confidence level; The step of determining the defect detection result of the image to be detected based on the cloud inference result, the edge probability vector, the edge prediction category, and the comprehensive edge confidence includes: The cloud-edge conflict intensity is calculated based on the cloud inference result, the edge probability vector, the edge prediction category, and the comprehensive edge credibility. The cloud-edge conflict intensity is used to characterize the degree of discrepancy between the edge inference result and the cloud inference result. The defect detection result of the image to be detected is determined based on the comprehensive edge confidence, the cloud confidence, and the cloud-edge conflict intensity.

[0010] In one embodiment, the step of calculating the cloud-edge conflict intensity based on the cloud inference result, the edge probability vector, the edge prediction category, and the comprehensive edge credibility includes: Based on the edge prediction category and the cloud prediction category, a category conflict indication item and a defect risk direction item are determined; based on the comprehensive edge confidence and the cloud confidence, a confidence difference item is determined; and based on the edge probability vector and the cloud probability vector, a probability distribution difference item is determined. The cloud-edge conflict intensity is calculated based on the category conflict indicator, the confidence difference, the probability distribution difference, the defect risk direction, and the matching quality of the region of interest. The lower the matching quality of the region of interest, the higher the cloud-edge conflict intensity.

[0011] In one embodiment, before the step of determining the defect detection result of the image to be detected based on the comprehensive edge confidence, the cloud confidence, and the cloud-edge conflict intensity, the method further includes: When the edge prediction category is consistent with the cloud prediction category and the confidence difference is less than the confidence divergence threshold, the conflict type is determined to be no conflict. When the edge prediction category is consistent with the cloud prediction category and the confidence difference item is greater than or equal to the confidence divergence threshold, the conflict type is determined to be a confidence divergence conflict. When the edge prediction category is inconsistent with the cloud prediction category, the conflict type is determined to be a category conflict; When the edge prediction category is normal, the cloud prediction category is defective, and the cloud confidence level is greater than or equal to the cloud high confidence threshold, the conflict type is determined to be a missed detection risk conflict. When the overall edge confidence is lower than the low confidence threshold and the cloud-edge conflict intensity is greater than or equal to the high conflict threshold, the conflict type is determined to be a low confidence severe conflict. When the overall edge confidence is greater than or equal to the high confidence threshold, and the edge prediction category is inconsistent with the cloud prediction category after cloud review is triggered due to sampling inspection, batch anomaly, working condition anomaly or model drift detection strategy, or when the cloud-edge conflict intensity is greater than or equal to the preset conflict threshold, the conflict type is determined to be a high confidence sampling inspection conflict. When the same sample meets multiple conflict types, the primary conflict type is determined according to the priority of missed detection risk conflict, high-confidence sampling conflict, low-confidence severe conflict, category conflict, confidence level disagreement conflict, and no conflict.

[0012] In one embodiment, the step of determining the defect detection result of the image to be detected based on the comprehensive edge confidence, the cloud confidence, and the cloud-edge conflict intensity includes: When the overall edge confidence is less than the high confidence threshold and greater than or equal to the low confidence threshold, and the conflict type is non-conflict, the edge probability vector and the cloud probability vector are weighted and fused to obtain the final probability vector, and the category corresponding to the maximum value in the final probability vector is taken as the defect detection result of the image to be detected. When the conflict type is a missed detection risk conflict, the cloud prediction category is used as the defect detection result of the image to be detected; When the overall edge confidence is lower than the low confidence threshold and the cloud-edge conflict intensity is greater than or equal to the high conflict threshold, the cloud prediction category is used as the defect detection result of the image to be detected, or the image to be detected is sent to the re-inspection and confirmation process. When the conflict type is a confidence divergence conflict, the probability vector corresponding to the higher of the edge confidence and cloud confidence is used as the main factor, and the probability vector of the other end is combined for weighted fusion. When the conflict type is a category conflict, the edge-end confidence, cloud confidence, cloud-edge conflict intensity and region of interest matching quality are considered to determine whether to use the edge-end result, cloud result, fusion result or re-checking confirmation process. When the conflict type is a high-confidence sampling conflict, the sample is marked as a high-confidence sampling failure sample and used for threshold calibration or edge model optimization. The image detection method further includes: If it is determined that cloud-based review is not required, when the overall edge confidence is greater than or equal to the high confidence threshold, the edge prediction category is used as the defect detection result of the image to be detected.

[0013] Furthermore, to achieve the above objectives, this application also proposes an image detection device, which includes: The local inference module is used to obtain the region of interest of the image to be detected, and to perform local inference on the region of interest through the edge lightweight model to obtain the edge probability vector and the edge prediction category. The calculation module is used to calculate the comprehensive edge credibility based on the edge probability vector, the matching quality of the region of interest, and the operating status parameters of the edge device; The cloud review triggering module is used to upload the region of interest to the cloud server when it is determined that cloud review needs to be triggered, so that the cloud server can perform cloud inference on the region of interest and return the cloud inference result; The result determination module is used to determine the defect detection result of the image to be detected based on the cloud inference result, the edge probability vector, the edge prediction category, and the comprehensive edge credibility.

[0014] In addition, to achieve the above objectives, this application also proposes an image detection device, the device comprising: a memory, a processor, and an image detection program stored in the memory and executable on the processor, the image detection program being configured to implement the steps of the image detection method described above.

[0015] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which an image detection program is stored, and when the image detection program is executed by a processor, it implements the steps of the image detection method described above.

[0016] In addition, to achieve the above objectives, the present invention also provides a computer program product, the computer program product including an image detection program, which, when executed by a processor, implements the steps of the image detection method as described above.

[0017] One or more technical solutions proposed in this application have at least the following technical effects: The image detection method of this application includes: acquiring a region of interest (ROI) of an image to be detected, and performing local inference on the ROI using a lightweight edge-end model to obtain an edge-end probability vector and an edge-end prediction category; calculating a comprehensive edge-end credibility based on the edge-end probability vector, the matching quality of the ROI, and the operating status parameters of the edge device; uploading the ROI to a cloud server when it is determined that cloud review needs to be triggered, so that the cloud server performs cloud inference on the ROI and returns a cloud inference result; and determining the defect detection result of the image to be detected based on the cloud inference result, the edge-end probability vector, the edge-end prediction category, and the comprehensive edge-end credibility.

[0018] This application utilizes a lightweight edge-based model to perform local inference on the acquired region of interest (ROI), obtaining an edge probability vector and an edge prediction category. When cloud verification is required, the ROI is uploaded to a cloud server to obtain cloud inference results. Simultaneously, the edge probability vector, edge prediction category, and overall edge confidence level are combined to determine the defect detection result. On one hand, this scheme prioritizes local inference and preliminary judgment at the edge, uploading the ROI rather than the entire image only when cloud verification is triggered. This reduces network data transmission volume and cloud response time, mitigates privacy risks associated with the external transmission of production data, and ensures production line real-time performance and data security. On the other hand, after the cloud inference results are returned, the scheme combines the edge probability vector, edge prediction category, and overall edge confidence level with the cloud inference results for joint judgment. This allows for correction and compensation of samples that are prone to misjudgment or omission at the edge, even under conditions such as changes in lighting, printing speed, equipment aging, or weak defect features, thanks to high-precision cloud inference. Through the above mechanism, this application can effectively improve the accuracy of laser printing defect detection while ensuring the real-time performance of the production line and data security, thus taking into account the advantages of low latency at the edge and high precision in the cloud. Attached Figure Description

[0019] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0020] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a flowchart illustrating the first embodiment of the image detection method of this application; Figure 2 This is a schematic diagram of the edge real-time inference process of the image detection method of this application; Figure 3 This is a flowchart illustrating the second embodiment of the image detection method of this application; Figure 4 This is a flowchart illustrating the comprehensive edge credibility grading determination process of the image detection method in this application. Figure 5 This is a flowchart illustrating the third embodiment of the image detection method of this application; Figure 6 This is a schematic diagram illustrating the overall concept of the image detection method of this application; Figure 7 This is a structural block diagram of the first embodiment of the image detection device of this application; Figure 8 This is a schematic diagram of the image detection device of this application.

[0022] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0023] To make the above-mentioned objectives, features, and advantages of this application more apparent and understandable, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0024] This application provides an image detection method, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the image detection method of this application.

[0025] In this embodiment, the image detection method is applied to an edge device, and the method includes the following steps: Step S10: Obtain the region of interest (ROI) of the image to be detected, and perform local inference on the ROI using the edge lightweight model to obtain the edge probability vector and edge prediction category.

[0026] It should be noted that the executing entity of this application embodiment can be an image detection device capable of executing the image detection method of this application, such as an edge device, and this embodiment does not limit this. The following uses an edge device as an example to describe this embodiment and the following embodiments.

[0027] Understandably, the aforementioned Region of Interest (ROI) can be a local image region extracted from the image to be detected, containing laser-printed characters or markings. The aforementioned lightweight edge model can be a lightweight neural network model deployed on an image detection device, after compression, pruning, quantization, or ONNX format conversion, such as a YOLO classification model or a convolutional neural network model. The aforementioned edge probability vector can be the probability distribution output by the lightweight edge model after inference on the ROI, reflecting the likelihood that the region belongs to each category. For example, in a binary classification scenario, the edge probability vector can be represented as [normal sample probability, abnormal sample probability]. The aforementioned edge predicted category can be the classification result determined based on the category corresponding to the maximum probability value in the edge probability vector. For example, when the probability of an abnormal sample is greater than the probability of a normal sample, the edge predicted category is the defect category.

[0028] In the specific implementation, please refer to Figure 2 , Figure 2 This is a schematic diagram illustrating the real-time edge inference process of the image detection method in this application. Combined with... Figure 2 It is known that the edge device can acquire video frames of the laser printing work area in real time using an industrial camera, USB camera, or webcam, or it can read batch images from a folder of images to be inspected. In real-time inspection mode, the edge device can continuously acquire images at a preset frame rate; in batch inspection mode, it can read image files in a preset order. Next, the edge device can extract the region of interest (ROI) from the acquired images. Specifically, after preprocessing the input image such as grayscale conversion and contrast enhancement, the edge device can determine the location of the laser printing area through template matching, target region localization, target detection box localization, or preset coordinate localization, and then crop this area from the image to be inspected as the ROI. Subsequently, the edge device can perform model input preprocessing on the extracted ROI. The preprocessing process includes at least the following: scaling the region of interest to the input size required by the edge-lightweight model; converting the image from the BGR color space to the RGB color space; normalizing the pixel values ​​to the 0 to 1 range; standardizing the image data according to the mean and variance used in the training phase; converting the image data from HWC format to CHW format and expanding the batch dimension to obtain tensor data that conforms to the input format of the edge-lightweight model.

[0029] Subsequently, the edge device can input the preprocessed tensor data into the edge lightweight model for local inference. The edge lightweight model performs a forward propagation calculation on the input tensor and outputs an edge probability vector. The edge device can determine the category corresponding to the maximum probability value based on the edge probability vector and use this category as the edge prediction category. Specifically, the ROI (Region of Interest) can be extracted from the image to be detected using the ROI localization module, and the ROI image, ROI coordinates, and ROI matching quality can be output. The ROI coordinates can be represented as... The ROI matching quality can be expressed as In one implementation, the ROI positioning module uses template matching to determine the laser printing area and uses the template matching score as the ROI matching quality. When the ROI matching quality of the current frame is lower than the preset ROI quality threshold and the ROI coordinates of the previous frame are valid, the ROI positioning module uses the ROI coordinates of the previous frame as the backtracking ROI to reduce the risk of positioning failure caused by on-site vibration, sudden changes in lighting, image blurring, or partial occlusion.

[0030] Step S20: Calculate the overall edge credibility based on the edge probability vector, the matching quality of the region of interest, and the operating status parameters of the edge device.

[0031] It should be noted that the matching quality of the region of interest mentioned above refers to the reliability of the positioning obtained by the edge device when locating the region of interest. For example, when using template matching, the template matching score is the matching quality; a higher matching score indicates more accurate positioning. The operating status parameters of the edge device mentioned above can refer to system operating indicators of the edge device when performing inference tasks, including one or more of the following: CPU utilization, memory usage, inference time, current frame rate, network connection status, and cloud interface availability. The comprehensive edge credibility mentioned above can be a quantitative indicator that comprehensively measures the credibility of the edge inference results. It can be jointly determined by the edge model output features, the region of interest matching quality, and the edge device operating status, reflecting the credibility level of the edge inference results in terms of model discrimination clarity, input data reliability, and device operating stability.

[0032] It should be understood that edge devices can extract features from the edge probability vector in three dimensions: maximum class probability, probability margin, and prediction entropy. The maximum class probability reflects the edge model's absolute bias towards the most likely class; the probability margin reflects the difference between the maximum and second-largest class probabilities, indicating the model's clarity in class distinctions; and the prediction entropy characterizes the degree of disorder in the probability distribution of the edge probability vector. The edge device normalizes the collected operational parameters to obtain normalized edge load. Subsequently, the edge device substitutes the maximum class probability, probability margin, prediction entropy, region of interest matching quality, and normalized edge load into a pre-defined comprehensive edge credibility calculation formula, and obtains the comprehensive edge credibility through weighted calculation.

[0033] Step S30: If it is determined that cloud review needs to be triggered, the region of interest is uploaded to the cloud server so that the cloud server can perform cloud inference on the region of interest and return the cloud inference result.

[0034] It should be noted that the aforementioned cloud server can be a computing device or node deployed in the cloud, such as a cloud server, a private server, an industrial edge cloud platform, or a KubeEdge cloud node. The cloud server is equipped with a high-precision defect detection model, sample management services, a difficult example library, model training services, and model version management services, used for high-precision inference and collaborative management of data uploaded from edge devices. Data transmission between the cloud server and edge devices occurs through a communication interface, which can use HTTP, MQTT, WebSocket, or the KubeEdge edge-cloud messaging channel. The aforementioned cloud inference results refer to the data output by the cloud server after completing cloud inference, including cloud-predicted categories, cloud probability vectors, and cloud confidence scores. Cloud inference refers to the process by which the cloud server calls the high-precision model deployed on it to perform inference calculations on the received data. The high-precision model deployed on the cloud server can be a neural network model with a larger number of parameters, higher input resolution, or more comprehensive training data, such as the YOLO classification model, a convolutional neural network model, or a visual Transformer model.

[0035] In the implementation, after determining that the current sample requires cloud-based verification, the edge device uploads the region of interest (ROI) extracted from the image to be detected, along with edge-based inference metadata, to the cloud server. The edge device prioritizes uploading the ROI rather than the entire original image to reduce bandwidth consumption and privacy exposure. The ROI can be transmitted using JPEG compression, Base64 encoding, or other lightweight encoding methods. The edge-based inference metadata uploaded along with the ROI includes the edge prediction category, edge probability vector, overall edge confidence level, ROI coordinates, ROI matching quality, edge load status, inference time, trigger reason, and timestamp. Upon receiving the ROI and edge-based inference metadata, the cloud server invokes a high-precision defect detection model deployed in the cloud to perform cloud-based inference on the ROI. The high-precision cloud model performs forward propagation calculations on the ROI and outputs the cloud-based inference results. The cloud server then returns the cloud-based inference results to the edge device.

[0036] Step S40: Based on the cloud inference result, the edge probability vector, the edge prediction category, and the comprehensive edge credibility, determine the defect detection result of the image to be detected.

[0037] It should be noted that the above defect detection results can be the final output of the edge device, used to reflect whether there are defects in the image to be detected.

[0038] In practical implementation, the edge device can determine the defect detection result of the image to be detected based on the cloud inference result, edge probability vector, edge predicted category, and comprehensive edge confidence, combined with preset decision rules. Specifically, the edge device compares whether the edge predicted category is consistent with the cloud predicted category, and evaluates the relative levels of the comprehensive edge confidence and cloud confidence, selecting the appropriate decision strategy based on the comparison and evaluation results. When the edge predicted category is consistent with the cloud predicted category and the comprehensive edge confidence is high, the edge device can use the edge predicted category as the final category. When the edge predicted category is inconsistent with the cloud predicted category, the edge device comprehensively compares the relative levels of the comprehensive edge confidence and cloud confidence, selecting the category corresponding to the higher confidence level as the final category; if both the comprehensive edge confidence and cloud confidence are low, the image to be detected can be sent to the re-inspection and confirmation process. The edge device uses the finally determined category as the final category of the defect detection result, and determines the final confidence level based on the probability value corresponding to the final category. The defect detection results output by the edge device should include at least the final category, final confidence level, result source, whether the cloud is used, conflict type, conflict intensity K, and triggering cause. The defect detection results can be transmitted back to the edge device's on-site visualization interface, alarm system, rejection control system, batch statistics system, or quality traceability system.

[0039] This embodiment uses a lightweight edge-based model to perform local inference on the acquired region of interest (ROI), obtaining an edge probability vector and an edge prediction category. When cloud verification is required, the ROI is uploaded to a cloud server to obtain cloud inference results. The edge probability vector, edge prediction category, and overall edge confidence level are then combined to determine the defect detection result. On one hand, this scheme prioritizes local inference and preliminary judgment at the edge, uploading the ROI rather than the entire image only when cloud verification is required. This reduces network data transmission and cloud response time, mitigates privacy risks associated with the external transmission of production data, and ensures production line real-time performance and data security. On the other hand, after the cloud inference results are returned, the scheme uses a combined approach, integrating the edge probability vector, edge prediction category, and overall edge confidence level with the cloud inference results for joint judgment. This allows for correction and compensation of samples that are prone to misjudgment or omission at the edge, even under conditions such as changes in lighting, printing speed, equipment aging, or weak defect features, thanks to high-precision cloud inference. Through the above mechanism, this embodiment can effectively improve the accuracy of laser printing defect detection while ensuring the real-time performance of the production line and data security, thus taking into account the advantages of low latency at the edge and high precision in the cloud.

[0040] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to the above embodiment can be referred to the above description, and will not be repeated hereafter. On this basis, a second embodiment of the image detection method of this application is proposed; please refer to... Figure 3 , Figure 3 This is a flowchart illustrating the second embodiment of the image detection method of this application. Please also refer to... Figure 4 , Figure 4 This is a schematic diagram of the process for determining the overall edge credibility level in the image detection method of this application.

[0041] Furthermore, in order to more comprehensively characterize the credibility of edge inference results in terms of model discrimination clarity, input data reliability, and device operational stability, so that the calculated comprehensive edge credibility more accurately reflects the true reliability level of edge inference results, step S20 may include: Step S201: Determine the maximum class probability, probability margin, and prediction entropy based on the edge probability vector. The probability margin is the difference between the maximum class probability and the second largest class probability. The prediction entropy is used to characterize the degree of disorder in the probability distribution of the edge probability vector.

[0042] It should be noted that the maximum class probability mentioned above can be the largest probability value in the edge probability vector, representing the probability corresponding to the class most favored by the lightweight edge model. The probability margin mentioned above can be the difference between the maximum class probability and the second largest class probability, used to reflect the clarity of the edge model's discrimination between classes; a larger difference indicates a more definitive model judgment. The second largest class probability mentioned above can be the second largest probability value in the edge probability vector. The prediction entropy mentioned above can be a quantitative indicator used to measure the degree of disorder in the probability distribution in the edge probability vector; a larger entropy value indicates a more dispersed probability distribution and a more uncertain model output.

[0043] In practical implementation, the edge device can find the largest probability value in the edge probability vector, take this probability value as the maximum class probability, and take the class corresponding to the maximum probability value as the edge prediction class; it can find the second largest probability value in the edge probability vector as the second largest class probability, subtract the second largest class probability from the maximum class probability, and take the difference as the probability margin; it can calculate the prediction entropy based on the probability values ​​corresponding to each class in the edge probability vector. The prediction entropy is used to reflect the degree of disorder in the probability distribution of the edge probability vector. The larger the entropy value, the more ambiguous and uncertain the edge model's judgment of the current sample is.

[0044] Step S202: Normalize the operating status parameters of the edge device to obtain the normalized edge load.

[0045] It should be understood that after obtaining the edge probability vector output by the lightweight edge model, the edge device can simultaneously collect its current operating status parameters. These parameters include one or more of the following: CPU utilization, memory usage, inference time, current frame rate, network connection status, and cloud interface availability. The collected operating status parameters are then normalized, converting parameters with different dimensions and numerical ranges into a unified numerical interval, resulting in normalized edge load. A higher edge load indicates a heavier current operating burden on the edge device, lower real-time inference stability, and a corresponding decrease in overall edge reliability. Normalized edge load can be obtained by normalizing one or more of the following: CPU utilization, memory usage, inference time, and frame rate degradation.

[0046] Step S203: Calculate the comprehensive edge credibility based on the maximum category probability, the probability margin, the prediction entropy, the matching quality of the region of interest, and the normalized runtime load.

[0047] It should be noted that, for binary classification laser printing defect detection scenarios, the edge probability vector It can be represented as: ; in, This indicates the probability that the ROI belongs to an abnormal printing defect. This indicates the probability that the ROI belongs to a normal printed sample. The edge device is based on... Determine the edge prediction category Maximum class probability The probability margin and prediction entropy H are also considered. for The maximum probability value in the range; Margin is the difference between the maximum probability and the second-highest probability, which can be expressed as follows in the binary classification case: The prediction entropy H represents the degree of uncertainty in the predicted distribution of the edge model.

[0048] Unlike methods that only use the maximum class probability as the confidence level, the comprehensive edge confidence level in this invention is jointly determined by the edge model output features, ROI matching quality, and edge load state. In one feasible implementation, the comprehensive edge confidence level... It can be represented as: ; in, The maximum class probability represents the class probability that the edge model most favors. Assuming the edge inference outputs normal=0.85 and abnormal=0.15, then... Assuming the edge inference outputs normal=0.55 and abnormal=0.45, then... Although the maximum probability is 0.55, the two categories are very close, indicating that the model is actually not very certain.

[0049] Margin represents the probability margin, reflecting the clarity of class distinction. In this example... If normal=0.85, abnormal=0.15, and margin=0.70, it means that the difference between the two categories is large and the model's judgment is relatively clear; if normal=0.55, abnormal=0.45, and margin=0.10, it means that the model's judgment is relatively ambiguous and the edge results are not very reliable.

[0050] H is the prediction entropy, used to measure the degree of disorder in the probability distribution. To predict the maximum entropy, in this example scheme... This example is in a binary classification scenario. Pick If normal = 0.85 and abnormal = 0.15, it indicates that the model's inference output is very definite and the entropy is very low; if normal = 0.55 and abnormal = 0.45, it indicates that the model's output is very ambiguous and the entropy is very high. That is, the higher the entropy, the less reliable the model is.

[0051] To ensure the quality of the matching for the region of interest, template matching is used in this example. This indicates a high degree of matching between the region to be detected and the template; conversely, if... This indicates a very low match rate, and the ROI may be shifted.

[0052] The edge load (i.e., normalized operating load) serves as a system state characteristic for inference reliability. When the edge CPU exceeds the threshold, it causes anomalies such as increased inference latency, untimely image frame processing, unstable ROI tracking, and decreased system real-time performance, resulting in low overall reliability. In such cases, the system will call cloud inference.

[0053] to The weighting coefficients can be set empirically or determined through validation sets, field samples, grid search, Bayesian optimization, or other parameter calibration methods. In this example implementation, to After normalization, the sum is 1.

[0054] It can be obtained by normalizing one or more of CPU utilization, memory utilization, inference time, and frame rate drop. The higher the edge load, the lower the stability of real-time inference at the edge, and the lower the overall edge reliability.

[0055] In this implementation, the embodiment extracts three dimensions of features from the edge probability vector: maximum class probability, probability margin, and prediction entropy. These features are then combined with the matching quality of the region of interest (ROI) and the normalized operating load of the edge device to jointly calculate the comprehensive edge credibility. Specifically, the maximum class probability reflects the edge model's absolute bias towards the most likely class; the probability margin reflects the model's clarity in class distinctions; the prediction entropy reflects the degree of disorder in the probability distribution; the ROI matching quality reflects the reliability of the input data; and the normalized edge load reflects the impact of the device's operating state on inference stability. By comprehensively calculating these multiple features, compared to relying solely on the maximum class probability as the basis for credibility, this approach more comprehensively characterizes the credibility of the edge inference results in terms of model discrimination clarity, input data reliability, and device operating stability. This allows the calculated comprehensive edge credibility to more accurately reflect the true reliability level of the edge inference results.

[0056] Furthermore, in order to reduce unnecessary data transmission and cloud computing overhead while ensuring the reliability of defect detection under various abnormal and uncertain scenarios, step S20 may further include: Step S21: If the overall edge credibility is lower than the preset high credibility threshold, it is determined that cloud review needs to be triggered.

[0057] It should be noted that a preset high-confidence threshold can be set. and preset low confidence threshold ,in Greater than .when When a sample is deemed a high-confidence sample, it is considered a high-confidence sample. When, the sample is determined to be a moderately reliable sample; when At that time, the sample was determined to be a low-confidence sample. This example implementation scheme sets... Other scenarios can be dynamically adjusted based on actual results.

[0058] In the specific implementation, and The system can dynamically adjust based on factors such as the cost of missed detections, the cost of false detections, average cloud latency, network bandwidth, edge load, ROI positioning stability, batch anomaly rate, and historical conflict ratio. For example, when network congestion or increased average cloud latency occurs, the cloud review trigger threshold is raised to reduce invalid uploads; when missed detection risk conflicts, decreased ROI positioning quality, or abnormal on-site conditions occur in consecutive batches, the cloud review trigger threshold is lowered to increase the cloud participation rate and reduce the risk of missed defects. High-confidence and stable samples can have their detection results directly output from the edge; medium-confidence and low-confidence samples enter the cloud review candidate queue. For high-confidence samples, the system can still upload some samples to the cloud according to the sampling strategy or abnormal condition strategy to detect high-confidence false positives and calibrate the threshold.

[0059] Step S22: and / or, if the edge device load in the operating status parameters exceeds a preset load threshold, determine that a cloud review needs to be triggered.

[0060] It should be noted that the edge device load in the above-mentioned operating status parameters can refer to the system operating load indicators of the edge device when performing inference tasks, including one or more of the following: CPU utilization, memory usage, inference time, and the degree of current frame rate decline. The above-mentioned preset load threshold can be a pre-set numerical limit used to determine whether the current operating load of the edge device has reached a level that affects the real-time performance and stability of local inference. When the edge device load exceeds this limit, continued local inference by the edge device may cause abnormal situations such as increased inference latency, untimely image frame processing, unstable region of interest tracking, or decreased system real-time performance.

[0061] In its implementation, the edge device synchronously collects operational status parameters during the calculation of the overall edge reliability. The edge device load in these parameters is compared with a preset load threshold. If the edge device load exceeds the threshold, the edge device determines that the current sample requires cloud-based verification. This determination is based on the fact that when the edge device load is too high, continuing local inference may affect real-time detection performance, leading to untimely image frame processing or delayed inference result output. Offloading the sample to the cloud server for verification can maintain the real-time performance of production line detection when the edge device load is too high. If the edge device load does not exceed the preset load threshold, the edge device does not trigger cloud-based verification due to load issues, and the sample continues to be processed according to other determination criteria.

[0062] Step S23: and / or, if the matching quality of the region of interest is lower than a preset quality threshold, determine that a cloud-based review needs to be triggered.

[0063] It should be noted that the matching quality of the aforementioned region of interest (ROI) refers to the reliability of positioning obtained by the edge device when locating the ROI. When using template matching, the template matching score is the matching quality; a higher matching score indicates more accurate positioning. The aforementioned preset quality threshold can be a pre-set numerical limit used to determine whether the positioning quality of the ROI has reached a reliable level. When the matching quality is below this limit, it indicates that the positioning of the ROI is unstable or at risk of deviation.

[0064] It should be understood that the edge device acquires the matching quality of the region of interest (ROI) while obtaining it. This matching quality is compared to a preset quality threshold. If the matching quality is lower than the threshold, the edge device determines that the current sample needs to trigger cloud-based verification. This determination is based on the fact that when the matching quality of the ROI is low, the ROI localization may be unstable or offset, and the input data processed by the edge model may deviate from the actual printed area, leading to a decrease in the reliability of the edge inference results. Therefore, high-precision cloud-based inference is needed for verification. When the matching quality is higher than or equal to the preset quality threshold, the edge device does not trigger cloud-based verification due to matching quality issues, and the sample continues to be processed according to other criteria.

[0065] Step S24: and / or, if the image to be detected has abnormal image quality, determine that a cloud-based review needs to be triggered.

[0066] It should be noted that the aforementioned image quality anomalies can refer to quality problems that occur during the acquisition or transmission of the image under test, including one or more of the following: image blurring, overexposure, underexposure, excessive noise, and occlusion. Image blurring may be caused by device shake or inaccurate focusing; overexposure or underexposure may be caused by sudden changes in lighting or improper exposure parameters; excessive noise may be caused by low light or insufficient sensor performance; and occlusion may be caused by foreign objects obstructing the lens or the printed area being covered.

[0067] It should be understood that after acquiring the image to be detected, the edge device can use image quality assessment algorithms to analyze the image quality and determine whether there are abnormalities such as blurriness, overexposure, underexposure, excessive noise, or occlusion. If the edge device detects any one or more of these image quality abnormalities, it determines that the current sample needs to trigger cloud-based verification. This determination is based on the fact that image quality abnormalities can lead to inaccurate extraction of regions of interest or distortion of the edge model's input data, thus compromising the reliability of the edge inference results at the input stage, requiring verification through high-precision cloud-based inference. If the image to be detected does not have image quality abnormalities, the edge device will not trigger cloud-based verification due to image quality issues, and the sample will continue to be processed according to other determination criteria.

[0068] Step S25: and / or, if the fluctuation of the edge prediction category, the comprehensive edge confidence, or the maximum category probability exceeds a preset fluctuation threshold, determine that cloud review needs to be triggered.

[0069] It should be noted that the aforementioned fluctuations can refer to the degree of change in edge prediction categories, overall edge confidence, or maximum class probability across multiple consecutive frames, reflecting the stability of the inference results within the current time window. Fluctuations may originate from sudden changes in operating conditions, abnormal inputs, or model instability. The aforementioned preset fluctuation threshold can be a pre-defined numerical limit or state change limit, used to determine whether the fluctuation level of the inference results across consecutive frames has reached an abnormal level. When the fluctuation exceeds this limit, it indicates a possible sudden change in operating conditions or an inference anomaly.

[0070] It should be understood that if the edge-predicted category jumps between consecutive frames, or if the inter-frame difference in the overall edge confidence or maximum class probability exceeds a preset fluctuation threshold, the edge device determines that the current sample needs to trigger cloud-based verification. This determination is based on the fact that significant fluctuations in the inference results between consecutive frames indicate potential changes in production line status, lighting conditions, printing speed, increased equipment vibration, abnormal input images, or instability of the edge model under those conditions, requiring verification using high-precision cloud-based inference. If the edge-predicted category remains stable and the fluctuations in the overall edge confidence and maximum class probability do not exceed the preset fluctuation threshold, the edge device will not trigger cloud-based verification due to fluctuations, and the sample will continue to be processed according to other determination criteria.

[0071] Step S26: and / or, if the current sampling period for high-confidence samples is within a certain period, determine that cloud-based verification needs to be triggered.

[0072] It should be noted that the aforementioned high-confidence sample sampling cycle can refer to a pre-set time interval or batch number interval for cloud-based sampling of high-confidence samples. High-confidence samples are those whose overall edge confidence reaches a preset high-confidence threshold. Normally, edge devices directly output test results for these samples. However, when the high-confidence sample sampling cycle arrives, even if the sample's overall edge confidence falls within the high-confidence range, the edge device will still upload the sample to the cloud for verification to detect edge high-confidence misjudgments and calibrate the grading threshold. The high-confidence sample sampling cycle can be set using a preset ratio, preset time interval, or preset batch number.

[0073] It should be understood that if the current sample is within a high-confidence sampling period, the edge device determines that the current sample needs to trigger cloud verification and uploads the region of interest to the cloud server for verification, regardless of whether the overall edge confidence of the sample falls within the high-confidence range. This determination method is based on the fact that for samples with high overall edge confidence, the edge device usually outputs the detection result directly without cloud verification. However, the edge model may still exhibit high-confidence misclassifications, meaning that the edge model outputs the wrong category with high confidence. By setting a high-confidence sample sampling period and periodically uploading some high-confidence samples to the cloud for verification, high-confidence misclassifications can be detected in a timely manner, the grading threshold can be calibrated, and the edge model can be optimized.

[0074] In addition, if the current production line status, lighting conditions, printing speed, equipment vibration, or batch anomaly rate changes, the abnormal operation status review strategy will be triggered, and cloud review will also be triggered.

[0075] In its implementation, this embodiment sets multiple conditions to trigger cloud-based review, including: the overall edge confidence level being lower than a preset high confidence threshold; the edge device load exceeding a preset load threshold; the matching quality of the region of interest being lower than a preset quality threshold; the image quality of the image to be detected exhibiting anomalies; the fluctuation of the edge prediction category, overall edge confidence level, or maximum category probability exceeding a preset fluctuation threshold; and the current period being within a high-confidence sample sampling cycle. This ensures that cloud-based review triggering no longer relies solely on a single low-confidence condition. Cloud-based review is triggered when any condition is met, allowing cloud computing power to be allocated strategically to the most valuable samples. This avoids invalid uploads when edge inference is reliable and operating conditions are stable. Simultaneously, it ensures timely correction using high-precision cloud inference when edge inference uncertainty is high, device overload occurs, input quality is poor, operating conditions fluctuate, or active verification is required. This effectively guarantees the reliability of defect detection under various abnormal and uncertain scenarios while reducing unnecessary data transmission and cloud computing overhead.

[0076] Furthermore, the cloud inference result includes a cloud probability vector, a cloud predicted category, and a cloud confidence level. To avoid blindly adopting the cloud result when the cloud inference result is incorrect or the confidence level is insufficient, which could lead to a decrease in detection accuracy, step S40 may include: Step S401: Calculate the cloud-edge conflict intensity based on the cloud inference result, the edge probability vector, the edge prediction category, and the comprehensive edge credibility. The cloud-edge conflict intensity is used to characterize the degree of discrepancy between the edge inference result and the cloud inference result.

[0077] It should be noted that the aforementioned cloud-edge conflict intensity can be a comprehensive indicator used to quantify the degree of discrepancy between edge-side and cloud-side inference results. The cloud-edge conflict intensity is jointly determined by the consistency between the edge-side and cloud-side predicted categories, the difference between the combined edge-side confidence and cloud-side confidence, the distribution differences between the edge-side and cloud-side probability vectors, the defect risk direction, and the matching quality of the region of interest. It reflects the comprehensive degree of discrepancy between the inference results at both the edge and cloud ends across multiple levels, including category, confidence, probability distribution, risk direction, and input quality.

[0078] It should be understood that the calculated cloud-edge conflict intensity is used for subsequent conflict level classification and differentiated decision-making. The higher the cloud-edge conflict intensity value, the greater the degree of discrepancy between the edge inference results and the cloud inference results, and the more the credibility of the edge inference results needs to be questioned.

[0079] Step S402: Determine the defect detection result of the image to be detected based on the comprehensive edge confidence, the cloud confidence, and the cloud-edge conflict intensity.

[0080] It should be noted that the aforementioned cloud confidence score can be a quantitative indicator used to characterize the credibility of the cloud inference result after the cloud server completes cloud inference. The higher the cloud confidence score, the higher the degree of certainty of the cloud model regarding its own output result.

[0081] It should be understood that edge devices can determine the conflict level of the current sample based on the magnitude of the cloud-edge conflict intensity, which includes low, medium, and high conflict levels. The edge device further combines the relative levels of edge confidence and cloud confidence, and selects an appropriate decision strategy based on preset differentiated decision rules. When the cloud-edge conflict intensity is low, the discrepancy between the edge inference result and the cloud inference result is small, and the edge device can execute an edge-cloud weighted fusion strategy, weighting and fusing the edge probability vector and the cloud probability vector to obtain the final category. When the cloud-edge conflict intensity is medium, the edge device determines which side to prioritize based on the relative levels of the edge confidence and cloud confidence. When the cloud-edge conflict intensity is high, the discrepancy between the edge inference result and the cloud inference result is large, and the edge device prioritizes the cloud prediction category as the final category, or, when both the edge confidence and cloud confidence are low, sends the image to be detected into a re-inspection and confirmation process.

[0082] In one feasible implementation, a low conflict threshold can be set. and high conflict threshold When the cloud-edge conflict intensity K is less than the low conflict threshold, the sample is a low conflict intensity sample; when K is greater than or equal to the low conflict threshold and less than the high conflict threshold, the sample is a medium conflict intensity sample; when K is greater than the high conflict threshold, the sample is a high conflict intensity sample. In one embodiment of laser printing defect detection, the low conflict threshold can be 0.3, and the high conflict threshold can be 0.8. The above-mentioned low and high conflict thresholds can also be dynamically adjusted based on the validation set, field data, cost of missed detection, cost of false detection, and cloud verification capabilities. In its implementation, this embodiment calculates the cloud-edge conflict intensity based on cloud inference results, edge probability vectors, edge prediction categories, and comprehensive edge confidence. The resulting cloud-edge conflict intensity characterizes the degree of discrepancy between edge and cloud inference results, allowing the inconsistency to be quantified into a comparable numerical indicator. Based on this, the defect detection result of the image to be detected is determined jointly by comprehensive edge confidence, cloud confidence, and cloud-edge conflict intensity. This ensures that the final decision does not rely on inference results from a single source, nor does it simply overwrite edge results with cloud results. Instead, it comprehensively considers the confidence level of edge inference, the confidence level of cloud inference, and the degree of discrepancy between the two inference sources. Through the above mechanism, when edge inference and cloud inference are highly consistent, this embodiment can enhance decision confidence by using bilateral results to mutually verify each other; when there are discrepancies between the two inferences, this embodiment can make a more reasonable trade-off between fast edge response, high cloud accuracy and conflict risk based on the relative relationship between edge confidence, cloud confidence and conflict intensity, thereby effectively avoiding the decrease in detection accuracy caused by blindly adopting cloud results when cloud inference results are wrong or have insufficient confidence.

[0083] Based on the first and / or second embodiments of this application, the content that is the same as or similar to that of the first and second embodiments described above in the third embodiment of this application can be referred to the above description and will not be repeated hereafter. On this basis, a third embodiment of the image detection method of this application is proposed; please refer to... Figure 5 , Figure 5 This is a flowchart illustrating the third embodiment of the image detection method of this application.

[0084] Furthermore, in order to more comprehensively and precisely characterize the true degree of divergence between edge inference and cloud inference, thereby providing a more accurate quantitative basis for subsequent defect detection decisions, step S401 may include: Step S4011: Determine the category conflict indication item and the defect risk direction item based on the edge prediction category and the cloud prediction category; determine the confidence difference item based on the comprehensive edge confidence and the cloud confidence; and determine the probability distribution difference item based on the edge probability vector and the cloud probability vector.

[0085] It should be noted that the aforementioned category conflict indicator can be a binary index used to characterize whether the edge-predicted category and the cloud-predicted category are consistent. The category conflict indicator takes a value of 1 when the two predicted categories are inconsistent, and a value of 0 when they are consistent. The aforementioned defect risk direction term can be a quantitative index used to characterize the asymmetry of misjudgment risks between normal and defective categories in industrial defect detection. In laser printing defect detection scenarios, the risk of missing a defective part by misjudging it as a normal part is higher than the risk of misjudging a normal part as a defective part. The aforementioned confidence difference term can be a quantitative index used to characterize the degree of difference between the overall edge confidence and cloud confidence, reflecting the gap in confidence levels between the edge and the cloud. The aforementioned probability distribution difference term can be a quantitative index used to characterize the degree of difference in the distribution between the edge probability vector and the cloud probability vector, reflecting the overall deviation of the two probability vectors in the probability values ​​of each category.

[0086] Step S4012: Calculate the cloud-edge conflict intensity based on the category conflict indicator, the confidence difference, the probability distribution difference, the defect risk direction, and the matching quality of the region of interest.

[0087] In one feasible implementation, the edge result can be represented as ,in For edge prediction categories, Let be the edge probability vector. To comprehensively assess the credibility of the edge, Match quality to ROI. Cloud results can be represented as... ,in For cloud-based prediction categories, This represents the probability vector in the cloud. This refers to the confidence level in the cloud. In this implementation, the edge probability vector... It can be represented as Cloud probability vector It can also be expressed as Edge prediction category and cloud prediction categories Both fall into one of the categories of normal and abnormal. Furthermore, the cloud-edge conflict intensity K can be calculated based on the following formula: ; in, For category conflict indicators, when the edge predicts the category With cloud prediction category When inconsistent A value of 1 indicates that the edge is predicted as a category. With cloud prediction category When consistent The value is 0; for example, when the edge inference outputs "normal" and the cloud inference outputs "abnormal", the function... If both edge inference and cloud inference output "abnormal", the function... .

[0088] This represents the confidence level difference term. For edge confidence parameters, This refers to the confidence parameter in the cloud. We can take the edge with the highest probability. Alternatively, the overall edge credibility can be taken. In this embodiment, Preferred selection This item represents the difference in confidence level between the edge and the cloud. Even if their output categories are the same, there may be discrepancies in confidence levels. For example, the edge inference category is "normal" and... The cloud-based inference category is normal and Although the two are of the same category, the cloud-based inference is very certain while the edge-based inference is uncertain, indicating that the edge model is not capable of identifying samples and should be marked as potential conflict samples or potential difficult sample samples.

[0089] The relationship between the edge probability vector and the cloud probability vector Divergence is used to measure the difference between two probability distributions; for example, edge probability vectors. Cloud probability vector Although both models predicted abnormal, the probability distributions were very different, indicating that although the edge model output the correct category, its internal judgment was unstable, and the conflict intensity of this sample should be increased.

[0090] This is a defect risk direction term, used to characterize the asymmetry of misclassification risks between "normal" and "abnormal" in industrial defect detection. In this laser printing defect detection scenario, the risk of missing a defective part by misclassifying it as a normal part is generally higher than the risk of misclassifying a normal part as a defective part. Therefore, when the edge prediction is "normal" while the cloud prediction is "abnormal," Take the higher value; when the edge prediction is abnormal while the cloud prediction is normal. Take the lower value; when the two predict the same category, It can be set to 0. This setting allows the system to assign higher priority to potential missed detection risks and conflicts. In one implementation, The values ​​are as follows: when the edge determines "normal" and the cloud determines "abnormal". This indicates a high risk, suggesting potential missed defects; when the edge determines "abnormal" and the cloud determines "normal",... This indicates medium risk, and there may be false positives; when both the edge and cloud determine "normal," This indicates low risk; when both the edge and cloud determine "abnormal," This indicates low risk.

[0091] Matching quality to the ROI. If the ROI positioning quality is low, discrepancies between edge and cloud results are more likely related to unstable input areas, positioning offsets, or image jitter. In the expression... This indicates that the lower the ROI quality, the larger this value, and the higher the conflict intensity K. If If the confidence level is 0.4, it indicates that the template matching confidence is low. In this case, even if the edge inference confidence is high, the edge result should not be used completely.

[0092] to This refers to the conflict intensity weighting coefficient. The conflict intensity weighting coefficient can be set empirically or determined through validation sets, field samples, grid search, Bayesian optimization, or other parameter calibration methods. In this example implementation, to After normalization, the sum is 1.

[0093] In the calculation of cloud-edge conflict intensity, the category conflict indicator significantly increases the conflict intensity when the predicted category at the edge is inconsistent with the predicted category in the cloud; the confidence difference term ensures that the conflict intensity can still reflect the implicit divergence even when the predicted categories on both sides are consistent but the confidence levels differ significantly; the probability distribution difference term takes into account the overall deviation of the probability distributions on both sides; the defect risk direction term gives higher weight to the missed detection risk conflict; and the matching quality of the region of interest increases the conflict intensity accordingly when the reliability of the input data is low.

[0094] In its implementation, this embodiment determines category conflict indicators and defect risk direction terms based on edge prediction categories and cloud prediction categories, determines confidence difference terms based on the combined edge confidence and cloud confidence, and determines probability distribution difference terms based on edge probability vectors and cloud probability vectors. It also calculates the cloud-edge conflict intensity in conjunction with the matching quality of the region of interest. This allows the cloud-edge conflict intensity to comprehensively measure the degree of discrepancy between edge and cloud inference results from five dimensions: category conflict, defect risk direction, confidence difference, probability distribution difference, and input quality. Specifically, the defect risk direction term indicates a higher severity of missed detection risk when the edge prediction is normal while the cloud prediction is abnormal; the confidence difference and probability distribution difference terms allow the cloud-edge conflict intensity to capture implicit discrepancies even when the prediction categories are the same, if there are significant differences in confidence or probability distribution; and the matching quality of the region of interest allows the cloud-edge conflict intensity to reflect the impact of insufficient positioning quality on conflict judgment when the input data reliability is low. The cloud-edge conflict intensity calculated by the above multi-dimensional approach can more comprehensively and precisely characterize the true degree of divergence between edge inference and cloud inference compared to the judgment method that relies solely on category consistency, providing a more accurate quantitative basis for subsequent defect detection decisions.

[0095] Furthermore, in order to extend conflict perception from a single numerical intensity calculation to a semantically meaningful type classification, thereby enabling subsequent decision-making processes to adopt more targeted processing strategies based on different conflict types, step S402 may further include: Step S41: When the edge prediction category is consistent with the cloud prediction category and the confidence difference is less than the confidence divergence threshold, the conflict type is determined to be non-conflict.

[0096] Step S42: When the edge prediction category is consistent with the cloud prediction category and the confidence difference term is greater than or equal to the confidence divergence threshold, the conflict type is determined to be a confidence divergence conflict.

[0097] Step S43: When the edge prediction category is inconsistent with the cloud prediction category, the conflict type is determined to be a category conflict.

[0098] Step S44: When the edge prediction category is normal, the cloud prediction category is defective, and the cloud confidence level is greater than or equal to the cloud high confidence threshold, the conflict type is determined to be a missed detection risk conflict.

[0099] Step S45: When the overall edge confidence is lower than the low confidence threshold and the cloud-edge conflict intensity is greater than or equal to the high conflict threshold, the conflict type is determined to be a low confidence severe conflict. Step S46: When the overall edge confidence is greater than or equal to the high confidence threshold, and the edge prediction category is inconsistent with the cloud prediction category after cloud review is triggered due to sampling inspection, batch anomaly, working condition anomaly or model drift detection strategy, or the cloud-edge conflict intensity is greater than or equal to the preset conflict threshold, the conflict type is determined to be a high confidence sampling inspection conflict. Step S47: When the same sample meets multiple conflict types, determine the main conflict type according to the priority of missed detection risk conflict, high confidence sampling conflict, low confidence serious conflict, category conflict, confidence level disagreement conflict, and no conflict.

[0100] It should be understood that when a sample simultaneously satisfies multiple conflict types, the primary conflict type is determined according to the priority of: missed detection risk conflict, high-confidence sampling conflict, low-confidence severe conflict, category conflict, confidence level disagreement conflict, and no conflict. Sample types may include: No conflict samples: ,and Less than the confidence level divergence threshold ,and Less than the probability distribution difference threshold ,and ; Confidence level disagreement / conflict sample: ,but Greater than or equal to the confidence divergence threshold ,or Greater than or equal to the probability distribution difference threshold This indicates that although the edge and the cloud belong to the same category, there is a significant difference in the degree of trust between them in that category; Category conflict samples: This indicates that the category labels output from the edge and the cloud are inconsistent; Sample of potential missed detection risks: , ,and Greater than or equal to the cloud-based high-trust threshold This indicates that the edge may misidentify a defective part as a normal part, and this type of conflict has the highest processing priority in defect detection; Low-confidence severe conflict samples: and This indicates that the overall credibility of the edge is low and the divergence between cloud and edge is significant. High-confidence sampling conflict samples: The samples meet the following requirements. Furthermore, due to issues such as random inspections, batch anomalies, abnormal operating conditions, or model drift detection strategies, the results uploaded to the cloud for review are inconsistent with the results from the edge. Or the conflict intensity K exceeds a preset threshold. These samples are used to identify high-confidence false positives at the edge, calibrate the grading threshold, and optimize the edge model.

[0101] Furthermore, for high-confidence samples, the system typically outputs results directly from the edge. However, under conditions of sampling cycle abnormalities, batch anomalies, operating condition anomalies, or model drift detection, the system still uploads some high-confidence samples to the cloud for review. If the cloud results conflict with the edge results, the sample is marked as a high-confidence sampling conflict sample and added to the difficult example library. In one embodiment, the system can be set to sample 1 out of every N high-confidence samples, or sample once every M seconds; when operating condition anomalies, batch anomalies, or an increased high-confidence sampling failure rate are detected, the system reduces N or M to increase the proportion of high-confidence samples reviewed in the cloud.

[0102] In its specific implementation, this embodiment categorizes the conflict types between edge-based and cloud-based inference into four types based on the consistency between the edge-predicted category and the cloud-based predicted category, the comparison results of the confidence difference item and the confidence divergence threshold, and the comparison results of the cloud-based confidence level and the cloud-based high-confidence threshold. These types include no conflict, confidence divergence conflict, category conflict, and missed detection risk conflict. Both no conflict and confidence divergence conflict correspond to situations where the edge-based and cloud-based predicted categories are consistent. However, confidence divergence conflict can further identify implicit divergences where the two ends have the same category but significantly different levels of confidence. Category conflict directly corresponds to situations where the edge-based and cloud-based predicted categories are inconsistent. Missed detection risk conflict specifically targets scenarios where the edge-based prediction is for a normal category while the cloud-based prediction is for a defect category with a high cloud-based confidence level. This type of conflict is independently identified as a missed detection risk conflict, highlighting the industry characteristic that the risk of missed defects is higher than the risk of false detection in laser printing defect detection. Through this classification method, this embodiment expands conflict perception from a single numerical intensity calculation to a semantically meaningful type classification, enabling subsequent decision-making processes to adopt more targeted processing strategies based on different conflict types.

[0103] Furthermore, in order to achieve a more reasonable trade-off between response speed and detection accuracy, step S402 may include: Step S4021: When the overall edge confidence is less than the high confidence threshold and greater than or equal to the low confidence threshold, and the conflict type is non-conflict, the edge probability vector and the cloud probability vector are weighted and fused to obtain the final probability vector. The category corresponding to the maximum value in the final probability vector is taken as the defect detection result of the image to be detected.

[0104] Step S4022: When the conflict type is a missed detection risk conflict, the cloud prediction category is used as the defect detection result of the image to be detected.

[0105] Step S4023: When the overall edge confidence is lower than the low confidence threshold and the cloud edge conflict intensity is greater than or equal to the high conflict threshold, the cloud prediction category is used as the defect detection result of the image to be detected, or the image to be detected is sent to the re-inspection and confirmation process.

[0106] Step S4024: When the conflict type is a confidence divergence conflict, the probability vector corresponding to the higher of the edge confidence and cloud confidence is used as the main factor, and the probability vector of the other end is combined for weighted fusion. Step S4025: When the conflict type is a category conflict, determine whether to use the edge result, cloud result, fusion result or re-check confirmation process based on the comprehensive edge confidence, cloud confidence, cloud-edge conflict intensity and the matching quality of the region of interest. Step S4026: When the conflict type is a high-confidence sampling conflict, the sample is marked as a high-confidence sampling failure sample and used for threshold calibration or edge model optimization.

[0107] The image detection method further includes: Step S4027: If it is determined that cloud review is not required, when the overall edge confidence is greater than or equal to the high confidence threshold, the edge prediction category is used as the defect detection result of the image to be detected.

[0108] In one feasible implementation, unlike directly overriding edge results with cloud results, this invention adopts different decision-making strategies based on different conflict types to reduce the risk of missed defect detections while minimizing invalid cloud calls. The specific differentiated collaborative reasoning decision-making is as follows: For highly reliable stable samples, i.e. when Furthermore, if no sampling inspection, batch anomaly, or operating condition anomaly is triggered, the system uses edge prediction of the category. As the final result, it is output in real time at the edge.

[0109] For a reliable, conflict-free sample, that is, when ,and ,and At that time, the system's edge probability vector and cloud probability vector A weighted fusion is performed to obtain the final probability vector Score, and the category corresponding to the maximum value in Score is taken as the final predicted category.

[0110] In one implementation, weighted fusion is represented as: ; in, ε is a very small positive number to prevent the denominator from being zero. The larger α is, the higher the edge weight; the larger 1-α is, the higher the cloud weight.

[0111] For confidence level discrepancies and conflicting samples, i.e. when the edge and cloud categories are consistent. But confidence level difference or probability distribution differences When the probability vector is large, the system prioritizes the end with higher overall credibility and combines it with the probability vector of the other end for weighted fusion; at the same time, the sample is marked as a potential hard case for subsequent analysis of the uncertainty of the edge model for this type of sample.

[0112] For class conflict samples, i.e. when At that time, the system compares and integrates the credibility of the edge terminals. Cloud-based confidence The conflict intensity K and the direction of defect risk. If the cloud confidence level is greater than the cloud high confidence threshold. And the conflict intensity is greater than or equal to the high conflict threshold. If the overall credibility of the edge computing is low, then the cloud-based result will be used as the final result; Significantly higher than cloud confidence level And ROI matching quality If the quality of the ROI is higher than the preset threshold, the system can use the edge results or perform weighted fusion; if neither the edge nor the cloud is trustworthy, the system will proceed to the re-inspection and confirmation process.

[0113] For samples with a risk of missed detection, i.e., when , ,and In this case, the system executes a defect-first strategy, that is, it prioritizes using the abnormal results from the cloud as the final result, or sends the sample into a re-inspection and confirmation process. In one implementation, the system introduces a defect risk correction item on top of the weighted fusion to enhance the score of the abnormal category: ; Here, β is the defect risk coefficient. This correction term improves the anomaly category score when anomalies are reliably identified in the cloud and the conflict intensity is high, thereby reducing the probability of missed defect detection.

[0114] For low-confidence, severely conflicting samples, i.e. when and When the system prioritizes high-precision inference results from the cloud, it will use the results with higher confidence levels from the cloud. Still below the preset threshold Alternatively, the system may further determine that the conflict intensity K exceeds the high conflict threshold. If the sample fails to pass the initial verification, it will be sent to the re-inspection and confirmation process. This process includes at least one of the following: manual verification, secondary image acquisition, cloud-based multi-model verification, and batch isolation re-inspection.

[0115] For high-confidence sampling conflict samples, i.e. when The samples were found to be positive after random sampling or verification of abnormal operating conditions. or At that time, the system executes cloud-first, defect-first, edge-cloud fusion, or re-inspection and confirmation strategies according to the conflict type; at the same time, the sample is marked as a high-confidence sampling failure sample for subsequent threshold calibration and edge model optimization.

[0116] Furthermore, after completing the differentiated collaborative inference decision, the system filters difficult example samples based on the comprehensive edge credibility, cloud verification results, conflict intensity, and final decision results. The difficult example samples include at least one of the following: low-credibility samples, confidence divergence conflict samples, category conflict samples, missed detection risk conflict samples, edge overload unloading samples, ROI location anomaly samples, and high-credibility sampling failure samples. The system saves the ROI image and corresponding inference metadata of the difficult example samples to the difficult example library. The inference metadata includes one or more of the following: ROI coordinates, ROI matching quality, edge prediction category, edge probability vector, comprehensive edge credibility, cloud prediction category, cloud probability vector, cloud confidence, conflict intensity, conflict type, triggering reason, edge load status, inference time, timestamp, and final label. The difficult example library is used for subsequent sample verification, model training, threshold calibration, conflict rule optimization, and quality traceability. By structurally saving the low-credibility and conflict samples generated during the inference process, this invention can transform uncertain samples in the cloud-edge collaborative inference process into subsequent optimization data sources, thereby improving the system's adaptability to changes in on-site operating conditions.

[0117] In this example implementation, when the number of difficult sample cases reaches a preset data volume threshold, the time elapsed since the last model optimization reaches a preset time threshold, or the number of conflicting samples of a certain type exceeds a preset proportion in consecutive batches, the cloud server triggers the model optimization process based on the difficult sample library.

[0118] The model optimization process includes at least one of the following: retraining or fine-tuning of the edge-based lightweight model, retraining or fine-tuning of the cloud-based high-precision model, model probability calibration, threshold parameter calibration, edge-based model format conversion, and model compression. The threshold parameters include one or more of the following: high confidence threshold, low confidence threshold, ROI quality threshold, load threshold, confidence divergence threshold, and conflict intensity threshold.

[0119] After model optimization is complete, the cloud server generates a model package or configuration file and distributes it to the edge device. The edge device performs integrity and compatibility checks on the model package or configuration file. If the checks pass, it updates the local model or threshold configuration. If inference anomalies or performance degradation occur after the update, it can be rolled back to a previous version. This step is used to achieve continuous optimization of the cloud-edge collaborative inference system, but it is not a necessary step for executing a single cloud-edge collaborative inference decision.

[0120] Please refer to Figure 6 , Figure 6 This is a schematic diagram illustrating the overall concept of the image detection method of this application. Combined with... Figure 6 As can be seen, the overall concept of this invention is: to obtain the ROI region of the image to be detected in real time at the edge, and after the lightweight model outputs the class probability, instead of deciding whether to upload to the cloud based solely on the single maximum probability, it calculates the comprehensive edge credibility. ;according to The edge load, ROI positioning quality, and operating status determine whether a sample enters the cloud for review. After cloud review, the conflict type and conflict intensity K are constructed based on the edge and cloud results. Then, the final inference strategy is selected based on K, cloud credibility, defect risk level R, and edge load status. At the same time, low-credibility samples, conflict samples, cloud-edge divergence samples, and overload unloading samples are saved as hard examples for subsequent training and model updates.

[0121] In laser printing defect detection, the edge device first extracts the Region of Interest (ROI) of the printed character / identifier using template matching or target area localization algorithms. After normalizing the ROI by size, converting its color space, and standardizing it, the data is input into a lightweight ONNX model, which outputs two probabilities: abnormal and normal. The system calculates the maximum class probability, the difference between the two probabilities, the prediction entropy, the ROI matching score, and the edge device's CPU load to generate a comprehensive edge device reliability score.

[0122] When the sample is of high confidence and the operating conditions are stable, the edge directly outputs the result; when the sample is of medium confidence or low confidence, the edge is overloaded, the ROI matching quality is insufficient, or the system is in a sampling inspection cycle, the ROI is compressed and uploaded to the cloud high-precision model for verification. After the cloud returns the category and probability, the system performs conflict detection: if the categories are different, an explicit category conflict occurs; if the categories are the same but the confidence difference or probability distribution difference exceeds the threshold, a implicit confidence conflict occurs; if the edge judges it as normal while the cloud judges it as abnormal and the cloud's confidence is high, a defect missed detection risk conflict occurs.

[0123] The system ultimately makes differentiated decisions based on the type of conflict: when there is no conflict and both ends are trustworthy, a weighted fusion result is output; for minor conflicts, the result is weighted according to trustworthiness; for severe conflicts or conflicts with a risk of missed detection, a cloud-based or defect-first strategy is prioritized; low-trust, severely conflicting samples are entered into a manual verification or re-examination queue. The above process forms a technical link of "rapid edge detection—comprehensive trustworthiness grading—selective cloud-based verification—conflict intensity perception—differentiated fusion—closed loop for difficult cases".

[0124] In its implementation, this embodiment employs four differentiated decision-making strategies based on the overall edge confidence level, conflict type, and cloud-edge conflict intensity: direct edge output, weighted edge-cloud fusion, cloud priority, and re-inspection confirmation. This ensures that the decision-making method no longer solely relies on cloud results to overwrite edge results, but rather matches the appropriate processing method to the sample's different confidence levels and conflict types. For samples with high overall edge confidence, direct edge output is used to ensure response speed; for moderately confident and conflict-free samples, weighted edge-cloud fusion is used to consider information from both ends; for conflict samples with a risk of missed detection, cloud-predicted categories are prioritized to reduce the probability of misclassifying defective parts as normal parts; for low-confidence and high-conflict samples, cloud-predicted categories are used or samples are sent to a re-inspection confirmation process to ensure the accuracy of high-risk samples. While ensuring rapid response for high-confidence samples, cloud information is introduced for fusion correction of moderately confident samples, and a more conservative high-precision priority or manual intervention strategy is adopted for conflict samples with a risk of missed detection and low-confidence severe conflict samples, thereby achieving a more reasonable trade-off between response speed and detection accuracy.

[0125] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the image detection method of this application. Any simple modifications based on this technical concept are within the scope of protection of this application. All actions requiring user authorization in this application (such as acquiring the region of interest of the image to be detected) are performed in compliance with the relevant data protection regulations of the country where the application is located and with authorization from the owner of the relevant device.

[0126] This application also provides an image detection device, please refer to... Figure 7 , Figure 7 This is a structural block diagram of the first embodiment of the image detection device of this application.

[0127] like Figure 7 As shown, the image detection apparatus proposed in this application includes: The local inference module 701 is used to obtain the region of interest of the image to be detected, and to perform local inference on the region of interest through the edge lightweight model to obtain the edge probability vector and the edge prediction category. Calculation module 702 is used to calculate the comprehensive edge credibility based on the edge probability vector, the matching quality of the region of interest, and the operating status parameters of the edge device; The cloud review triggering module 703 is used to upload the region of interest to the cloud server when it is determined that cloud review needs to be triggered, so that the cloud server can perform cloud inference on the region of interest and return the cloud inference result; The result determination module 704 is used to determine the defect detection result of the image to be detected based on the cloud inference result, the edge probability vector, the edge prediction category, and the comprehensive edge credibility.

[0128] This embodiment uses a lightweight edge-based model to perform local inference on the acquired region of interest (ROI), obtaining an edge probability vector and an edge prediction category. When cloud verification is required, the ROI is uploaded to a cloud server to obtain cloud inference results. The edge probability vector, edge prediction category, and overall edge confidence level are then combined to determine the defect detection result. On one hand, this scheme prioritizes local inference and preliminary judgment at the edge, uploading the ROI rather than the entire image only when cloud verification is required. This reduces network data transmission and cloud response time, mitigates privacy risks associated with the external transmission of production data, and ensures production line real-time performance and data security. On the other hand, after the cloud inference results are returned, the scheme uses a combined approach, integrating the edge probability vector, edge prediction category, and overall edge confidence level with the cloud inference results for joint judgment. This allows for correction and compensation of samples that are prone to misjudgment or omission at the edge, even under conditions such as changes in lighting, printing speed, equipment aging, or weak defect features, thanks to high-precision cloud inference. Through the above mechanism, this embodiment can effectively improve the accuracy of laser printing defect detection while ensuring the real-time performance of the production line and data security, thus taking into account the advantages of low latency at the edge and high precision in the cloud.

[0129] Based on the first embodiment of the image detection device described in this application, a second embodiment of the image detection device of this application is proposed.

[0130] In this embodiment, as one implementation method, the calculation module 702 is further configured to determine the maximum class probability, probability margin, and prediction entropy based on the edge probability vector, wherein the probability margin is the difference between the maximum class probability and the second largest class probability, and the prediction entropy is used to characterize the degree of disorder in the probability distribution of the edge probability vector; normalize the operating state parameters of the edge device to obtain the normalized edge load; and calculate the comprehensive edge credibility based on the maximum class probability, the probability margin, the prediction entropy, the matching quality of the region of interest, and the normalized operating load.

[0131] In one implementation, the cloud-based verification triggering module 703 is further configured to: determine that cloud verification needs to be triggered when the overall edge confidence is lower than a preset high confidence threshold; and / or determine that cloud verification needs to be triggered when the edge device load in the operating status parameters exceeds a preset load threshold; and / or determine that cloud verification needs to be triggered when the matching quality of the region of interest is lower than a preset quality threshold; and / or determine that cloud verification needs to be triggered when the image to be detected has image quality anomalies; and / or determine that cloud verification needs to be triggered when the fluctuation of the edge prediction category, the overall edge confidence, or the maximum category probability exceeds a preset fluctuation threshold; and / or determine that cloud verification needs to be triggered when the current period is within a high confidence sample sampling cycle.

[0132] In one implementation, the cloud inference result includes a cloud probability vector, a cloud prediction category, and a cloud confidence level. The result determination module 704 is further configured to calculate the cloud-edge conflict intensity based on the cloud inference result, the edge probability vector, the edge prediction category, and the comprehensive edge confidence level. The cloud-edge conflict intensity is used to characterize the degree of discrepancy between the edge inference result and the cloud inference result. The defect detection result of the image to be detected is determined based on the comprehensive edge confidence level, the cloud confidence level, and the cloud-edge conflict intensity.

[0133] In one implementation, the result determination module 704 is further configured to determine a category conflict indicator and a defect risk direction item based on the edge prediction category and the cloud prediction category; determine a confidence difference item based on the comprehensive edge confidence and the cloud confidence; determine a probability distribution difference item based on the edge probability vector and the cloud probability vector; and calculate the cloud-edge conflict intensity based on the category conflict indicator, the confidence difference item, the probability distribution difference item, the defect risk direction item, and the matching quality of the region of interest; wherein, the lower the matching quality of the region of interest, the higher the cloud-edge conflict intensity.

[0134] In one implementation, the result determination module 704 is further configured to: determine the conflict type as no conflict when the edge prediction category is consistent with the cloud prediction category and the confidence difference is less than the confidence divergence threshold; determine the conflict type as confidence divergence conflict when the edge prediction category is consistent with the cloud prediction category and the confidence difference is greater than or equal to the confidence divergence threshold; determine the conflict type as category conflict when the edge prediction category is inconsistent with the cloud prediction category; and determine the conflict type as missed detection when the edge prediction category is a normal category, the cloud prediction category is a defect category, and the cloud confidence is greater than or equal to the cloud high confidence threshold. Risk of conflict; when the overall edge confidence is lower than the low confidence threshold and the cloud-edge conflict intensity is greater than or equal to the high conflict threshold, the conflict type is determined to be a low confidence severe conflict; when the overall edge confidence is greater than or equal to the high confidence threshold, and the edge prediction category is inconsistent with the cloud prediction category after cloud review is triggered due to sampling inspection, batch anomaly, operating condition anomaly, or model drift detection strategy, or when the cloud-edge conflict intensity is greater than or equal to the preset conflict threshold, the conflict type is determined to be a high confidence sampling inspection conflict; when the same sample meets multiple conflict types, the main conflict type is determined according to the priority of missed detection risk conflict, high confidence sampling inspection conflict, low confidence severe conflict, category conflict, confidence divergence conflict, and no conflict.

[0135] In one implementation, the result determination module 704 is further configured to: when the overall edge confidence is less than a high confidence threshold but greater than or equal to a low confidence threshold, and the conflict type is no conflict, perform weighted fusion of the edge probability vector and the cloud probability vector to obtain a final probability vector, and use the category corresponding to the maximum value in the final probability vector as the defect detection result of the image to be detected; when the conflict type is a missed detection risk conflict, use the cloud predicted category as the defect detection result of the image to be detected; when the overall edge confidence is lower than a low confidence threshold and the cloud edge conflict intensity is greater than or equal to a high conflict threshold, use the cloud predicted category as the defect detection result of the image to be detected, or send the image to be detected for re-inspection. The confirmation process is as follows: When the conflict type is a confidence divergence conflict, the probability vector corresponding to the higher of the overall edge confidence and cloud confidence is used as the main factor, and the other probability vector is combined for weighted fusion; when the conflict type is a category conflict, the edge result, cloud result, fusion result, or re-inspection confirmation process is determined based on the overall edge confidence, cloud confidence, cloud-edge conflict intensity, and matching quality of the region of interest; when the conflict type is a high-confidence sampling conflict, the sample is marked as a high-confidence sampling failure sample and used for threshold calibration or edge model optimization; when it is determined that cloud review is not required, when the overall edge confidence is greater than or equal to the high-confidence threshold, the edge predicted category is used as the defect detection result of the image to be detected.

[0136] In addition, this application also proposes a cloud-edge collaborative industrial vision inspection system, including an edge system, a cloud system, and a communication interface; The edge system includes an ROI positioning module, an edge reasoning module, a comprehensive credibility calculation module, a cloud review triggering module, and an edge result receiving module; The cloud system includes a cloud-based high-precision inference module, a conflict perception module, a differential decision-making module, and a difficult example sample management module. The communication interface is used to upload ROI images and edge inference metadata, and return cloud inference results or final detection results.

[0137] The image detection device provided in this application, employing the image detection method described in the above embodiments, can solve the technical problem of overcoming the contradiction between limited computing power at the edge and transmission latency and data security in industrial quality inspection scenarios, thereby effectively improving the accuracy of defect detection while ensuring the real-time performance of the production line. Compared with the prior art, the beneficial effects of the image detection device provided in this application are the same as those of the image detection method provided in the above embodiments, and other technical features in the image detection device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0138] This application provides an image detection device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the image detection method in Embodiment 1 above.

[0139] The following reference Figure 8 The diagram illustrates a structural schematic of an image detection device suitable for implementing embodiments of this application. The image detection device in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 8 The image detection device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0140] like Figure 8As shown, the image detection device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory 1002 or a program loaded from a storage device 1003 into a random access memory 1004. The random access memory 1004 also stores various programs and data required for the operation of the image detection device. The processing unit 1001, the read-only memory 1002, and the random access memory 1004 are interconnected via a bus 1005. An input / output interface 1006 is also connected to the bus. Typically, the following systems can be connected to the input / output interface 1006: input devices 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. The communication device 1009 allows the image inspection device to communicate wirelessly or wiredly with other devices to exchange data. Although the figure shows image inspection devices with various systems, it should be understood that it is not required to implement or possess all of the systems shown. More or fewer systems may be implemented alternatively.

[0141] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0142] The image inspection device provided in this application, employing the image inspection method described in the above embodiments, can solve the technical problem of overcoming the contradiction between limited computing power at the edge and transmission latency and data security in industrial quality inspection scenarios, thereby effectively improving the accuracy of defect detection while ensuring the real-time performance of the production line. Compared with the prior art, the beneficial effects of the image inspection device provided in this application are the same as those of the image inspection method provided in the above embodiments, and other technical features of this image inspection device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0143] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0144] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0145] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to perform the image detection method described in the above embodiments.

[0146] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0147] The aforementioned computer-readable storage medium may be included in the image detection device; or it may exist independently and not be assembled into the image detection device.

[0148] The aforementioned computer-readable storage medium carries one or more programs that, when executed by the image detection device, enable the image detection device to write computer program code for performing the operations of this application in one or more programming languages ​​or a combination thereof. These programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++; and also conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, such as a local area network (LAN) or a wide area network (WAN), or connected to an external computer (e.g., via the Internet using an Internet service provider).

[0149] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0150] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0151] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described image detection method. This solves the technical problem of overcoming the contradiction between limited computing power at the edge and transmission latency and data security in cloud environments during industrial quality inspection, thereby effectively improving the accuracy of defect detection while ensuring the real-time performance of the production line. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the image detection method provided in the above embodiments, and will not be repeated here.

[0152] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the image detection method described above.

[0153] The computer program product provided in this application can solve the technical problem of how to overcome the contradiction between limited computing power at the edge and transmission latency and data security in cloud environments in industrial quality inspection scenarios, thereby effectively improving the accuracy of defect detection while ensuring the real-time performance of the production line. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the image detection method provided in the above embodiments, and will not be repeated here.

[0154] The above description is only a part of the embodiments of this application and does not limit the scope of protection of this application. All equivalent structural transformations made under the technical concept of this application and using the content of this application specification and drawings, or direct / indirect applications in other related technical fields, are included in the scope of protection of this application.

Claims

1. An image detection method, characterized in that, The method is applied to edge devices, and the method includes the following steps: The region of interest in the image to be detected is obtained, and local inference is performed on the region of interest through the edge lightweight model to obtain the edge probability vector and the edge prediction category. The overall edge credibility is calculated based on the edge probability vector, the matching quality of the region of interest, and the operating status parameters of the edge device. If it is determined that cloud review needs to be triggered, the region of interest is uploaded to the cloud server so that the cloud server can perform cloud inference on the region of interest and return the cloud inference result; Based on the cloud inference results, the edge probability vector, the edge prediction category, and the comprehensive edge credibility, the defect detection result of the image to be detected is determined.

2. The image detection method as described in claim 1, characterized in that, The step of calculating the comprehensive edge credibility based on the edge probability vector, the matching quality of the region of interest, and the operating status parameters of the edge device includes: The maximum class probability, probability margin, and prediction entropy are determined based on the edge probability vector. The probability margin is the difference between the maximum class probability and the second largest class probability. The prediction entropy is used to characterize the degree of disorder in the probability distribution of the edge probability vector. The operating status parameters of the edge device are normalized to obtain the normalized edge load. The overall edge confidence is calculated based on the maximum category probability, the probability margin, the prediction entropy, the matching quality of the region of interest, and the normalized runtime load. ； in, To comprehensively assess the credibility of the edge, Let H be the maximum class probability, Margin be the probability margin, and H be the prediction entropy. To predict the maximum entropy, For the matching quality of the region of interest, To normalize the operating load, to These are the weighting coefficients.

3. The image detection method as described in claim 1, characterized in that, After the step of calculating the comprehensive edge reliability based on the edge probability vector, the matching quality of the region of interest, and the operating status parameters of the edge device, the method further includes: If the overall edge credibility is lower than the preset high credibility threshold, it is determined that cloud-based review needs to be triggered. And / or, if the edge device load in the operating status parameters exceeds a preset load threshold, it is determined that a cloud review needs to be triggered; And / or, if the matching quality of the region of interest is lower than a preset quality threshold, it is determined that a cloud-based review needs to be triggered; And / or, if the image to be detected has abnormal image quality, it is determined that a cloud-based review needs to be triggered; And / or, if the fluctuation of the edge prediction category, the comprehensive edge confidence, or the maximum category probability exceeds a preset fluctuation threshold, it is determined that cloud review needs to be triggered; And / or, if it is determined that cloud-based verification needs to be triggered when the current sampling period for high-confidence samples is underway.

4. The image detection method as described in claim 1, characterized in that, The cloud inference results include cloud probability vectors, cloud prediction categories, and cloud confidence levels. The step of determining the defect detection result of the image to be detected based on the cloud inference result, the edge probability vector, the edge prediction category, and the comprehensive edge confidence includes: The cloud-edge conflict intensity is calculated based on the cloud inference result, the edge probability vector, the edge prediction category, and the comprehensive edge credibility. The cloud-edge conflict intensity is used to characterize the degree of discrepancy between the edge inference result and the cloud inference result. The defect detection result of the image to be detected is determined based on the comprehensive edge confidence, the cloud confidence, and the cloud-edge conflict intensity.

5. The image detection method as described in claim 4, characterized in that, The step of calculating the cloud-edge conflict intensity based on the cloud inference result, the edge probability vector, the edge prediction category, and the comprehensive edge credibility includes: Based on the edge prediction category and the cloud prediction category, a category conflict indication item and a defect risk direction item are determined; based on the comprehensive edge confidence and the cloud confidence, a confidence difference item is determined; and based on the edge probability vector and the cloud probability vector, a probability distribution difference item is determined. The cloud-edge conflict intensity is calculated based on the category conflict indicator, the confidence difference, the probability distribution difference, the defect risk direction, and the matching quality of the region of interest. The lower the matching quality of the region of interest, the higher the cloud-edge conflict intensity.

6. The image detection method as described in claim 5, characterized in that, Before the step of determining the defect detection result of the image to be detected based on the comprehensive edge confidence, the cloud confidence, and the cloud-edge conflict intensity, the method further includes: When the edge prediction category is consistent with the cloud prediction category and the confidence difference is less than the confidence divergence threshold, the conflict type is determined to be no conflict. When the edge prediction category is consistent with the cloud prediction category and the confidence difference item is greater than or equal to the confidence divergence threshold, the conflict type is determined to be a confidence divergence conflict. When the edge prediction category is inconsistent with the cloud prediction category, the conflict type is determined to be a category conflict; When the edge prediction category is normal, the cloud prediction category is defective, and the cloud confidence level is greater than or equal to the cloud high confidence threshold, the conflict type is determined to be a missed detection risk conflict. When the overall edge confidence is lower than the low confidence threshold and the cloud-edge conflict intensity is greater than or equal to the high conflict threshold, the conflict type is determined to be a low confidence severe conflict. When the overall edge confidence is greater than or equal to the high confidence threshold, and the edge prediction category is inconsistent with the cloud prediction category after cloud review is triggered due to sampling inspection, batch anomaly, working condition anomaly or model drift detection strategy, or when the cloud-edge conflict intensity is greater than or equal to the preset conflict threshold, the conflict type is determined to be a high confidence sampling inspection conflict. When the same sample meets multiple conflict types, the primary conflict type is determined according to the priority of missed detection risk conflict, high-confidence sampling conflict, low-confidence severe conflict, category conflict, confidence level disagreement conflict, and no conflict.

7. The image detection method as described in claim 6, characterized in that, The step of determining the defect detection result of the image to be detected based on the comprehensive edge confidence, the cloud confidence, and the cloud-edge conflict intensity includes: When the overall edge confidence is less than the high confidence threshold and greater than or equal to the low confidence threshold, and the conflict type is non-conflict, the edge probability vector and the cloud probability vector are weighted and fused to obtain the final probability vector, and the category corresponding to the maximum value in the final probability vector is taken as the defect detection result of the image to be detected. When the conflict type is a missed detection risk conflict, the cloud prediction category is used as the defect detection result of the image to be detected; When the overall edge confidence is lower than the low confidence threshold and the cloud-edge conflict intensity is greater than or equal to the high conflict threshold, the cloud prediction category is used as the defect detection result of the image to be detected, or the image to be detected is sent to the re-inspection and confirmation process. When the conflict type is a confidence divergence conflict, the probability vector corresponding to the higher of the edge confidence and cloud confidence is used as the main factor, and the probability vector of the other end is combined for weighted fusion. When the conflict type is a category conflict, the edge-end confidence, cloud confidence, cloud-edge conflict intensity and region of interest matching quality are considered to determine whether to use the edge-end result, cloud result, fusion result or re-checking confirmation process. When the conflict type is a high-confidence sampling conflict, the sample is marked as a high-confidence sampling failure sample and used for threshold calibration or edge model optimization. The image detection method further includes: If it is determined that cloud-based review is not required, when the overall edge confidence is greater than or equal to the high confidence threshold, the edge prediction category is used as the defect detection result of the image to be detected.

8. An image detection device, characterized in that, The image detection device includes: The local inference module is used to obtain the region of interest of the image to be detected, and to perform local inference on the region of interest through the edge lightweight model to obtain the edge probability vector and the edge prediction category. The calculation module is used to calculate the comprehensive edge credibility based on the edge probability vector, the matching quality of the region of interest, and the operating status parameters of the edge device; The cloud review triggering module is used to upload the region of interest to the cloud server when it is determined that cloud review needs to be triggered, so that the cloud server can perform cloud inference on the region of interest and return the cloud inference result; The result determination module is used to determine the defect detection result of the image to be detected based on the cloud inference result, the edge probability vector, the edge prediction category, and the comprehensive edge credibility.

9. An image detection device, characterized in that, The device includes: a memory, a processor, and an image detection program stored in the memory and executable on the processor, the image detection program being configured to implement the steps of the image detection method as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and the storage medium stores an image detection program, which, when executed by a processor, implements the steps of the image detection method as described in any one of claims 1 to 7.

Images