A defect detection method based on cloud edge collaboration, a cloud end and an electronic device

Abstract

The application discloses a defect detection method based on cloud edge cooperation, a cloud end and an electronic device, and is used for improving resource utilization. The method of the application comprises the following steps: acquiring resource state information and device performance information sent by an edge end; determining defect identification resource information of the edge end according to the resource state information and the device performance information; performing light-weight processing on a complete defect identification model in the local according to the defect identification resource information to obtain a target defect identification model, wherein the complete defect identification model is an image recognition model based on a neural network; and sending the target defect identification model to the edge end, so that the edge end performs a defect identification task through the target defect identification model.

Description

Technical Field

[0001] This application relates to the field of defect detection technology, and in particular to a defect detection method based on cloud-edge collaboration, cloud computing, and electronic devices. Background Technology

[0002] In industrial production and infrastructure operation and maintenance, defect detection, as a crucial means of ensuring product quality and structural safety, is increasingly relying on deep learning-based image recognition methods. In glass production, timely and accurate detection of surface defects such as bubbles, cracks, and inclusions is of great significance for improving product yield and reducing material waste. Furthermore, defect detection technology is widely used in various fields, including solder joint inspection in electronic component manufacturing, quality control of automotive parts, and surface inspection of building materials.

[0003] To enhance the intelligence of inspection, many systems choose to deploy complete defect identification models in the cloud, leveraging its powerful computing capabilities, and upload the collected image data to the cloud for centralized processing. However, with the rapid increase in the number of terminal devices, the risks of bandwidth consumption, data latency, and privacy leaks brought about by this centralized processing model are becoming increasingly prominent. Especially on high-speed production lines such as glass production lines, where the volume of image data is large and the real-time requirements are high, centralized processing methods are difficult to meet the real-time feedback needs on-site.

[0004] To reduce cloud load, some solutions have begun to deploy models at the edge to perform defect identification tasks. However, edge devices generally face limitations in computing power, storage space, and network stability, making it difficult to run complete models directly, leading to deployment failures or low efficiency. To adapt to resource constraints, some methods attempt to design lightweight model structures, but these methods often struggle to balance model versatility and accuracy, and are difficult to dynamically adapt to the resource conditions of different edge devices.

[0005] Therefore, in the existing technology, there is a lack of a means to fully assess the status of edge resources and configure the identification model accordingly. It is often impossible to reasonably allocate computing and storage resources for different devices, resulting in some devices having idle resources and some devices having excessive load, which affects the overall system performance and leads to low resource utilization. Summary of the Invention

[0006] This application provides a cloud-edge collaborative defect detection method, cloud platform, and electronic device, which can improve resource utilization.

[0007] The first aspect of this application provides a defect detection method based on cloud-edge collaboration, including:

[0008] Obtain resource status information and device performance information sent from the edge device;

[0009] The defect identification resource information of the edge terminal is determined based on the resource status information and the device performance information;

[0010] Based on the defect identification resource information, a lightweight processing is performed on the local complete defect identification model to obtain the target defect identification model. The complete defect identification model is an image recognition model based on a neural network.

[0011] The target defect identification model is sent to the edge device so that the edge device can perform a defect identification task through the target defect identification model.

[0012] Optionally, after sending the target defect identification model to the edge, the method further includes:

[0013] Receive suspicious defect information sent by the edge terminal, wherein the suspicious defect information is a suspicious defect identified by the edge terminal when performing the defect identification task using the target defect identification model;

[0014] Based on the complete defect identification model, defect identification is performed on the suspicious defect information to obtain the identification result;

[0015] The identification result is sent to the edge end.

[0016] Optionally, before performing lightweight processing on the local complete defect identification model based on the defect identification resource information, the method further includes:

[0017] The original image, preprocessed image, and preprocessing algorithm sent by the edge device are obtained. The preprocessed image is obtained by the edge device processing the original image according to the preprocessing algorithm.

[0018] Based on the original image and the preprocessed image, analyze the quality assessment data of the preprocessing algorithm performed at the edge; the quality assessment data includes at least processing effect indicators and / or resource consumption indicators.

[0019] Based on the quality assessment data and the preset optimization objectives, the preprocessing algorithm is optimized to generate an optimized algorithm; the optimization objectives include at least reducing resource consumption.

[0020] The optimization algorithm is sent to the edge device so that the edge device updates the preprocessing algorithm according to the optimization algorithm.

[0021] Optionally, after optimizing the preprocessing algorithm based on the quality assessment data and the preset optimization target to generate the optimized algorithm, the method further includes:

[0022] The defect identification resource information is updated according to the optimization algorithm.

[0023] Optionally, before sending the target defect identification model to the edge, the method further includes:

[0024] A virtual machine is created based on the defect identification resource information;

[0025] The target defect identification model is validated and its parameters are adjusted based on the virtual machine and the preprocessed image.

[0026] Optionally, determining the defect identification resource information at the edge based on the resource status information and the device performance information includes:

[0027] The total resource overhead, excluding the defect identification task, is assessed based on the aforementioned resource status information.

[0028] Obtain the preset redundancy resource threshold;

[0029] The defect identification resource information of the edge terminal is determined based on the device performance information, the total resource overhead, and the redundancy resource threshold.

[0030] Optionally, the step of performing lightweight processing on the local complete defect identification model based on the defect identification resource information to obtain the target defect identification model includes:

[0031] Based on the defect identification resource information, the computing power threshold and model deployment constraints of the edge are determined, so as to perform pruning on the complete defect identification model, and delete neural network connections and nodes that have little impact on defect identification accuracy but consume a lot of resources, to obtain a pruned model;

[0032] Based on the storage capacity constraint of the edge end in the defect identification resource information, weight quantization processing is performed on the pruning model to convert the floating-point weights in the model into fixed-point weights with low bit width, thus obtaining a weight conversion model.

[0033] Based on the weight transformation model as the teacher model, and combined with the constraints on inference latency and throughput performance in the defect identification resource information, a student model that satisfies the constraints is trained to fit the output of the teacher model, and the student model is used as the target defect identification model.

[0034] A second aspect of this application provides a cloud platform, including:

[0035] The acquisition unit is used to acquire resource status information and device performance information sent from the edge device.

[0036] The determining unit is used to determine the defect identification resource information of the edge end based on the resource status information and the device performance information;

[0037] The lightweight unit is used to perform lightweight processing on the local complete defect recognition model according to the defect recognition resource information to obtain the target defect recognition model, wherein the complete defect recognition model is an image recognition model based on a neural network.

[0038] The first sending unit is used to send the target defect identification model to the edge terminal, so that the edge terminal can perform a defect identification task through the target defect identification model.

[0039] Optionally, the cloud further includes a second sending unit, the second sending unit being used for:

[0040] Receive suspicious defect information sent by the edge terminal, wherein the suspicious defect information is a suspicious defect identified by the edge terminal when performing the defect identification task using the target defect identification model;

[0041] Based on the complete defect identification model, defect identification is performed on the suspicious defect information to obtain the identification result;

[0042] The identification result is sent to the edge end.

[0043] Optionally, the cloud further includes a third sending unit, which is used for:

[0044] The original image, preprocessed image, and preprocessing algorithm sent by the edge device are obtained. The preprocessed image is obtained by the edge device processing the original image according to the preprocessing algorithm.

[0045] Based on the original image and the preprocessed image, analyze the quality assessment data of the preprocessing algorithm performed at the edge; the quality assessment data includes at least processing effect indicators and / or resource consumption indicators.

[0046] Based on the quality assessment data and the preset optimization objectives, the preprocessing algorithm is optimized to generate an optimized algorithm; the optimization objectives include at least reducing resource consumption.

[0047] The optimization algorithm is sent to the edge device so that the edge device updates the preprocessing algorithm according to the optimization algorithm.

[0048] Optionally, the cloud also includes an update unit, which is used for:

[0049] The defect identification resource information is updated according to the optimization algorithm.

[0050] Optionally, the cloud also includes a verification unit, which is used for:

[0051] A virtual machine is created based on the defect identification resource information;

[0052] The target defect identification model is validated and its parameters are adjusted based on the virtual machine and the preprocessed image.

[0053] Optionally, the determining unit is specifically used for:

[0054] The total resource overhead, excluding the defect identification task, is assessed based on the aforementioned resource status information.

[0055] Obtain the preset redundancy resource threshold;

[0056] The defect identification resource information of the edge terminal is determined based on the device performance information, the total resource overhead, and the redundancy resource threshold.

[0057] Optionally, the lightweight unit is specifically used for:

[0058] Based on the defect identification resource information, the computing power threshold and model deployment constraints of the edge are determined, so as to perform pruning on the complete defect identification model, and delete neural network connections and nodes that have little impact on defect identification accuracy but consume a lot of resources, to obtain a pruned model;

[0059] Based on the storage capacity constraint of the edge end in the defect identification resource information, weight quantization processing is performed on the pruning model to convert the floating-point weights in the model into fixed-point weights with low bit width, thus obtaining a weight conversion model.

[0060] Based on the weight transformation model as the teacher model, and combined with the constraints on inference latency and throughput performance in the defect identification resource information, a student model that satisfies the constraints is trained to fit the output of the teacher model, and the student model is used as the target defect identification model.

[0061] A third aspect of this application provides an electronic device, comprising:

[0062] Processor, memory, input / output units, and bus;

[0063] The processor is connected to the memory, the input / output unit, and the bus;

[0064] The memory stores a program, and the processor calls the program to execute the method of the first aspect and any possible implementation of the first aspect.

[0065] The fourth aspect of this application provides a computer-readable storage medium storing a program that, when executed on a computer, causes the computer to perform the methods of the first aspect and any possible implementation thereof.

[0066] As can be seen from the above technical solutions, this application has the following advantages:

[0067] In this embodiment, the cloud accurately judges the defect identification resource information of the edge based on the resource status information and device performance information sent by the edge, thereby performing precise lightweight processing on the complete defect identification model. This allows the generated target defect identification model to fully utilize its available computing power and memory capabilities without wasting edge resources, achieving reasonable allocation and efficient use of resources, which is conducive to improving resource utilization. Attached Figure Description

[0068] Figure 1 This is a flowchart illustrating an embodiment of the cloud-edge collaborative defect detection method in this application;

[0069] Figure 2 This is a flowchart illustrating one embodiment of sending identification results to the edge in this application;

[0070] Figure 3 This is a flowchart illustrating one embodiment of sending the optimization algorithm to the edge in this application;

[0071] Figure 4 This is a flowchart illustrating one embodiment of creating a virtual machine in this application;

[0072] Figure 5 This is a flowchart illustrating one embodiment of determining defect identification resource information at the edge end in this application;

[0073] Figure 6 This is a flowchart illustrating one embodiment of the lightweighting process in this application;

[0074] Figure 7 This is a schematic diagram of the structure of one embodiment of the cloud in this application;

[0075] Figure 8 This is a schematic diagram of the structure of one embodiment of the electronic device in this application. Detailed Implementation

[0076] This application provides a cloud-edge collaborative defect detection method, cloud platform, and electronic device to improve resource utilization.

[0077] The embodiments of this application will now be described with reference to the accompanying drawings.

[0078] Please see Figure 1 , Figure 1 An embodiment of the cloud-edge collaborative defect detection method provided in this application includes:

[0079] 101. Obtain resource status information and device performance information sent from the edge device;

[0080] The cloud receives resource status and device performance information from the edge devices. Resource status information includes dynamic metrics such as current computing resource usage, memory utilization, storage space availability, and network bandwidth. Device performance information includes static hardware parameters such as processor type, memory capacity, configuration of acceleration units (e.g., GPU or NPU), and power consumption levels. Through the communication connection established with the edge devices, the cloud can continuously monitor the uploading process of this information, gaining timely insight into the real-time resource status and device capabilities of the edge devices.

[0081] 102. Determine the defect identification resource information at the edge based on resource status information and equipment performance information;

[0082] After collecting resource status and device performance information from the edge, the cloud analyzes this information to comprehensively evaluate the computing resources, memory resources, and network transmission capabilities available for performing defect identification tasks at the edge, thus determining the defect identification resource information for the edge. This defect identification resource information describes the acceptable range for the edge in terms of model size, computational complexity, running speed, and resource consumption for the defect identification model, serving as a reference for subsequent lightweight processing.

[0083] 103. Perform lightweight processing on the local complete defect identification model based on the defect identification resource information to obtain the target defect identification model. The complete defect identification model is an image recognition model based on a neural network.

[0084] By combining defect recognition resource information from the edge, the cloud selects a complete defect recognition model locally and performs lightweight processing to meet the operating conditions of the edge. The complete defect recognition model is a neural network-based image recognition model, typically containing multiple convolutional layers and fully connected layers, with high recognition accuracy and parameter complexity. To adapt to the resource constraints of the edge, the cloud employs techniques such as pruning, quantization, structural simplification, or knowledge distillation to lightweight the model, generating a target defect recognition model with smaller parameter size, faster inference speed, and lower memory consumption, while maintaining the model's recognition accuracy as much as possible.

[0085] 104. Send the target defect identification model to the edge terminal so that the edge terminal can perform defect identification tasks through the target defect identification model.

[0086] After the target defect identification model is built, the cloud sends the model to the edge device via the network. During transmission, auxiliary information related to model deployment can be included, such as model input / output formats, version number, and inference interface descriptions, to ensure the edge device can correctly receive and deploy the target defect identification model. Once the model is distributed, the edge device can use it to process locally acquired image data for defect identification tasks, achieving localized task execution.

[0087] In this embodiment, the cloud accurately determines the defect identification resource information at the edge based on the resource status and device performance information transmitted from the edge. This allows for precise, lightweight processing of the complete defect identification model, enabling the generated target defect identification model to fully utilize its available computing power and memory capabilities without wasting edge resources. This achieves rational allocation and efficient use of resources, thereby improving resource utilization. Furthermore, after deployment at the edge, the target defect identification model can perform on-site image processing, reducing data backhaul, lowering network load, and simultaneously improving the response speed of defect identification and the adaptability and scalability of the overall detection system.

[0088] Please see Figure 2 In some embodiments of this application, after step 104 of the above embodiments sends the target defect identification model to the edge, the defect detection method based on cloud-edge collaboration may further include the following steps:

[0089] 201. Receive suspicious defect information sent by the edge terminal. Suspicious defect information refers to suspicious defects identified by the edge terminal when performing defect identification tasks using the target defect identification model.

[0090] The cloud receives suspicious defect information sent from the edge device. This information is marked by the edge device based on the confidence level of the target defect recognition model when performing defect identification tasks. The target defect recognition model is configured with a high classification threshold during deployment, especially when judging an image as "defect-free." Only when the model's confidence level reaches the preset high threshold is the image considered defect-free; otherwise, even if the model does not clearly detect a defect, the image will be marked as suspicious and sent to the cloud as suspicious defect information. This mechanism makes defect judgment more cautious and avoids misjudging images as defect-free.

[0091] 202. Based on the complete defect identification model, perform defect identification on the suspicious defect information and obtain the identification results;

[0092] Upon receiving suspicious defect information, the cloud-based system invokes the full defect identification model to re-identify the information. This full defect identification model possesses higher recognition capabilities, handling more complex image features and samples that are difficult to determine at the edge. Through this model's analysis, the cloud can obtain more accurate identification results, determining whether a defect truly exists, thus compensating for the limitations of edge-based systems in terms of model capabilities and computational accuracy.

[0093] 203. Send the recognition results to the edge.

[0094] After the cloud-based system completes the identification process, it sends the results back to the edge device. These results may include the identified defect type, location information, confidence level, and optional processing suggestions. The edge device can then perform subsequent operations such as result confirmation, data updates, or manual review based on the identification results, thus completing the identification process in a closed loop.

[0095] In this embodiment, the target defect identification model sets a high classification threshold when running at the edge, making the "no defect" judgment more cautious. Only when the model has a sufficiently high confidence level in the absence of a defect is the result confirmed; otherwise, the result is marked as a suspicious defect and sent to the cloud for processing. This strategy effectively raises the judgment threshold in defect identification tasks, thereby improving identification security while ensuring that difficult samples can be further analyzed by the complete defect identification model in the cloud, thus achieving accuracy compensation. Through this cloud-edge collaborative mechanism, on the one hand, efficient utilization of edge resources is achieved, handling only most simple tasks and reducing computational resource waste; on the other hand, the powerful computing capabilities of the cloud for identifying complex samples are preserved, balancing processing efficiency and identification quality, ultimately improving the overall performance of the entire defect detection system in terms of resource utilization, missed detection control, identification accuracy, and system stability.

[0096] Please see Figure 3 In some embodiments of this application, before step 103 in the above embodiments performs lightweight processing on the local complete defect identification model based on defect identification resource information, the cloud-edge collaborative defect detection method may further include the following steps:

[0097] 301. Obtain the original image, preprocessed image, and preprocessing algorithm sent by the edge device. The preprocessed image is obtained by the edge device processing the original image according to the preprocessing algorithm.

[0098] The cloud receives image-related data from the edge device, including the original image, a pre-processed image generated from the original image, and the pre-processing algorithm used to generate the pre-processed image. The original image is the raw image data collected on-site by the edge device, while the pre-processed image is the image result generated by the edge device after performing pre-processing algorithms locally. These pre-processing algorithms may involve operations such as image cropping, noise reduction, enhancement, and contrast adjustment. By acquiring this data, the cloud can comprehensively understand the algorithmic strategies and processing effects used by the edge device when performing pre-processing tasks.

[0099] 302. Based on the original image and the preprocessed image, analyze the quality assessment data of the preprocessing algorithm at the edge; the quality assessment data shall include at least the processing effect index and / or resource consumption index.

[0100] After acquiring the original and preprocessed images, the cloud compares the differences between them and, combined with known preprocessing algorithm logic, analyzes their performance at the edge, generating quality assessment data. This quality assessment data includes at least two categories: one is processing effect metrics, such as image sharpness improvement, defect visibility enhancement, and feature preservation integrity; the other is resource consumption metrics, such as CPU utilization, memory usage, algorithm execution time, and power consumption. The cloud evaluates these metrics to determine the applicability and execution efficiency of the preprocessing algorithm at the edge.

[0101] 303. Based on the quality assessment data and the preset optimization objectives, optimize the preprocessing algorithm to generate an optimized algorithm; the optimization objectives should at least include reducing resource consumption.

[0102] Based on the quality assessment data analyzed in step 302, the cloud platform optimizes the preprocessing algorithm according to preset optimization objectives. These objectives include at least reducing resource consumption when the algorithm is executed at the edge, such as reducing execution time, compressing memory usage, and reducing power consumption. Simultaneously, streamlining the processing logic can be considered without sacrificing image processing performance. Through parameter adjustment, structural simplification, or operator replacement, the cloud platform generates an optimized algorithm to replace the current preprocessing algorithm running at the edge.

[0103] 304. Send the optimization algorithm to the edge so that the edge can update the preprocessing algorithm according to the optimization algorithm.

[0104] After the optimization algorithm is generated, it is sent from the cloud to the edge device via the network. The optimization algorithm can include complete algorithm logic, parameter configuration, and calling methods, making it easy for the edge device to directly deploy and replace the original preprocessing algorithm. After receiving the optimization algorithm, the edge device can perform update operations, making subsequent processing of the original image more lightweight and efficient, while maintaining the image quality requirements of the recognition task.

[0105] 305. Update the defect identification resource information according to the optimization algorithm.

[0106] Since the preprocessing algorithm has a direct impact on the overall resource consumption at the edge, the cloud reassesses its impact on resource status after the optimized algorithm is deployed and updates the defect identification resource information accordingly. The updated defect identification resource information more accurately reflects the available resources at the edge under the new preprocessing algorithm conditions, providing a more precise parameter constraint and adaptation basis for the subsequent lightweight target defect identification model.

[0107] In this embodiment, after acquiring the original image, preprocessed image, and preprocessing algorithm from the edge, the cloud analyzes its execution effect and resource consumption to generate quality assessment data. Based on preset optimization targets, the cloud optimizes the preprocessing algorithm and then sends the optimized algorithm back to the edge, updating the preprocessing flow accordingly. Finally, the defect identification resource information is updated. During this process, the preprocessing algorithm is specifically optimized, allowing the edge to maintain image processing quality while reducing computational resources and energy consumption, thereby effectively improving overall resource utilization. Simultaneously, the updated defect identification resource information provides a more accurate basis for the lightweighting of the target defect identification model, enabling the model to more reasonably adapt to the current state of the edge, further improving the stability of identification performance and deployment efficiency. Furthermore, through proactive optimization of the preprocessing stage, the system achieves multi-stage collaboration from image acquisition and processing to identification, giving the entire cloud-edge collaborative defect detection method stronger dynamic adaptability and sustainable optimization capabilities.

[0108] Please see Figure 4 In some embodiments of this application, before sending the target defect identification model to the edge in step 104 of the above embodiments, the defect detection method based on cloud-edge collaboration may further include the following steps:

[0109] 401. Create a virtual machine based on the defect identification resource information;

[0110] After determining the defect identification resource information, the cloud creates a virtual machine based on this information. The created virtual machine's resource configuration is consistent with or highly similar to the actual operating environment at the edge. Specifically, the cloud configures the same or similar computing environment parameters based on the edge's processor architecture, memory capacity, acceleration unit type, and runtime limitations to build a virtual execution environment that simulates the real operating conditions at the edge, providing a controllable test platform for subsequent model verification and parameter adjustment.

[0111] 402. Verify and adjust the parameters of the target defect identification model based on the virtual machine and pre-processed images.

[0112] After the virtual machine is built, the cloud uses this virtual machine's runtime environment and previously acquired pre-processed images to validate and adjust the parameters of the lightweight target defect recognition model. The validation process includes testing the model's inference speed, memory usage, accuracy, and stability, with a focus on its performance under edge resource constraints. Based on the test results, the cloud can further fine-tune the model's parameters, such as adjusting input resolution, confidence thresholds, and batch processing strategies, to ensure the model achieves a balance between performance and effectiveness before deployment to the edge.

[0113] In this embodiment, before sending the target defect identification model, the cloud first creates a virtual machine consistent with the edge environment based on defect identification resource information. On this virtual machine, the model is validated and its parameters are adjusted using pre-processed images. This allows the target defect identification model to complete performance optimization in an equivalent resource environment before formal deployment, thus avoiding resource waste or operational failures due to model incompatibility after deployment and improving resource utilization. Furthermore, pre-deployment simulation validation further enhances the model's stability and availability in actual operation, reducing the risks associated with edge deployment.

[0114] Please see Figure 5 In some embodiments of this application, step 102 in the above embodiments determines the defect identification resource information of the edge end based on resource status information and device performance information, which may specifically include the following steps:

[0115] 501. Evaluate the total resource overhead excluding defect identification tasks based on resource status information;

[0116] After receiving resource status information from the edge device, the cloud assesses the resources currently used by running tasks on the edge device based on this information to calculate the total resource overhead in the system, excluding defect identification tasks. The assessment may include CPU utilization, memory usage, GPU utilization, bandwidth usage, etc., to understand the resource usage of the edge device under the current load and prepare for space estimation for defect identification tasks.

[0117] 502. Obtain the preset redundancy resource threshold;

[0118] The cloud retrieves a preset redundancy resource threshold from the configuration to reserve sufficient safety margin during resource scheduling. This redundancy resource threshold is used to prevent resource contention or performance interference from other important tasks at the edge after the defect identification task has run, ensuring system stability and multi-task coexistence capabilities. This threshold can be preset according to device type or business requirements, or it can be dynamically configured.

[0119] 503. Determine the defect identification resource information at the edge based on equipment performance information, total resource overhead, and redundant resource threshold.

[0120] After obtaining device performance information, current total resource overhead, and redundancy thresholds, the cloud comprehensively analyzes the available resources at the edge that can actually be allocated for defect identification tasks. Based on the device's theoretical maximum processing capacity minus the current resource overhead and redundancy thresholds, the types and upper limits of remaining available resources are calculated, such as available memory size and CPU / GPU availability ratio. This ultimately forms defect identification resource information, which guides subsequent lightweight model processing.

[0121] In this embodiment, the cloud assesses the total resource overhead of the current task at the edge, introduces a preset redundancy resource threshold, and combines this with device performance information to determine the resources available for defect identification. This ensures that available space is accurately reserved during resource assessment, effectively avoiding over- or under-allocation of resources and improving resource utilization. Furthermore, the introduction of a redundancy resource protection mechanism during the assessment process ensures that defect identification tasks do not compete for resources with other critical tasks, further enhancing system stability and the coordination of multi-task scheduling.

[0122] Please see Figure 6 In some embodiments of this application, step 103 in the above embodiments performs lightweight processing on the local complete defect identification model based on defect identification resource information to obtain the target defect identification model, which may specifically include the following steps:

[0123] 601. Based on the defect identification resource information, determine the computing power threshold and model deployment constraints at the edge end, so as to perform pruning on the complete defect identification model, delete neural network connections and nodes that have little impact on defect identification accuracy but consume a lot of resources, and obtain the pruned model;

[0124] After acquiring defect identification resource information, the cloud first extracts the edge computing power thresholds, such as the maximum acceptable number of floating-point operations (FLOPs), the peak computing power of available CPUs or GPUs, and model deployment constraints, such as maximum model size and real-time requirements. Using these parameters as constraints, the cloud performs pruning on the locally stored complete defect identification model, removing network connections and neurons that have a small impact on the final classification result but high resource consumption. For example, if the complete model contains multiple residual modules or densely connected structures, the cloud may prune some convolutional layers with a large number of channels or redundant intermediate layers to reduce computation and model size, thus generating a pruned model.

[0125] 602. Based on the storage capacity constraints of the edge end in the defect identification resource information, perform weight quantization processing on the pruning model to convert the floating-point weights in the model into fixed-point weights with low bit width, and obtain the weight transformation model.

[0126] After obtaining the pruned model, the cloud further reads the storage capacity limitations in the defect identification resource information, such as the maximum available storage capacity or the upper limit of model parameter loading. Based on these limitations, the cloud performs weight quantization on the pruned model, converting the 32-bit floating-point weights that occupy a large amount of storage space in the original model into fixed-point weights with lower bit widths, such as 8-bit integers or 4-bit fixed-point formats. During the quantization process, the cloud can use symmetric or asymmetric quantization strategies to compress the model size as much as possible while ensuring model accuracy. For example, in some edge devices, after the weights are compressed from float32 to int8, the overall model size can be reduced to one-quarter of the original, which helps to improve loading speed and save memory resources, ultimately resulting in a weight-converted model.

[0127] 603. Based on the weight transformation model as the teacher model, and combined with the constraints on inference latency and throughput performance in the defect identification resource information, a student model that satisfies the constraints is trained to fit the output of the teacher model, and the student model is used as the target defect identification model.

[0128] After weight quantization, the cloud uses this weighted transformation model as the teacher model and trains a lighter student model using knowledge distillation. The student model's design considers both the inference latency (e.g., the maximum acceptable processing time per image) and throughput performance (e.g., the number of images that can be processed per unit time) requirements of the defect recognition resource information. For example, if the resource information indicates that the edge device requires an inference time of no more than 100 milliseconds, the cloud will prioritize building a shallower network structure with smaller convolutional kernels and control the number of model channels to meet latency constraints. During training, the student model learns the output distribution of the teacher model on different image samples, thereby achieving the goal of replicating the teacher model's recognition performance with lower computational cost. After training, this student model becomes the final target defect recognition model.

[0129] In this embodiment, the cloud performs pruning, weight quantization, and knowledge distillation on the complete defect identification model sequentially based on defect identification resource information. This ensures that the final target defect identification model meets the resource constraints of the edge in terms of computational load, storage size, and runtime latency, thereby effectively improving resource utilization. In particular, by finely pruning the neural network structure and quantizing the weight data, the model size and computational burden are significantly reduced, while the student model trained through knowledge distillation further optimizes the model's inference efficiency and execution speed. Furthermore, the entire model lightweighting process is dynamically generated based on the actual capabilities of the edge, enabling the deployed model to balance identification accuracy and system response, enhancing the model's deployability and practicality.

[0130] Please see Figure 7 , Figure 7One embodiment of the cloud provided in this application includes:

[0131] The acquisition unit 701 is used to acquire resource status information and device performance information sent from the edge terminal;

[0132] The determining unit 702 is used to determine the defect identification resource information at the edge end based on the resource status information and the equipment performance information;

[0133] The lightweight unit 703 is used to perform lightweight processing on the local complete defect recognition model based on defect recognition resource information to obtain the target defect recognition model. The complete defect recognition model is an image recognition model based on a neural network.

[0134] The first sending unit 704 is used to send the target defect identification model to the edge end so that the edge end can perform the defect identification task through the target defect identification model.

[0135] Optionally, the cloud also includes a second sending unit, which is used for:

[0136] Receive suspicious defect information sent by the edge terminal. Suspicious defect information refers to suspicious defects identified by the edge terminal when performing defect identification tasks using the target defect identification model.

[0137] Defect identification is performed on suspicious defect information based on the complete defect identification model, and the identification results are obtained.

[0138] Send the recognition results to the edge.

[0139] Optionally, the cloud also includes a third sending unit, which is used for:

[0140] The system acquires the original image, preprocessed image, and preprocessing algorithm sent from the edge device. The preprocessed image is obtained by the edge device processing the original image according to the preprocessing algorithm.

[0141] Based on the original image and the preprocessed image, analyze the quality assessment data of the preprocessing algorithm executed at the edge; the quality assessment data includes at least the processing effect index and / or resource consumption index.

[0142] Based on the quality assessment data and the preset optimization objectives, the preprocessing algorithm is optimized to generate an optimized algorithm; the optimization objectives include at least reducing resource consumption.

[0143] The optimization algorithm is sent to the edge device so that the edge device updates the preprocessing algorithm according to the optimization algorithm.

[0144] Optionally, the cloud also includes an update unit, which is used for:

[0145] The defect identification resource information is updated based on the optimization algorithm.

[0146] Optionally, the cloud also includes a verification unit, which is used for:

[0147] Create a virtual machine based on defect identification resource information;

[0148] The target defect identification model was validated and its parameters were adjusted based on the virtual machine and pre-processed images.

[0149] Optionally, the determining unit 702 is specifically used for:

[0150] The total resource overhead, excluding defect identification tasks, is assessed based on resource status information.

[0151] Obtain the preset redundancy resource threshold;

[0152] Defect identification resource information at the edge is determined based on equipment performance information, total resource overhead, and redundant resource threshold.

[0153] Optionally, the lightweight unit 703 is specifically used for:

[0154] Based on the defect identification resource information, the computing power threshold and model deployment constraints at the edge are determined so as to perform pruning on the complete defect identification model, deleting neural network connections and nodes that have little impact on defect identification accuracy but consume a lot of resources, thus obtaining the pruned model;

[0155] Based on the storage capacity constraints at the edge of the defect identification resource information, weight quantization processing is performed on the pruning model to convert the floating-point weights in the model into fixed-point weights with low bit width, thus obtaining the weight transformation model.

[0156] Based on the weight transformation model as the teacher model, and combined with the constraints on inference latency and throughput performance in the defect identification resource information, a student model that satisfies the constraints is trained to fit the output of the teacher model, and the student model is used as the target defect identification model.

[0157] In this implementation, the functions of each unit are as described above. Figures 1 to 6 The steps in the illustrated embodiments are the same and will not be repeated here.

[0158] Please see Figure 8 , Figure 8 One embodiment of the electronic device provided in this application includes:

[0159] Processor 801, memory 802, input / output unit 803, and bus 804;

[0160] The processor 801 is connected to the memory 802, the input / output unit 803, and the bus 804;

[0161] The memory 802 stores a program, which the processor 801 calls to execute. Figures 1 to 6 The steps in the illustrated embodiment.

[0162] In this embodiment, the function of processor 801 is the same as described above. Figures 1 to 6 The steps in the illustrated embodiments are the same and will not be repeated here.

[0163] This application also provides a computer-readable storage medium on which a program is stored. When the program is executed on a computer, it causes the computer to perform the aforementioned actions. Figures 1 to 6 The method in any possible implementation.

[0164] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0165] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.

[0166] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0167] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0168] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A defect detection method based on cloud-edge collaboration, applied in the cloud, characterized in that, include: Obtain resource status information and device performance information sent from the edge device; The defect identification resource information of the edge terminal is determined based on the resource status information and the device performance information; Based on the defect identification resource information, a lightweight processing is performed on the local complete defect identification model to obtain the target defect identification model. The complete defect identification model is an image recognition model based on a neural network. The target defect identification model is sent to the edge device so that the edge device can perform a defect identification task through the target defect identification model; Before performing lightweight processing on the local complete defect identification model based on the defect identification resource information, the method further includes: The original image, preprocessed image, and preprocessing algorithm sent by the edge device are obtained. The preprocessed image is obtained by the edge device processing the original image according to the preprocessing algorithm. Based on the original image and the preprocessed image, analyze the quality assessment data of the preprocessing algorithm performed at the edge; the quality assessment data includes at least processing effect indicators and / or resource consumption indicators. Based on the quality assessment data and the preset optimization objectives, the preprocessing algorithm is optimized to generate an optimized algorithm; the optimization objectives include at least reducing resource consumption. The optimization algorithm is sent to the edge device so that the edge device updates the preprocessing algorithm according to the optimization algorithm; After optimizing the preprocessing algorithm based on the quality assessment data and the preset optimization objective to generate the optimized algorithm, the method further includes: The defect identification resource information is updated according to the optimization algorithm; Before sending the target defect identification model to the edge end, the method further includes: A virtual machine is created based on the defect identification resource information; The target defect identification model is validated and its parameters are adjusted based on the virtual machine and the preprocessed image.

2. The method according to claim 1, characterized in that, After sending the target defect identification model to the edge, the method further includes: Receive suspicious defect information sent by the edge terminal, wherein the suspicious defect information is a suspicious defect identified by the edge terminal when performing the defect identification task using the target defect identification model; Based on the complete defect identification model, defect identification is performed on the suspicious defect information to obtain the identification result; The recognition result is sent to the edge end.

3. The method according to claim 1, characterized in that, The step of determining the defect identification resource information at the edge end based on the resource status information and the device performance information includes: The total resource overhead, excluding the defect identification task, is assessed based on the aforementioned resource status information. Obtain the preset redundancy resource threshold; The defect identification resource information of the edge terminal is determined based on the device performance information, the total resource overhead, and the redundancy resource threshold.

4. The method according to any one of claims 1 to 3, characterized in that, The step of performing lightweight processing on the local complete defect identification model based on the defect identification resource information to obtain the target defect identification model includes: Based on the defect identification resource information, the computing power threshold and model deployment constraints of the edge are determined, so as to perform pruning on the complete defect identification model, and delete neural network connections and nodes that have little impact on defect identification accuracy but consume a lot of resources, to obtain a pruned model; Based on the storage capacity constraint of the edge end in the defect identification resource information, weight quantization processing is performed on the pruning model to convert the floating-point weights in the model into fixed-point weights with low bit width, thus obtaining a weight conversion model. Based on the weight transformation model as the teacher model, and combined with the constraints on inference latency and throughput performance in the defect identification resource information, a student model that satisfies the constraints is trained to fit the output of the teacher model, and the student model is used as the target defect identification model.

5. A cloud platform, characterized in that, include: The acquisition unit is used to acquire resource status information and device performance information sent from the edge device; The determining unit is used to determine the defect identification resource information of the edge end based on the resource status information and the device performance information; The lightweight unit is used to perform lightweight processing on the local complete defect recognition model according to the defect recognition resource information to obtain the target defect recognition model, wherein the complete defect recognition model is an image recognition model based on a neural network. The first sending unit is used to send the target defect identification model to the edge terminal, so that the edge terminal can perform a defect identification task through the target defect identification model; The cloud also includes a third sending unit, which is used for: The original image, preprocessed image, and preprocessing algorithm sent by the edge device are obtained. The preprocessed image is obtained by the edge device processing the original image according to the preprocessing algorithm. Based on the original image and the preprocessed image, analyze the quality assessment data of the preprocessing algorithm performed at the edge; the quality assessment data includes at least processing effect indicators and / or resource consumption indicators. Based on the quality assessment data and the preset optimization objectives, the preprocessing algorithm is optimized to generate an optimized algorithm; the optimization objectives include at least reducing resource consumption. The optimization algorithm is sent to the edge device so that the edge device updates the preprocessing algorithm according to the optimization algorithm; The cloud also includes an update unit, which is used for: The defect identification resource information is updated according to the optimization algorithm; The cloud platform also includes a verification unit, which is used for: A virtual machine is created based on the defect identification resource information; The target defect identification model is validated and its parameters are adjusted based on the virtual machine and the preprocessed image.

6. An electronic device, characterized in that, include: Processor, memory, input / output units, and bus; The processor is connected to the memory, the input / output unit, and the bus; The memory stores a program, and the processor calls the program to execute the method as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium contains a program that, when executed on a computer, causes the computer to perform the method as described in any one of claims 1 to 4.

Images