An industrial anomaly detection method and device based on cloud-edge collaboration

Abstract

The present invention discloses an industrial anomaly detection method and device based on cloud-edge collaboration, comprising: configuring a cloud server containing a large model and an edge client containing an anomaly detection model at an industrial site; using a camera to collect and label image data of products on the industrial production line in real time; using the image data to train the anomaly detection model on the edge client; uploading the trained anomaly detection model parameters to the cloud server for aggregation, collaboratively generating an integrated soft label through the large model and multiple small models, and using the integrated soft label to update the parameters of the anomaly detection model on the edge client; cyclically training and updating the anomaly detection model, and detecting the real-time collected image data in parallel. The present invention integrates the generalization capability of the large model with the local experience of the small model, enhances the collective wisdom of the model, significantly improves the accuracy, stability and cross-line generalization capability of anomaly detection on the industrial assembly line, and meets the real-time requirements under the condition of limited edge device resources.

Description

Technical Field

[0001] The present invention relates to the field of industrial Internet of Things, and in particular to an industrial anomaly detection method and device based on cloud-edge collaboration. Background Art

[0002] In the Industrial Internet of Things (IIoT), product anomaly detection is a crucial component in ensuring product quality on the production line. Deep learning and neural network technologies, due to their superior performance in image processing and pattern recognition, are widely used in industrial product anomaly detection tasks, such as surface flaw detection and product defect identification. However, product anomaly detection still faces several technical challenges. Traditional centralized approaches rely on cloud servers to process and analyze image data collected from the production line. While these approaches offer high detection accuracy, they struggle to meet the real-time detection and data security requirements of industrial scenarios due to limitations in network bandwidth, transmission latency, and data privacy. Furthermore, existing technologies often focus on improving detection accuracy, resulting in overly large models that are difficult to run efficiently on edge devices with limited computing resources. Achieving efficient product anomaly detection in a distributed environment while balancing real-time performance and resource utilization remains a pressing technical challenge. Summary of the Invention

[0003] To address the above issues, the present invention proposes an industrial anomaly detection method and device based on cloud-edge collaboration to achieve efficient and real-time product anomaly detection in industrial sites. To achieve the above invention objectives, the present invention proposes the following technical solution: an industrial anomaly detection method based on cloud-edge collaboration, comprising the following steps:

[0004] S1,Industrial site configuration includes a cloud server containing a large model and an edge client containing an anomaly detection model;

[0005] S2. Use cameras to collect and label image data of products on industrial production lines in real time;

[0006] S3. Using the image data, train an anomaly detection model on the edge client; uploading the trained anomaly detection model parameters to the cloud server for aggregation, generating an integrated soft label by collaborating the large model with multiple small models, and using the integrated soft label to update the parameters of the anomaly detection model on the edge client;

[0007] S4. Circulate the training and updating of the anomaly detection model and perform detection on the real-time collected image data in parallel.

[0008] Furthermore, the configuration includes a cloud server including a large model and an edge client including an anomaly detection model, specifically including:

[0009] Each production line is equipped with an edge client device and a server in the cloud. The edge client communicates with the cloud server through the network. The edge client deploys a small, resource-constrained anomaly detection model that uses a lightweight convolutional neural network structure, while the cloud server deploys a large multimodal pre-trained model. The cloud server is responsible for aggregating and optimizing data and model parameters from multiple clients, enabling the small edge model to acquire the global generalization capabilities of the large model.

[0010] Furthermore, the real-time collection of image data of products on the industrial production line using a camera specifically includes: adjusting the shooting frequency of the camera so that the captured images can completely cover each product on the production line.

[0011] Furthermore, the training of the anomaly detection model includes: updating parameters of the anomaly detection model using a cross entropy loss function.

[0012] Furthermore, uploading the trained anomaly detection model parameters to the cloud server for aggregation specifically includes: extracting part of the anomaly detection model, uploading the network parameters therein to the server, and performing weighted averaging on the small model parameters through the FedAvg algorithm.

[0013] Furthermore, the step of collaboratively generating an integrated soft label by using a large model and multiple small models specifically includes:

[0014] For each sample in the cloud training set: use the large model to perform forward propagation to generate a predicted distribution; use each selected client small model to perform forward propagation to generate a predicted distribution; calculate the entropy of the predicted distribution of each selected client small model as the prediction confidence for the current sample; based on the prediction confidence, take a weighted average of the predicted distribution of each small model to obtain the small model integrated prediction; fuse the large model predicted distribution with the small model integrated prediction to form the final integrated soft label.

[0015] Furthermore, the use of the integrated soft label to update the parameters of the anomaly detection model of the edge client includes:

[0016] The aggregated parameters are used as the parameters of the aggregated small model. The knowledge distillation method is adopted, and the integrated soft labels are used as teacher signals to guide the training of the aggregated small model. The parameters of the aggregated small model are updated through gradient descent, and the updated aggregated small model parameters are sent to the edge client. The local model parameters and the parameters sent from the cloud are fused according to the weights of the cloud model update.

[0017] Furthermore, the update weights based on the cloud model are specifically fused with the local model parameters and the parameters sent from the cloud:

[0018] The aggregation model extracts the mean of all sample feature vectors of the client's local dataset, then calculates the mean of all sample feature vectors of the cloud training set, and calculates the cosine similarity between the feature vector means of the local dataset and the cloud training set. The obtained cosine similarity is normalized and used as the weight of the aggregation parameter, which is then integrated and updated with the parameters of the client's small model.

[0019] According to another aspect of the specification, an industrial anomaly detection device based on cloud-edge collaboration is also provided, including a memory and one or more processors, wherein the memory stores executable code, and when the processor executes the executable code, the industrial anomaly detection method based on cloud-edge collaboration is implemented.

[0020] According to another aspect of the specification, a computer-readable storage medium is also provided, on which a program is stored. When the program is executed by a processor, the industrial anomaly detection method based on cloud-edge collaboration is implemented.

[0021] Beneficial effects:

[0022] (1) The method proposed in the present invention deploys a lightweight anomaly detection model on the edge client for local training and real-time inference, which significantly reduces the demand for computing power and storage resources of the edge device. It effectively solves the problem in the background technology that the model is too large to run efficiently on resource-constrained edge devices, and improves the real-time performance of detection.

[0023] (2) The present invention innovatively adopts a knowledge transfer mechanism in which a large model and multiple small models collaborate to generate integrated soft labels. In particular, in the process of generating integrated soft labels, the present invention introduces a dynamic weighting method based on the prediction confidence of each small model for the current processing sample to integrate the prediction results of multiple small models. The predictions of the dynamically weighted integrated small models are then fused with the predictions of the large model through preset weights. This mechanism not only balances the global generalization ability of the large model with the local expertise of the small models through fusion weights, but also enables the integration process to intelligently and adaptively adjust the contribution of each small model for each sample through dynamic weighting of confidence, giving priority to the opinions of small models that are more accurate and confident in predicting the current sample, thereby generating higher quality and more discriminative soft labels. This achieves a more refined and efficient knowledge exchange between the large model and the small models, and between the small models, overcoming the limitations of simple averaging or fixed weight fusion of multiple teacher models in traditional knowledge distillation, significantly enhancing the collective wisdom of the edge model, and improving the accuracy and stability of anomaly detection.

[0024] (3) The proposed method combines the large-scale and distributed characteristics of actual industrial production processes and adopts a cloud-edge federated learning framework to achieve the co-evolution of model anomaly detection capabilities on multiple production lines. In the model parameter update stage, a dynamic weight allocation mechanism based on the similarity between the client's local data distribution and the global distribution is introduced. This design objectively quantifies the degree to which different clients rely on cloud knowledge and adaptively integrates local model parameters with cloud-migrated parameters during parameter updates. This effectively addresses the challenge of balancing global model consistency and local data adaptability in a federated learning environment, ensuring the model's robustness under diverse working conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 This is a flow chart of an industrial anomaly detection method based on cloud-edge collaboration provided by an embodiment of the present invention;

[0026] Figure 2 This is a system architecture diagram of the industrial product anomaly detection method provided by an embodiment of the present invention;

[0027] Figure 3 A schematic diagram of an industrial anomaly detection device based on cloud-edge collaboration provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0028] The specific implementation method and working principle of the present invention are described in detail below with reference to the accompanying drawings:

[0029] like Figure 1 As shown, the present invention proposes an industrial anomaly detection method based on cloud-edge collaboration to achieve efficient and real-time product anomaly detection in industrial sites. Specifically, the method steps are described as follows:

[0030] Step S1: Configure the cloud server and edge client at the industrial site.

[0031] In the industrial site where this embodiment is located, each production line is equipped with an edge client device and a server is configured in the cloud. The edge client communicates with the cloud server through the network. The edge client deploys a small resource-constrained anomaly detection model. The small anomaly detection model adopts a lightweight convolutional neural network structure. In this embodiment, MobileNetV3 is used. This minimizes the number of model parameters and computational complexity while ensuring a certain level of detection accuracy to adapt to the resource limitations of the edge device. The cloud server deploys a multimodal pre-trained large model. In this embodiment, the visual-language alignment pre-trained model CLIP is used. This large model is pre-trained on a dataset containing a large number of image-text pairs. It has powerful cross-modal understanding and feature extraction capabilities and can provide rich prior knowledge for the small edge model. The cloud server is responsible for aggregating and optimizing data and model parameters from multiple clients, so that the small edge model can obtain the global generalization capabilities of the large model.

[0032] In this embodiment, there are Production lines, corresponding Edge Clients and Small model , and has 1 cloud server and 1 multimodal pre-trained large model .

[0033] Step S2: Use a camera to collect image data of products on the industrial production line in real time.

[0034] In this embodiment, the camera on the production line is responsible for collecting image data of products on the production line in real time. According to the product type, production speed and other conditions of the production line, the camera's shooting frequency is adjusted to ensure that the image can fully cover every product on the production line. The camera will collect a large number of pictures. Randomly select a small proportion of them and manually label them to form picture-label pairs. ,in Indicates a picture, These image-label pairs constitute the training set .Will Transmit to the corresponding edge client and cloud servers.

[0035] Step S3: Use the image data to train an anomaly detection model.

[0036] like Figure 2 As shown in the figure, in the cloud-edge collaborative framework, the anomaly detection model is trained through the following process:

[0037] Step S301: train a small anomaly detection model on the edge client.

[0038] In this embodiment, each edge client is based on the training set , for small models deployed on the client Perform training. The update formula for the small model parameters is:

[0039]

[0040] in, are the parameters of the current client model, is the learning rate for local training of the model, is the cross entropy loss function, and the formula is:

[0041]

[0042]

[0043] Step S302: The network parameters of the small model are uploaded to the cloud server for aggregation.

[0044] In this embodiment, a portion of all clients are randomly selected to form a set The following operations will be performed by the set It is performed on the client and cloud server in the process.

[0045] The edge client uploads the trained small model parameters to the cloud server. In the cloud server, the FedAvg algorithm is used to perform weighted averaging on the small model parameters to obtain the aggregated small model. , the formula is:

[0046]

[0047] in, is the aggregated small model parameter, For the client Dataset The number of samples.

[0048] In step S303 , the large model and multiple small models collaborate to generate integrated soft labels and update the aggregated small model.

[0049] In this embodiment, the training sets of the selected clients are uploaded to the cloud server and merged into the cloud training set.

[0050]

[0051] For cloud training sets Each sample in , perform the following operations to generate an integrated soft label:

[0052] a. Use a large model Perform forward propagation to obtain the predicted distribution ;

[0053] b. Use each selected client model (i.e. the uploaded model) performs forward propagation respectively to obtain the predicted distribution ;

[0054] c. Calculate the prediction confidence of each small model. Specifically, for the The predicted distribution of a small model , its confidence It is calculated by the entropy of its predicted distribution. The calculation formula is:

[0055]

[0056] in, represents a collection of categories, represents the total number of categories, Represents a small model Predict that the current sample belongs to the category probability.

[0057] d. Based on the confidence of each small model for the current sample , its predicted distribution Perform weighted averaging to obtain the small model ensemble prediction:

[0058]

[0059] e. Distribute the predictions of the large model Ensemble prediction with small models Fusion is performed to form the final integrated soft label:

[0060]

[0061] in, is the preset fusion weight. In this embodiment, it is possible to , which is used to balance the global knowledge of the large model and the local experience of the small model based on dynamic integration of confidence.

[0062] Adopting knowledge distillation method and using integrated soft labels Acts as a teacher signal to guide the aggregation of small models Calculate the knowledge transfer loss:

[0063]

[0064] Update the aggregated small model parameters by gradient descent:

[0065]

[0066] in, is the updated small model parameter, This process enables the aggregated small model to simultaneously learn the generalization ability of the large model and the local experience of multiple small edge models, achieving multi-directional knowledge exchange.

[0067] Step S304: redistribute the small model after knowledge migration to the edge client to update the network parameters.

[0068] In this embodiment, the cloud server sets the small model parameters Send to collection The edge client in the client updates the weights based on the objectively calculated cloud model To integrate local model parameters and parameters sent from the cloud. Weight The calculation method is as follows:

[0069] a. Feature extraction: Use the current aggregated small model Extract client local dataset The feature vector of all samples.

[0070] b. Calculate the feature mean: Calculate the mean of the feature vector of the client local data set .

[0071] c. Calculate the global feature mean: Calculate the cloud training set The mean of all sample eigenvectors .

[0072] d. Calculate cosine similarity: Calculate and The cosine similarity between:

[0073]

[0074] e. Get the normalized weight:

[0075]

[0076] The client updates its model parameters:

[0077]

[0078] In this embodiment, the above steps S301 to S304 are continuously cycled, and the small model of the edge client is continuously optimized through the collaborative knowledge transfer mechanism to improve the anomaly detection performance. In this process, the cloud server can dynamically adjust the frequency and fusion weight of executing cloud-edge collaborative training according to the actual needs of the production line. and related hyperparameters for model updating.

[0079] Step S4: Use an anomaly detection model to detect the image data.

[0080] In this embodiment, the edge client uses the current small model to infer real-time image data and generate anomaly detection results. The inference process runs in parallel with model training to ensure real-time performance on the production line. When the small model is updated, the new model parameters are immediately applied to the inference task, providing real-time feedback to production line operators.

[0081] Corresponding to the aforementioned embodiment of an industrial anomaly detection method based on cloud-edge collaboration, the present invention also provides an embodiment of an industrial anomaly detection device based on cloud-edge collaboration.

[0082] See also Figure 3 An embodiment of the present invention provides an industrial anomaly detection device based on cloud-edge collaboration, including a memory and one or more processors. The memory stores executable code, and when the processor executes the executable code, it is used to implement an industrial anomaly detection method based on cloud-edge collaboration in the above embodiment.

[0083] An embodiment of an industrial anomaly detection device based on cloud-edge collaboration provided by the present invention can be applied to any device with data processing capabilities, and the device with data processing capabilities can be a device or apparatus such as a computer. The device embodiment can be implemented through software, or through hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, it is formed by the processor of any device with data processing capabilities in which it is located reading the corresponding computer program instructions in the non-volatile memory into the memory for execution. From the hardware level, if Figure 3 As shown in the figure, it is a hardware structure diagram of any device with data processing capability where an industrial anomaly detection device based on cloud-edge collaboration provided by the present invention is located. Figure 3 In addition to the processor, memory, network interface, and non-volatile memory shown, any device with data processing capabilities in which the apparatus in the embodiment is located may also include other hardware, generally based on the actual functions of the device with data processing capabilities, which will not be described in detail.

[0084] The implementation process of the functions and effects of each unit in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, and will not be repeated here.

[0085] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to the partial description of the method embodiments. The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present invention. A person of ordinary skill in the art can understand and implement the present invention without inventive work.

[0086] An embodiment of the present invention also provides a computer-readable storage medium on which a program is stored. When the program is executed by a processor, an industrial anomaly detection method based on cloud-edge collaboration in the above embodiment is implemented.

[0087] The computer-readable storage medium may be an internal storage unit of any device with data processing capabilities described in any of the aforementioned embodiments, such as a hard disk or memory. The computer-readable storage medium may also be an external storage device of any device with data processing capabilities, such as a plug-in hard disk, a smart media card (SMC), an SD card, a flash card, etc. equipped on the device. Furthermore, the computer-readable storage medium may also include both an internal storage unit and an external storage device of any device with data processing capabilities. The computer-readable storage medium is used to store the computer program and other programs and data required by any device with data processing capabilities, and may also be used to temporarily store data that has been output or is to be output.

[0088] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements the industrial anomaly detection method based on cloud-edge collaboration.

[0089] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the contents disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present application that follow the general principles of the present application and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered merely as exemplary, and the true scope and spirit of the present application are indicated by the claims.

[0090] It should be understood that the above general description and the detailed description that follows are exemplary and explanatory only and do not limit the present application. The present application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes may be made without departing from the scope of the present application. The scope of the present application is limited only by the appended claims.

Claims

1. An industrial anomaly detection method based on cloud-edge collaboration, characterized in that: The following steps are involved: S1,Industrial site configuration includes a cloud server containing a large model and an edge client containing an anomaly detection model; S2. Use cameras to collect and label image data of products on industrial production lines in real time; S3. Using the image data, train an anomaly detection model on the edge client; uploading the trained anomaly detection model parameters to the cloud server for aggregation, generating an integrated soft label by collaborating the large model with multiple small models, and using the integrated soft label to update the parameters of the anomaly detection model on the edge client; The method of collaboratively generating an integrated soft label by using a large model and multiple small models specifically includes: For each sample in the cloud training set: forward propagation is performed using the large model to generate a predicted distribution; forward propagation is performed using each selected client-side small model to generate a predicted distribution; the entropy of the predicted distribution of each selected client-side small model is calculated as the prediction confidence for the current sample; based on the prediction confidence, the predicted distribution of each small model is weighted averaged to obtain the small model ensemble prediction; the large model predicted distribution is fused with the small model ensemble prediction to form the final ensemble soft label; S4. Circulate the training and updating of the anomaly detection model and perform detection on the real-time collected image data in parallel.

2. The industrial anomaly detection method based on cloud-edge collaboration according to claim 1 is characterized in that: The configuration includes a cloud server with a large model and an edge client with an anomaly detection model, specifically including: Each production line is equipped with an edge client device and a server in the cloud. The edge client communicates with the cloud server through the network. The edge client deploys a small, resource-constrained anomaly detection model that uses a lightweight convolutional neural network structure, while the cloud server deploys a large multimodal pre-trained model. The cloud server is responsible for aggregating and optimizing data and model parameters from multiple clients, enabling the small edge model to acquire the global generalization capabilities of the large model.

3. The industrial anomaly detection method based on cloud-edge collaboration according to claim 1 is characterized in that: The method of using a camera to collect image data of products on an industrial production line in real time specifically includes adjusting the camera's shooting frequency so that the captured images can completely cover each product on the production line.

4. The industrial anomaly detection method based on cloud-edge collaboration according to claim 1 is characterized in that: The training of the anomaly detection model includes: updating the parameters of the anomaly detection model using a cross entropy loss function.

5. The industrial anomaly detection method based on cloud-edge collaboration according to claim 1 is characterized in that: The uploading of the trained anomaly detection model parameters to the cloud server for aggregation specifically includes: extracting part of the anomaly detection model, uploading the network parameters therein to the server, and performing weighted averaging of the small model parameters through the FedAvg algorithm.

6. The industrial anomaly detection method based on cloud-edge collaboration according to claim 1 is characterized in that: The use of the integrated soft label to update the parameters of the anomaly detection model of the edge client includes: The aggregated parameters are used as the parameters of the aggregated small model. The knowledge distillation method is adopted, and the integrated soft labels are used as teacher signals to guide the training of the aggregated small model. The parameters of the aggregated small model are updated through gradient descent, and the updated aggregated small model parameters are sent to the edge client. The local model parameters and the parameters sent from the cloud are fused according to the weights of the cloud model update.

7. The industrial anomaly detection method based on cloud-edge collaboration according to claim 6 is characterized in that: The method of updating the weights based on the cloud model to fuse the local model parameters and the parameters sent from the cloud is as follows: The aggregation model extracts the mean of all sample feature vectors of the client's local dataset, then calculates the mean of all sample feature vectors of the cloud training set, and calculates the cosine similarity between the feature vector means of the local dataset and the cloud training set. The obtained cosine similarity is normalized and used as the weight of the aggregation parameter, which is then integrated and updated with the parameters of the client's small model.

8. An industrial anomaly detection device based on cloud-edge collaboration, comprising a memory and one or more processors, wherein the memory stores executable code, characterized in that: When the processor executes the executable code, it implements an industrial anomaly detection method based on cloud-edge collaboration as described in any one of claims 1-7.

9. A computer-readable storage medium having a program stored thereon, characterized in that: When the program is executed by the processor, an industrial anomaly detection method based on cloud-edge collaboration as described in any one of claims 1 to 7 is implemented.

Images