Data forgetting method for dynamic allocation pruning ratio of heterogeneous resources of industrial equipment

Abstract

The invention discloses a data forgetting method for dynamically allocating pruning ratios for heterogeneous resources of industrial equipment, which comprises the following steps: when a server detects that target equipment fails, the server allocates adaptive pruning ratios to the industrial equipment according to resource information of the industrial equipment; pruning the local model by each industrial device according to the allocated pruning ratio, removing parameters related to the abnormal data of the target device, and realizing forgetting of the target data; other industrial devices except the target device update the pruned local model to obtain an updated local model and upload the updated local model to a server; and aggregating the updated local model through the server to obtain a global model which ensures generalization performance and forgets abnormal data of the target equipment. According to heterogeneous computing and communication resources of the equipment, a proper pruning ratio can be adaptively allocated to the equipment, so that the waiting time of all the equipment is reduced, the resource utilization rate of the system is improved, and the execution efficiency of forgotten tasks is improved.

Description

Technical Field

[0001] The present invention belongs to the field of Internet of Things and privacy protection technology, and in particular relates to a data forgetting method for dynamically allocating pruning ratios for heterogeneous resources of industrial equipment. Background Art

[0002] In the Industrial Internet, industrial equipment operates under high loads or in extreme weather conditions for extended periods, making it prone to operational anomalies and even sudden failures due to factors such as environmental corrosion and electromagnetic interference. These failures not only cause industrial equipment to generate abnormal or erroneous data in a short period of time, but can also cause the equipment to fall out of the training cluster, negatively impacting the global federated learning model. To address this issue, Federated Unlearning (Federated Unlearning) has been proposed. This technology can selectively remove the contribution of abnormal data (also called target data) from a faulty device (also called a target device), thereby mitigating the adverse effects of abnormal data and device failures on the global model.

[0003] When implementing federated forgetting learning, existing solutions typically use channel pruning, removing channels in convolutional neural networks whose correlation with the target data exceeds a certain threshold to meet data forgetting requirements. However, the effectiveness of this technique is critically dependent on the chosen correlation threshold. If the threshold is too large, fewer channels will be removed (less pruning), resulting in some channels related to the target data not being removed, leading to the risk of data retention (incomplete forgetting). If the threshold is too small, more irrelevant channels will be mistakenly removed, resulting in reduced model generalization. Furthermore, devices in the Industrial Internet vary significantly in their hardware and software environments, resulting in significant heterogeneity in their resources (such as computing, storage, and communication). For example, among NVIDIA's commercial series of industrial edge computing devices, the Jetson TX2 device has a computing power of 1.33 TOPS, while the newly released Jetson AGX Orin device, equipped with a 2048-core Ampere architecture GPU, has a peak computing power of 275 TOPS, approximately 207 times the computing power of the Jetson TX2. Industrial devices also vary in memory capacity (e.g., 8GB, 16GB, 64GB) and network connectivity (e.g., 4G, 5G, WiFi). Existing solutions typically assign the same pruning ratio to all devices. This results in significant latency on heterogeneous industrial devices, severely impacting the efficiency of data forgetting tasks. Therefore, a method is needed that can dynamically allocate pruning ratios based on device heterogeneity to minimize the time required to delete target data. Summary of the Invention

[0004] To solve the above technical problems, the present invention proposes a data forgetting method for dynamically allocating pruning ratios for heterogeneous industrial equipment resources. In the case of heterogeneous industrial equipment resources, the pruning ratio of each device is adaptively adjusted to minimize the completion time of the data forgetting task.

[0005] To achieve the above objectives, in a first aspect, the present invention provides a data forgetting method for dynamically allocating pruning ratios for heterogeneous resources of industrial equipment, comprising:

[0006] When the server detects a target device failure, it assigns an adaptive pruning ratio to each industrial device based on its resource information;

[0007] Each industrial device prunes the local model according to the assigned pruning ratio, removing parameters related to abnormal data of the target device, thereby achieving forgetting of the target data;

[0008] Other industrial devices except the target device update the pruned local model, obtain the updated local model and upload it to the server;

[0009] The updated local models are aggregated through the server to obtain a global model that guarantees generalization performance and forgets abnormal data of the target device.

[0010] Preferably, the step of the server detecting a target device failure includes:

[0011] Monitor the operating status of each industrial equipment through the server;

[0012] When it is detected that the target device has abnormal data or is out of the training cluster, it is determined that the target device has failed.

[0013] Preferably, the step of allocating an adaptive pruning ratio to each industrial device according to resource information of each industrial device by the server includes:

[0014] Obtain computing resource information and communication resource information of each industrial device through the server;

[0015] The server dynamically adjusts the pruning ratio of each device based on the resource information of each device based on the multi-armed bandit algorithm.

[0016] Preferably, the step of the server dynamically adjusting the pruning ratio of each device based on the multi-armed bandit algorithm includes:

[0017] Initialize the pruning ratio space of each industrial device to an interval;

[0018] In each round of training, the server dynamically divides the pruning ratio space according to the preset exploration granularity;

[0019] The server selects the optimal partition based on the high-confidence boundary value and randomly samples the pruning ratio within the selected partition.

[0020] Preferably, the step of the server selecting the optimal partition according to the high-confidence boundary value includes:

[0021] Calculate the utilization and exploration terms for each partition;

[0022] Calculate the high confidence boundary value of each partition based on the utilization term and exploration term;

[0023] The partition with the largest high-confidence boundary value is selected as the optimal partition.

[0024] Preferably, the step of each industrial device pruning the local model according to the assigned pruning ratio includes:

[0025] Each industrial device calculates the local model parameters and the word frequency-inverse text frequency index of the target device abnormal data;

[0026] Sorting local model parameters according to the word frequency-inverse document frequency index;

[0027] The sorted model parameters are truncated according to the assigned pruning ratio and set to zero or removed.

[0028] Preferably, the word frequency-inverse document frequency index is calculated as follows:

[0029] Calculate the word frequency index of the model parameters to indicate the local sensitivity of the model parameters to abnormal data of the target device;

[0030] Calculate the inverse frequency index of the model parameters, which indicates the universality of the model parameters on all device data;

[0031] The word frequency index is multiplied by the inverse text frequency index to obtain a word frequency-inverse text frequency index.

[0032] Preferably, the step of each industrial device updating the pruned local model includes:

[0033] Each industrial device updates the pruned local model by gradient descent based on local data;

[0034] The loss function value of the updated local model is calculated by each industrial device.

[0035] Preferably, the step of calculating the loss function value of the updated local model by each industrial device includes:

[0036] Each industrial device calculates the model's predicted output based on local data;

[0037] Each industrial device calculates the loss function value based on the predicted output and the true label.

[0038] Preferably, the step of aggregating the updated local model by the server includes:

[0039] Receive updated local models uploaded by each industrial device through the server;

[0040] The server performs weighted averaging on the parameters of each local model according to the federated averaging algorithm to obtain a new global model.

[0041] Compared with the prior art, the present invention has the following advantages and technical effects:

[0042] The present invention provides a data forgetting method for dynamically allocating pruning ratios for heterogeneous resources of industrial equipment, comprising: first, when a server detects a target device failure, the server allocates an adaptive pruning ratio to each industrial device based on resource information of each industrial device; second, each industrial device prunes a local model according to the allocated pruning ratio, removes parameters related to abnormal data of the target device, thereby forgetting the target data; then, other industrial equipment except the target device updates the pruned local model, obtains an updated local model and uploads it to the server; finally, the updated local model is aggregated by the server to obtain a global model that guarantees generalization performance and forgets abnormal data of the target device.

[0043] Compared with the existing method of allocating pruning ratios, the present invention innovatively uses model pruning in the data forgetting task caused by equipment failure to improve the execution efficiency of the forgetting task; compared with the existing fault detection method, the present invention innovatively uses model pruning to perform post-processing of fault detection, forgetting the abnormal data of the faulty equipment, rather than using it to detect whether the equipment failure has occurred.

[0044] In the industrial Internet scenario, when a participating device in the training system fails and abnormal data needs to be forgotten, the present invention can adaptively allocate appropriate pruning ratios to them based on the heterogeneous computing and communication resources of the devices, thereby reducing the waiting time of all devices, improving the resource utilization of the system, and enhancing the execution efficiency of the forgetting task. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The accompanying drawings, which constitute part of this application, are intended to provide a further understanding of this application. The exemplary embodiments and descriptions of this application are intended to explain this application and do not constitute an improper limitation on this application. In the accompanying drawings:

[0046] Figure 1 Schematic diagram of a model training system under the Industrial Internet according to an embodiment of the present invention. DETAILED DESCRIPTION

[0047] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0048] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0049] Example 1

[0050] like Figure 1 As shown, the model training system under the industrial Internet includes three important components: servers, access points and industrial equipment. The training system considered in the present invention includes a server, multiple access points and N industrial equipment. The server acts as a global coordinator, responsible for aggregating local model updates (such as gradients or model parameters) from industrial equipment through a federated averaging algorithm, and dynamically adjusting the pruning ratio of the equipment to adapt to the heterogeneity of industrial equipment. The access point serves as a network communication hub, focusing on physical layer connection and protocol conversion. For example, it converts industrial protocols such as Modbus and OPC UA supported by industrial equipment into general protocols such as HTTP / GRPC to achieve reliable data transmission between industrial equipment and servers. Industrial equipment acts as the core execution unit, and obtains production line data (such as temperature, humidity, energy consumption) in real time through accessories such as PLCs and sensors. It uses edge computing capabilities to perform data preprocessing (such as noise filtering and feature extraction), and then performs local model training on the processed data, and uploads model updates to the server through access points for aggregation.

[0051] First, the server sends the initial global model w to all industrial devices through the access point. When device i receives the global model from the server, it will use the local model w i Replaced with the global model, that is, w i = w. Then, device i uses the gradient descent method to calculate the local dataset D i On the local model w i To update:

[0052]

[0053] in represents the local model of device i in the tth training round, η represents the learning rate when the model is updated, and F i represents the loss function of device i, Represents the gradient. After the local update is completed, device i will update the local model Upload to the server. The server updates the global model by aggregating the received local models:

[0054]

[0055] where |D i |Represents dataset D i The number of samples included, It represents the sum of all the data contained in all devices. Industrial devices and servers perform local updates and global model aggregation alternately until the model converges.

[0056] This embodiment provides a data forgetting method for dynamically allocating pruning ratios for heterogeneous resources of industrial equipment, including:

[0057] S1. When the server detects a target device failure, it allocates an adaptive pruning ratio to each industrial device based on its resource information.

[0058] Furthermore, the step of the server detecting a target device failure includes:

[0059] Monitor the operating status of each industrial equipment through the server;

[0060] When it is detected that the target device has abnormal data or is out of the training cluster, it is determined that the target device has failed.

[0061] Furthermore, the server allocates an adaptive pruning ratio to each industrial device according to resource information of each industrial device, including:

[0062] Obtain computing resource information and communication resource information of each industrial device through the server;

[0063] The server dynamically adjusts the pruning ratio of each device based on the resource information of each device based on the multi-armed bandit algorithm.

[0064] This example innovatively proposes an adaptive model pruning ratio allocation method that dynamically adjusts the pruning ratio based on each device's computing resources (such as CPU / GPU computing power) and communication resources (such as network bandwidth). This method fully utilizes the resources of high-performance devices while avoiding task delays caused by insufficient resources on low-performance devices.

[0065] Furthermore, the server dynamically adjusts the pruning ratio of each device based on the multi-armed bandit algorithm, including the following steps:

[0066] Initialize the pruning ratio space of each industrial device to an interval;

[0067] In each round of training, the server dynamically divides the pruning ratio space according to the preset exploration granularity;

[0068] The server selects the optimal partition based on the high-confidence boundary value and randomly samples the pruning ratio within the selected partition.

[0069] Furthermore, the step of selecting the optimal partition according to the high-confidence boundary value by the server includes:

[0070] Calculate the utilization and exploration terms for each partition;

[0071] Calculate the high confidence boundary value of each partition based on the utilization term and exploration term;

[0072] The partition with the largest high-confidence boundary value is selected as the optimal partition.

[0073] Specifically, in the context of the Industrial Internet, device heterogeneity has a serious negative impact on the efficiency of data forgetting tasks. From a computing power perspective, differences in hardware across different devices lead to diverging processing speeds—high-performance devices can quickly prune and update model parameters, while low-performance devices, due to insufficient processor power or memory limitations, may experience exponentially increased local update times compared to high-performance devices. Furthermore, the heterogeneity of communication resources exacerbates this impact. High-bandwidth, low-latency fixed network devices can quickly upload pruned parameters, while mobile devices that rely on unstable wireless connections or legacy devices with outdated protocols may experience significantly increased transmission delays due to network congestion, signal attenuation, or data format conversion. Therefore, the heterogeneity of computing and communication resources across industrial devices has a serious impact on the efficiency of data forgetting tasks, significantly increasing the time it takes to complete direct forgetting tasks.

[0074] To mitigate the impact of device heterogeneity in data forgetting tasks, the present invention models the device adaptive model pruning problem to minimize the completion time of the forgetting task. Specifically, the objective function of this optimization problem can be expressed as:

[0075]

[0076] in Indicates the actual time required for the tth round of update, that is, the maximum completion time of all devices Based on the above modeling, the adaptive model pruning problem considering heterogeneous device resources can be expressed as:

[0077]

[0078] Constraints include:

[0079] 1) Pruning ratio feasibility constraint: in It represents the pruning ratio of device i in round t. 0 and 1 are the preset lower limit (to prevent model failure) and upper limit (to avoid resource waste) of the pruning ratio.

[0080] 2) Computing resource constraints: Ensure that the computing resource consumption of device i in round t does not exceed its capacity C i .

[0081] 3) Communication resource constraints: Ensure that the communication resource consumption of device i in round t does not exceed its capacity B i .

[0082] 4) Model performance constraints: The loss function of the pruned model is required to converge to the threshold ε after T rounds of training.

[0083] This embodiment innovatively models the resource heterogeneity of industrial devices and defines the adaptive pruning ratio allocation problem as an optimization problem. By considering the device's computing and communication resources, as well as model performance constraints, the present invention minimizes the completion time of the forgetting task while ensuring that model performance is not significantly affected. This optimization method can better adapt to the diversity of device resources in industrial Internet scenarios and improve the overall efficiency of the system.

[0084] Since the above problem is a variant of the knapsack problem and the decision variables (pruning ratios of each device) are coupled with each other, the present invention adopts an online learning algorithm based on a multi-armed bandit to assign appropriate pruning ratios to heterogeneous devices. Specifically, the dynamic space of the multi-armed bandit is the pruning ratios of all devices {α1,α2,···,α N}, its value range is the continuous interval (0,1) N In each round of training, the server selects one of the arms of the slot machine, calculates the corresponding reward, and determines the optimal pruning ratio for each device by minimizing the difference between the current reward and the expected reward. The reward function of the multi-armed slot machine is defined as follows:

[0085]

[0086] Represents the difference between the loss function of device i in the tth round and the loss function of the previous round (t-1 round), Represents the local update time of device i in round t, This measures the difference between the update time of device i in round t and the update time of other devices. It can be seen that this reward function achieves a balance between model performance and computational efficiency. In addition, to achieve a balance between utilization and exploration, the present invention defines the high confidence boundary (UCB) value as follows:

[0087] U j =(R j / Q j )+sqrt(2·log(∑Q j ) / Q j )

[0088] where R j represents the cumulative reward of the jth partition, Q j It represents the number of times the th partition is selected. The definition of UCB value includes the exploration term (R j / Q j ) and using the term sqrt(2·log(∑Q j ) / Q j ), by selecting the partition with the largest UCB value, giving priority to utilizing the high reward history interval, while exploring the interval that has not been fully tried, so as to achieve a balance between utilization and exploration.

[0089] However, the solution to the multi-armed bandit problem based on the Unified Combination (UCB) strategy only involves a discrete set of arm selections and cannot be directly applied to pruning ratio decisions in continuous space. Therefore, this paper extends the UCB strategy to continuous space, determining the partitioning of the continuous space through a decision tree, and ultimately assigning an appropriate pruning ratio to each device within a selected interval. The specific process is shown in Algorithm 1. Initially, the pruning ratio space (0, 1) for each device is divided into a single interval. As training rounds progress, the algorithm dynamically splits the interval according to a preset exploration granularity θ: if the length of the currently selected interval exceeds θ, it is evenly divided into finer subintervals, gradually improving decision accuracy. In each training round, the algorithm selects a partition based on the UCB strategy. The UCB value is composed of the historical average reward (utilization term) and the weight of the exploration term. The former reflects the partition's past performance, while the latter encourages exploration of under-explored intervals. After the partition is selected, pruning ratios are randomly sampled and training is performed. The reward function comprehensively considers the reduction in model loss and the degree to which the completion time of the device's local update deviates from the system average, thereby balancing model performance and computational efficiency. Reward feedback is weighted and accumulated using a discount factor λ, giving greater weight to recent performance and adapting to dynamic changes in device resources. Ultimately, without requiring prior knowledge of device capabilities, the algorithm adapts to the optimal pruning ratio in heterogeneous environments through partition refinement and statistical updates, significantly reducing synchronization wait times and improving the efficiency of forgetting tasks.

[0090] Algorithm 1: Pruning ratio adaptive decision algorithm based on multi-armed tiger

[0091] Input: device set [N], number of training rounds T, exploration granularity θ, discount factor λ∈(0,1)

[0092] Output: Pruning ratio of each device in each round

[0093] 1) Initialization:

[0094] For each device i∈[N]:

[0095] Initialize the decision tree partition P0 = {(0,1)} # Initially treat the pruning ratio space (0,1) as an interval

[0096] Initialize the statistics for each partition j:

[0097] Total historical rewards R j =0

[0098] Selection count Q j =0

[0099] 2) For each round of training t=0 to T-1:

[0100] For each device i∈[N]:

[0101] #Step 1: Select the optimal partition

[0102] Calculate the current partition set P t = UCB value of each partition j in {(0,1)}:

[0103] U j =(R j / Q j )+sqrt(2·log(∑Q j ) / Q j )#Utilization item + exploration item

[0104] Select the partition with the largest UCB value j*=argmax j (U j )

[0105] #Step 2: Randomly sample pruning ratios within the selected partition

[0106] If the diameter of partition j* is greater than θ:#The diameter is defined as the length of the interval

[0107] Divide j* into 2 subintervals evenly

[0108] Update partition set P t+1 =(P t+1 \{j*})∪{subinterval}

[0109] Randomly sample pruning ratios from j*

[0110] #Step 3: Perform training and calculate rewards

[0111] Device i Conduct local training and communications

[0112] Record completion time and model loss changes

[0113] Calculate rewards:

[0114] #Step 4: Update partition statistics

[0115] turn up The final partition j' (considering the dynamic segmentation of P t+1 )

[0116] Update j''s statistics:

[0117] #Weighted cumulative rewards

[0118] Q j' =Q j' +1

[0119] 3) Return all This embodiment uses an online learning algorithm based on a multi-armed bandit (MAB) to dynamically determine the pruning ratio for each device. Using a high confidence bound (UCB) strategy, the algorithm strikes a balance between leveraging known optimal pruning ratios and exploring new ones, ultimately finding the most optimal pruning ratio for each device in a continuous space. This approach adaptively responds to dynamic changes in device resources and optimizes the execution efficiency of forgetting tasks.

[0120] S2. Each industrial device prunes the local model according to the assigned pruning ratio, removing parameters related to abnormal data of the target device, thereby achieving forgetting of the target data;

[0121] Furthermore, the steps of pruning the local model of each industrial device according to the assigned pruning ratio include:

[0122] Each industrial device calculates the local model parameters and the word frequency-inverse text frequency index of the target device abnormal data;

[0123] Sorting local model parameters according to the word frequency-inverse document frequency index;

[0124] The sorted model parameters are truncated according to the assigned pruning ratio and set to zero or removed.

[0125] Furthermore, the word frequency-inverse document frequency index is calculated as follows:

[0126] Calculate the word frequency index of the model parameters to indicate the local sensitivity of the model parameters to abnormal data of the target device;

[0127] Calculate the inverse frequency index of the model parameters, which indicates the universality of the model parameters on all device data;

[0128] The word frequency index is multiplied by the inverse text frequency index to obtain a word frequency-inverse text frequency index.

[0129] Specifically, assuming that after multiple rounds of training, the global model trained by the server and N devices is used Indicates. When a device fails (such as Figure 1 The server needs to remove the contribution of the abnormal data of device j (denoted by u') from the global model. Specifically, the server first assigns a pruning ratio α to each device (except the target device j) based on the currently available resources. i , and then find the parameters related to the target data u' according to the pruning ratio to prune the global model, thereby removing the influence of the target data. However, the index that measures the correlation between model parameters and target data is crucial to the quality of the forgetting effect. The present invention uses word frequency-inverse text frequency technology as the correlation index to guide pruning. The calculation method of this index is as follows. Assuming that the global model There are L layers in total, and the model parameters of the l∈[L]th layer are Indicates that the parameter The output is recorded as A l , then the word frequency index can be expressed as:

[0130]

[0131] Where u' represents the data of target device j, V l Indicates the number of parameters contained in the l-th layer model.

[0132] From this, we can see that word frequency measures the local sensitivity of model parameters to the target data, specifically by calculating the ratio of the output of the current parameter on the target data to the output of all parameters. For example, if the output of a certain parameter on the target data dominates the output of the parameters of this layer, it indicates that the parameter has made a significant contribution to the fitting of the target data and may store characteristic information of specific data. Similarly, inverse document frequency can be expressed as:

[0133]

[0134] Where N represents the number of devices, and D represents the local datasets for all devices. As can be seen, inverse document frequency evaluates the universality of a parameter from a global perspective. By counting the sensitivity ratio of the parameter across all device data, if only a few devices significantly influence the parameter (high IDF value), it indicates strong specificity but weak universality. Finally, the term frequency-inverse document frequency metric is calculated as follows:

[0135]

[0136] In summary, the TF-IDF metric is the product of term frequency and inverse document frequency, comprehensively reflecting the "unique" contribution of a model parameter to the target data. A high TF-IDF indicates that the parameter is both sensitive to the target data and has little impact on other data. This characteristic makes TF-IDF an important metric for guiding model pruning in federated forgetting learning.

[0137] When using TF-IDF for model pruning, the parameter-data relevance score dictionary is first initialized, and the proportion of retained parameters is calculated based on the input pruning ratio. Then, each parameter in the model is traversed, and its term frequency (TF) and inverse document frequency (IDF) are calculated based on the target dataset. The product of the two is stored in the dictionary as the TF-IDF score. Then, all parameters are sorted from high to low by TF-IDF score, and a specified proportion of parameters with high rankings is truncated according to the pruning ratio (for example, when the pruning ratio is 0.2, the top 20% of high-scoring parameters are selected) and set to zero or removed. Finally, the pruned sub-model parameters are output.

[0138] This paper innovatively introduces the term frequency-inverse document frequency (TF-IDF) metric to measure the relevance of model parameters to the target data. The TF-IDF metric comprehensively considers both the local sensitivity (term frequency) and global universality (inverse document frequency) of model parameters to the target data, enabling more accurate identification of parameters that are "uniquely" related to the target data. In this way, the pruning process effectively removes the influence of the target data while avoiding the accidental deletion of parameters that contribute significantly to other data. This allows the model to retain its generalization capabilities to the greatest extent possible while forgetting the target data.

[0139] S3. Other industrial devices except the target device update the pruned local model, obtain the updated local model, and upload it to the server;

[0140] Furthermore, each industrial device updates the pruned local model in the following steps:

[0141] Each industrial device updates the pruned local model by gradient descent based on local data;

[0142] The loss function value of the updated local model is calculated by each industrial device.

[0143] Furthermore, the step of calculating the loss function value of the updated local model by each industrial device includes:

[0144] Each industrial device calculates the model's predicted output based on local data;

[0145] Each industrial device calculates the loss function value based on the predicted output and the true label.

[0146] Specifically, the generalization performance of the pruned model may be affected. To compensate for the generalization performance of the model lost due to pruning, each device i needs to use the gradient descent method to prune the model after the tth round. Update to restore the performance of the model. Considering the heterogeneous resources of industrial equipment, the time required for each device to complete a round of update is different. and They represent the computing resources (e.g., CPU or GPU cycles) and communication resources (e.g., network bandwidth) available to device i during the tth round of updates. In addition, considering model pruning, the computing resources consumed by model training under different pruning ratios are also different. Let s represent the computing resources required to update one model parameter. Then the computing resources required for device i to complete the tth round of updates are in It represents the model According to this definition, the pruning ratio The larger it is, the fewer model parameters are retained, and the smaller the computing resources required to complete the model update. Combined with the resources required for model update and the computing power of device i The computation time required for the tth round of local updates It can be expressed as:

[0147]

[0148] S4. Aggregate the updated local models through the server to obtain a global model that guarantees generalization performance and forgets abnormal data of the target device.

[0149] Furthermore, the step of aggregating the updated local model through the server includes:

[0150] Receive updated local models uploaded by each industrial device through the server;

[0151] The server performs weighted averaging on the parameters of each local model according to the federated averaging algorithm to obtain a new global model.

[0152] Specifically, after the local update is completed, device i uploads the updated local model to the server for global aggregation to obtain a new global model Therefore, in addition to the computation time, the completion time of the tth round of updates for each device also includes the time to upload the model Combined with the bandwidth resources available for device i during the tth round of update The communication time can be expressed as:

[0153]

[0154] Where k represents the number of bytes of a single model parameter at different precisions. For example, when the parameter is FP32 precision, a single model parameter occupies 4 bytes, so k = 4. In summary, the total time required for device i to complete the tth round of update is:

[0155]

[0156] Since the federated averaging algorithm used by the server aggregates local models synchronously, the server needs to wait until all devices upload their local models before performing global aggregation. Therefore, the actual time required for the tth round of updates is the maximum local update time of all devices, that is:

[0157]

[0158] Beneficial effects of this embodiment:

[0159] This embodiment addresses the problem of adaptive model pruning under heterogeneous resource constraints, considering the computational overhead of the model when data forgetting is required due to equipment failure in industrial Internet scenarios. Furthermore, the present invention employs an adaptive model pruning algorithm based on a multi-armed bandit to improve the efficiency of forgetting tasks.

[0160] In the industrial Internet scenario, when a participating device in the training system fails and abnormal data needs to be forgotten, the present invention can adaptively allocate appropriate pruning ratios to them based on the heterogeneous computing and communication resources of the devices, thereby reducing the waiting time of all devices, improving the resource utilization of the system, and enhancing the execution efficiency of the forgetting task.

[0161] The above are merely preferred embodiments of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A data forgetting method for dynamically allocating pruning ratios for heterogeneous resources of industrial equipment, characterized by: The following steps are involved: When the server detects a target device failure, it assigns an adaptive pruning ratio to each industrial device based on its resource information; Each industrial device prunes the local model according to the assigned pruning ratio, removing parameters related to abnormal data of the target device, thereby achieving forgetting of the target data; Other industrial devices except the target device update the pruned local model, obtain the updated local model and upload it to the server; The updated local models are aggregated through the server to obtain a global model that guarantees generalization performance and forgets abnormal data of the target device.

2. The method according to claim 1, characterized in that The steps for the server to detect a target device failure include: Monitor the operating status of each industrial equipment through the server; When it is detected that the target device has abnormal data or is out of the training cluster, it is determined that the target device has failed.

3. The method according to claim 1, characterized in that The server allocates an adaptive pruning ratio to each industrial device according to resource information of each industrial device, including the following steps: Obtain computing resource information and communication resource information of each industrial device through the server; The server dynamically adjusts the pruning ratio of each device based on the resource information of each device based on the multi-armed bandit algorithm.

4. The method according to claim 3, characterized in that The steps for the server to dynamically adjust the pruning ratio of each device based on the multi-armed bandit algorithm include: Initialize the pruning ratio space of each industrial device to an interval; In each round of training, the server dynamically divides the pruning ratio space according to the preset exploration granularity; The server selects the optimal partition based on the high-confidence boundary value and randomly samples the pruning ratio within the selected partition.

5. The method according to claim 4, characterized in that The steps for the server to select the optimal partition based on the high-confidence boundary value include: Calculate the utilization and exploration terms for each partition; Calculate the high confidence boundary value of each partition based on the utilization term and exploration term; The partition with the largest high-confidence boundary value is selected as the optimal partition.

6. The method according to claim 1, characterized in that The steps for each industrial device to prune the local model according to the assigned pruning ratio include: Each industrial device calculates the local model parameters and the word frequency-inverse text frequency index of the target device abnormal data; Sorting local model parameters according to the word frequency-inverse document frequency index; The sorted model parameters are truncated according to the assigned pruning ratio and set to zero or removed.

7. The method according to claim 6, characterized in that The word frequency-inverse document frequency index is calculated as follows: Calculate the word frequency index of the model parameters to indicate the local sensitivity of the model parameters to abnormal data of the target device; Calculate the inverse frequency index of the model parameters, which indicates the universality of the model parameters on all device data; The word frequency index is multiplied by the inverse text frequency index to obtain a word frequency-inverse text frequency index.

8. The method according to claim 1, characterized in that The steps for each industrial device to update the pruned local model include: Each industrial device updates the pruned local model by gradient descent based on local data; The loss function value of the updated local model is calculated by each industrial device.

9. The method according to claim 8, characterized in that The steps of calculating the loss function value of the updated local model through each industrial device include: Each industrial device calculates the model's predicted output based on local data; Each industrial device calculates the loss function value based on the predicted output and the true label.

10. The method according to claim 1, characterized in that The steps to aggregate the updated local model through the server include: Receive updated local models uploaded by each industrial device through the server; The server performs weighted averaging on the parameters of each local model according to the federated averaging algorithm to obtain a new global model.

Images