Image data processing method and image processing apparatus

Abstract

The application discloses an image data processing method and an image processing device, and belongs to the technical field of image processing. The image data processing method comprises the following steps: receiving a global model sent by a server end; training the global model based on local training images and actual labels corresponding to the local training images, so as to obtain a new local model, and the new local model is used for outputting a predicted label corresponding to the local training image and a loss value corresponding to a target loss function being not greater than a loss threshold; the target loss function is a loss function with an added regularization term; and the new local model is sent to the server end, and the new local model is used for being aggregated by the server end to generate a new global model, and the new global model is used for being sent to a client end. The image processing method improves output accuracy, reduces the dependence of the client end on the defense mechanism of the server end, improves the defense effect, and thus improves the reliability and safety of the model.

Description

Technical Field

[0001] This application belongs to the field of image processing technology, and in particular relates to an image data processing method and an image processing apparatus. Background Technology

[0002] During federated learning model training, the client has complete control over local data and the local training process, making federated learning vulnerable to attacks launched by malicious clients. Related technologies primarily defend against attacks by detecting and rejecting malicious weights and providing model robustness certification within a limited attack scale. However, these methods are not very effective and negatively impact model performance. Summary of the Invention

[0003] This application aims to address at least one of the technical problems existing in the related art. To this end, this application proposes an image data processing method and an image processing apparatus that reduce the computational complexity of updating the local model, maintain a relatively stable output of the new local model, improve output accuracy, reduce the client's dependence on the server-side defense mechanism, improve the defense effect, and thus improve the reliability and security of the model.

[0004] In a first aspect, this application provides an image data processing method applied to a client, wherein the client is connected to a server; the method includes:

[0005] Receive the global model sent by the server;

[0006] Based on the local training images and the actual labels corresponding to the local training images, the global model is trained with the goal of outputting the predicted labels corresponding to the local training images and the loss value corresponding to the target loss function not being greater than the loss threshold, to obtain a new local model; the target loss function is a loss function with regularization added.

[0007] The new local model is sent to the server, where it is aggregated to generate a new global model, which is then sent to the client.

[0008] According to the image data processing method of this application, by receiving the global model sent by the server, the client trains the global model using local training images and the corresponding actual labels. During the training of the global model, the training process is optimized by adding a regularization term to the target loss function, thereby reducing the computational complexity of updating the local model and ensuring that the new local model maintains a relatively stable output, improving output accuracy, reducing the client's dependence on the server-side defense mechanism, improving the defense effect, and thus improving the reliability and security of the model.

[0009] According to the image data processing method of this application, the step of training the global model based on the local training image and the actual label corresponding to the local training image, with the objective of outputting the predicted label corresponding to the local training image and the loss value corresponding to the target loss function not exceeding a loss threshold, to obtain a new local model includes:

[0010] Based on the actual labels and the predicted labels output by the global model, the target loss function is processed to obtain the loss value;

[0011] Based on the identity matrix, the learning rate corresponding to the global model, and the estimated value of the BFGS matrix, calculate the first gradient of the model parameters corresponding to the global model.

[0012] The global model is updated based on the first gradient, the learning rate, and the hyperparameters corresponding to the regularization term. The process of processing the target loss function based on the actual label and the predicted label output by the global model to obtain the loss value continues until the iteration stop condition is met, resulting in the new local model.

[0013] According to the image data processing method of this application, updating the global model based on the first gradient, the learning rate, and the hyperparameters corresponding to the regularization term includes:

[0014] If the first gradient is not greater than the gradient threshold, the global model is updated based on the first gradient, the learning rate, and the hyperparameters corresponding to the regularization term.

[0015] If the first gradient is greater than the gradient threshold, the global model is updated based on the gradient threshold, the learning rate, and the hyperparameters corresponding to the regularization term.

[0016] According to the image data processing method of this application, the step of training the global model based on the local training image and the actual label corresponding to the local training image, with the objective of outputting the predicted label corresponding to the local training image and the loss value corresponding to the target loss function not exceeding a loss threshold, to obtain a new local model includes:

[0017] Image enhancement is performed on the local training images to obtain enhanced training images;

[0018] Based on the local training image, the enhanced training image, and the actual label corresponding to the local training image, with the goal of outputting the predicted label corresponding to the local training image and the loss value corresponding to the target loss function not being greater than the loss threshold, the global model is trained to obtain a new local model.

[0019] According to the image data processing method of this application, the local training images include at least one of the following: images corresponding to different storage objects in the refrigeration equipment, medical images, and industrial scene images.

[0020] Secondly, this application provides an image data processing apparatus applied to a client, wherein the client is connected to a server; the apparatus includes:

[0021] The first processing module is used to receive the global model sent by the server.

[0022] The second processing module is used to train the global model based on the local training image and the actual label corresponding to the local training image, with the goal of outputting the predicted label corresponding to the local training image and the loss value corresponding to the target loss function not being greater than the loss threshold, to obtain a new local model; the target loss function is a loss function with regularization added.

[0023] The third processing module is used to send the new local model to the server, whereby the server aggregates the new local model to generate a new global model, which is then sent to the client.

[0024] According to the image data processing apparatus of this application, by receiving a global model sent by the server, the client trains the global model using local training images and the corresponding actual labels. During the training of the global model, the training process is optimized by adding a regularization term to the target loss function, thereby reducing the computational complexity of updating the local model and ensuring that the new local model maintains a relatively stable output, improving output accuracy, reducing the client's dependence on the server-side defense mechanism, improving the defense effect, and thus improving the reliability and security of the model.

[0025] Thirdly, this application provides an image data processing method applied on a server side, wherein the server side is connected to at least one client side; the method includes:

[0026] Receive the local model sent by the at least one client;

[0027] The local models described are aggregated to obtain a new global model;

[0028] The new global model is sent to each of the clients, and the new global model is used by the clients to train on the local training images to generate new local models.

[0029] According to the image data processing method of this application, a more comprehensive and unified aggregated global model is obtained by aggregating the received local models, and noise is further added to the aggregated global model to reduce the model's sensitivity to small perturbations and improve the model's generalization ability.

[0030] According to the image data processing apparatus of this application, the step of aggregating each of the local models to obtain a new global model includes:

[0031] By aggregating the local models described above, we obtain the aggregated global model.

[0032] Noise is added to the model parameters corresponding to the aggregated global model to obtain the new global model.

[0033] According to the image data processing apparatus of this application, the step of adding noise to the model parameters corresponding to the aggregated global model to obtain the new global model includes:

[0034] Random noise is added to the model parameters corresponding to the aggregated global model to obtain random noise model parameters;

[0035] Based on the parameters of the random noise model, a smooth classifier is constructed;

[0036] The new global model is obtained based on the smooth classifier.

[0037] According to the image data processing apparatus of this application, obtaining the new global model based on the smoothing classifier includes:

[0038] Select test images from at least one of the acquired synthetic test images and publicly available test images;

[0039] Gaussian noise is added to the model parameters corresponding to the aggregated global model to obtain Gaussian noise model parameters;

[0040] Based on the Gaussian noise model parameters and the test image, the smooth classifier is run to obtain the classification categories output by the smooth classifier;

[0041] Based on the predicted probabilities of the classification categories corresponding to the target number, the smooth classifier is constrained to obtain the new global model.

[0042] Fourthly, this application provides an image processing apparatus applied on a server side, the server side being connected to at least one client; the apparatus includes:

[0043] The fourth processing module is used to receive the local model sent by the at least one client;

[0044] The fifth processing module is used to aggregate the local models to obtain a new global model;

[0045] The sixth processing module is used to send the new global model to each of the clients, the new global model being used by the clients to train based on the local training images to generate new local models.

[0046] According to the image data processing apparatus of this application, a more comprehensive and unified aggregated global model is obtained by aggregating the received local models, and noise is further added to the aggregated global model to reduce the model's sensitivity to small perturbations and improve the model's generalization ability.

[0047] Fifthly, this application provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the image data processing method as described in the first and third aspects above.

[0048] In a sixth aspect, this application provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the image data processing method as described in the first and third aspects above.

[0049] In a seventh aspect, this application provides a computer program product, including a computer program that, when executed by a processor, implements the image data processing method as described in the first and third aspects above.

[0050] The above-described one or more technical solutions in the embodiments of this application have at least one of the following technical effects:

[0051] By receiving the global model sent by the server, the client trains the global model using local training images and their corresponding labels. During the training process, the target loss function with a regularization term is added to optimize the training process, reduce the computational complexity of updating the local model, and ensure that the new local model maintains a relatively stable output, thereby improving output accuracy, reducing the client's dependence on the server-side defense mechanism, improving the defense effect, and thus enhancing the reliability and security of the model.

[0052] Furthermore, by applying the objective loss function, constraints are imposed on the model parameters of the new local model to adjust the update direction and update step size of the model parameters during model training. Through regularization terms and the first gradient, the robustness of the new local model is improved, and the computational cost and complexity are reduced. In addition, the BFGS matrix estimate allows the defense mechanism to adapt to the data distribution and model update patterns of different clients, improving the generalization ability of the defense mechanism and reducing overfitting.

[0053] Furthermore, by comparing the magnitude of the first gradient and the gradient threshold, if the first gradient is greater than the gradient threshold, the first gradient is replaced with the gradient threshold. By pruning the first gradient, the search step size for updating model parameters is limited, alleviating the problem of global model divergence caused by a large search step size during optimization. This reduces the impact of malicious clients on the global model through extreme updates, ensuring that the training of the global model remains stable and eventually converges. This enables the newly trained local model to respond to and defend against attacks in real time, improving the robustness of the new local model against backdoor attacks.

[0054] Furthermore, by adding random noise to the model parameters corresponding to the aggregated global model, random noise model parameters are obtained. Based on the random noise model parameters, a smooth classifier is constructed. While maintaining the model performance, the robustness of the model is increased by introducing noise, reducing its sensitivity to small changes in the input image. Thus, based on the smooth classifier, a new global model is obtained, improving the robustness of the new global model to input perturbations and reducing its sensitivity to potential backdoor attacks, thereby reducing the possibility of successful backdoor attacks.

[0055] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0056] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0057] Figure 1 This is one of the schematic flowcharts of the image data processing method provided in the embodiments of this application;

[0058] Figure 2 This is a second schematic flowchart of the image data processing method provided in the embodiments of this application;

[0059] Figure 3 This is the third schematic flowchart of the image data processing method provided in the embodiments of this application;

[0060] Figure 4 This is one of the structural schematic diagrams of the image data processing apparatus provided in the embodiments of this application;

[0061] Figure 5 This is the fourth flowchart illustrating the image data processing method provided in the embodiments of this application;

[0062] Figure 6 This is a second schematic diagram of the structure of the image data processing device provided in the embodiments of this application;

[0063] Figure 7 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0064] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0065] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0066] The image data processing method, image data processing device, electronic device, and readable storage medium provided in this application will be described in detail below with reference to the accompanying drawings and through specific embodiments and application scenarios.

[0067] The image data processing method can be applied to the terminal, and can be executed by the hardware or software in the terminal.

[0068] The terminal includes, but is not limited to, portable communication devices such as mobile phones or tablets. It should also be understood that, in some embodiments, the terminal may not be a portable communication device, but rather a desktop computer.

[0069] The following embodiments describe a terminal including a display and a touch-sensitive surface. However, it should be understood that the terminal may include one or more other physical user interface devices such as a physical keyboard, mouse, and joystick.

[0070] The image data processing method provided in this application embodiment can be executed by an electronic device or a functional module or entity in an electronic device that can implement the image data processing method. The electronic devices mentioned in this application embodiment include, but are not limited to, mobile phones, tablets, computers, cameras, and wearable devices. The image data processing method provided in this application embodiment will be described below using an electronic device as the execution subject as an example.

[0071] like Figure 1 As shown, the image data processing method is applied to the client, and the method includes steps 110, 120 and 130.

[0072] The client connects to the server.

[0073] Step 110: Receive the global model sent by the server;

[0074] In this step, the global model is either the global model initialized on the server side, or the model obtained by aggregating the local models trained by each client on the server side.

[0075] It should be noted that the global model can be continuously updated during the training process.

[0076] In actual execution, the server can first initialize a training task and then distribute the global model to multiple clients.

[0077] The global model can be a convolutional neural network model, a deep learning model, or a machine learning model, such as a residual network (ResNet), a visual geometry group (VGG), a support vector machine (SVM), or a random forest.

[0078] In actual execution, when the client receives the global model sent by the server for the first time, the received global model is the initialized global model. The structure and parameters of this initialized global model serve as the starting point for one or more clients to perform local training.

[0079] If the client is not receiving the global model from the server for the first time, the received global model is a global model obtained by the server based on the local models trained by each client.

[0080] like Figure 2 As shown, the server initializes the training task and sends the global model to one or more clients, and the clients receive the global model sent by the server.

[0081] Step 120: Based on the local training images and the actual labels corresponding to the local training images, with the goal of ensuring that the predicted labels corresponding to the local training images and the loss values ​​corresponding to the target loss function are not greater than the loss threshold, train the global model to obtain a new local model.

[0082] In this step, the local training images are images of the client's local domain.

[0083] Local training images can include multiple training images.

[0084] It should be noted that different clients correspond to different application scenarios, and the types of local training images collected by the clients are also different.

[0085] In some embodiments, the local training images may include at least one of the following: images corresponding to different storage objects within the refrigeration device, medical images, and industrial scene images.

[0086] In this embodiment, the storage object is an object placed inside the refrigeration device.

[0087] The stored objects can include: vegetables, fruits, meats, and skin care products, etc.

[0088] Medical imaging can be images of lesions acquired during the diagnostic process.

[0089] Medical imaging can include X-rays, computed tomography (CT), and magnetic resonance imaging (MRI).

[0090] The industrial scene images are images collected in different industrial scenarios.

[0091] Industrial scenarios can include food production, parts production, and circuit board production.

[0092] It should be noted that the products on the production line may differ in different industrial settings.

[0093] The client can capture images of products on the production line in different industrial scenarios.

[0094] In actual execution, each client can collect images of relevant application scenarios based on its required application scenarios, and train a global model based on the collected images.

[0095] According to the image data processing method provided in the embodiments of this application, by collecting local training images of different application scenarios, such as images corresponding to different storage objects in refrigeration equipment, medical images, and industrial scene images, the diversity of local training images is enriched, the generalization of subsequent global model training is improved, and the risk of overfitting is reduced.

[0096] After acquiring local training images, the client can perform data preprocessing such as cropping, data cleaning, noise reduction, standardization, normalization, and annotation on the collected local training images, and then use the preprocessed local training images for training the global model.

[0097] It should be noted that by preprocessing the local training images, the global model can be trained on high-quality data, thus mitigating the problem of performance degradation caused by low-quality data.

[0098] The specific category of the local training images can be determined based on images acquired by the client or based on user-defined categories; this application does not impose any restrictions.

[0099] The actual label is the actual category corresponding to the locally trained image.

[0100] The predicted label is the category of the locally trained image predicted by the global model.

[0101] In actual execution, by inputting the local training image into the global model, the predicted label corresponding to the local training image can be obtained from the output of the global model.

[0102] The target loss function is the loss function with regularization added.

[0103] The objective loss function is used to calculate the difference between the predicted label and the actual label.

[0104] The regularization term in the target loss function can remove features with low importance, allowing the model to focus more on features that truly affect the classification results, rather than being misled by triggering patterns designed by attackers, thus reducing the risk of being interfered with by irrelevant or redundant features.

[0105] The specific type of the target loss function can be determined based on the actual situation. For example, the target loss function can be a regularization term added to the mean squared error loss function or a regularization term added to the cross-entropy loss function, etc. This application does not limit it.

[0106] The regularization term restricts the numerical value of the model parameters corresponding to the global model, thereby reducing the complexity of the global model.

[0107] Regularization terms include, but are not limited to: L1 regularization and L2 regularization.

[0108] The loss threshold is a preset value used to judge the training effect during the global model training process.

[0109] The loss threshold can be the minimum value of the loss function during training, or it can be user-defined or determined based on the actual situation; this application does not impose any restrictions.

[0110] The new local model is obtained by continuously training the global model based on the client's local training images and the corresponding actual labels of the local training images.

[0111] like Figure 3As shown, in the actual execution process, after obtaining the global model, the client can input the local training images and the corresponding actual labels into the global model, so that the global model can learn the features of the local training images and the corresponding actual labels, so that the global model can output predicted labels with higher accuracy.

[0112] During training, the training process can be optimized by using algorithms such as gradient descent and stochastic gradient descent.

[0113] It should be noted that the objective loss function can affect the search path and search step size of the optimization algorithm, thereby affecting the convergence speed of global model training and the accuracy of the output results.

[0114] Step 130: Send the new local model to the server. The new local model is used by the server to aggregate and generate a new global model, which is then sent to the client.

[0115] In this step, the new global model is the model obtained by aggregating the new local models sent by multiple clients received by the server.

[0116] In actual execution, if the loss value of the new local model is not less than the loss threshold after training the global model based on the local training images and the corresponding actual labels, the new local model can be sent to the server.

[0117] If the number of iterations of the new local model trained on the local training images and their corresponding actual labels exceeds the iteration threshold, the new local model can also be sent to the server.

[0118] During the research and development process, the inventors discovered that related technologies primarily defend against attacks by detecting and rejecting malicious weights and providing model robustness certification under limited attack scale. However, these methods are not very effective and negatively impact model performance.

[0119] This application trains a global model by receiving a global model sent from a server, based on local training images and their corresponding labels. During training, a regularization term is added to the target loss function to penalize the model parameters in the global model and encourage the selection of smaller parameter values. This limits the growth of model parameters, reduces the risk of overfitting, and improves the model's generalization ability. In other words, the local model can maintain a relatively stable output when faced with different images, making it less susceptible to being misclassified by attackers. This improves the defense effect while reducing the computational complexity of updating the global model.

[0120] According to the image data processing method provided in the embodiments of this application, by receiving a global model sent by the server, the client trains the global model using local training images and the actual labels corresponding to the local training images. During the training of the global model, the training process is optimized by adding a regularization term to the target loss function, thereby reducing the computational complexity of updating the local model and ensuring that the new local model maintains a relatively stable output, improving output accuracy, reducing the client's dependence on the server-side defense mechanism, improving the defense effect, and thus improving the reliability and security of the model.

[0121] Continue to refer to Figure 2 In some embodiments, step 120 may further include:

[0122] Based on the actual labels and the predicted labels output by the global model, the target loss function is processed to obtain the loss value;

[0123] Based on the identity matrix, the learning rate corresponding to the global model, and the estimated value of the BFGS matrix, calculate the first gradient of the model parameters corresponding to the global model.

[0124] The global model is updated based on the hyperparameters corresponding to the first gradient, learning rate, and regularization term. Then, the steps of processing the target loss function and obtaining the loss value based on the actual label and the predicted label output by the global model are returned until the stopping iteration condition is met, resulting in a new local model.

[0125] In this embodiment, the identity matrix is ​​a matrix in which the diagonal elements are 1 and the rest are 0.

[0126] The learning rate is the distance the model parameters move along the negative gradient direction when the model parameters corresponding to the global model are updated.

[0127] The learning rate can be user-defined or determined based on actual circumstances; this application does not impose any restrictions.

[0128] The BFGS matrix estimate is an approximate estimate of the Hessian matrix or its inverse matrix in the BFGS (Broyden–Fletcher–Goldfarb–Shanno) algorithm.

[0129] The BFGS matrix estimate can be determined based on the following formula:

[0130]

[0131] in, Let be the change in model parameters of client k in the (i+1)th local iteration of global round t; Let η be the change in model parameters of client k in the i-th local iteration of global round t; tThis represents the learning rate for the global model.

[0132] It should be noted that the regularization term added to the loss function can be used to adjust the BFGS matrix so that the diagonal elements of the BFGS matrix are close to 1 (the diagonal elements of the identity matrix), thereby optimizing the BFGS matrix.

[0133] BFGS matrix tending to zero refers to adjusting the regularization term so that the diagonal elements of the BFGS matrix are close to 1 rather than zero, in order to improve the robustness of the new local model.

[0134] The BFGS matrix estimate is obtained by approximating the change in the first gradient during the client's iterative process.

[0135] By using the first gradient variation to estimate the BFGS matrix, the computational cost and complexity can be significantly reduced compared to the calculation of scenic areas.

[0136] In actual implementation, the following formula can be used:

[0137]

[0138] Estimate the diagonal elements of the BFGS matrix.

[0139] in, Let be the estimated value of the BFGS matrix in the i-th row and j-th column during the t-th iteration; Let θ be the model parameter in the t-th iteration. i The first gradient.

[0140] The first gradient is the direction in which the directional derivative at the loss value reaches its maximum value, and the maximum value of the reciprocal of that direction.

[0141] In actual execution, the following functional relationship can be used as a basis:

[0142]

[0143] Calculate the first gradient.

[0144] in, η is the gradient operator; I is the identity matrix; η is the gradient operator. t This represents the learning rate for the global model. This is the estimated BFGS matrix for client k in global iteration t and local iteration i.

[0145] In some embodiments, the target loss function can be determined based on the following formula:

[0146] Loss=L(M(x′ s→t ),y t )+α·||m||

[0147] Where M is the global model, L is the cross-entropy loss; x′ s→t For the local training image with actual label s and predicted label t; y t To predict the actual label corresponding to the label, α is a hyperparameter used to balance the weight of the trigger size ||m|| in the loss function; ||m|| represents the L1 norm of the trigger mask m, which is the sum of the absolute values ​​of all elements in the mask, used to quantize the size of the trigger.

[0148] The stopping iteration condition is the condition under which the global model stops training.

[0149] Stopping iteration conditions may include: the loss value is not greater than the loss threshold and the number of iterations is greater than the iteration threshold.

[0150] In actual implementation, the target loss function can be used to guide the optimization process of the BFGS algorithm so that the new local model can converge.

[0151] During the first iteration of the global model, the local training images can be processed based on the model parameters of the global model to obtain the predicted labels of the local training images. Based on the predicted labels and the actual labels, the above formula is processed to obtain the loss value.

[0152] Then, the first gradient of the loss function with respect to the model parameters is calculated. Based on the calculated first gradient and the initial BFGS matrix estimate, the direction and magnitude of the model parameter update in the second iteration are determined. Based on the direction and magnitude of the model parameter update, the new model parameters are determined, and a new local model is obtained.

[0153] The second iteration is performed based on the new local model. The first gradient of the loss function of the second iteration with respect to the model parameters is calculated. Based on the first gradient of the new model parameters, the estimated value of the BFGS matrix is ​​updated. Based on the updated estimated value of the BFGS matrix and the first gradient, a new search direction and search step size are determined. This process is repeated until the loss value corresponding to the target loss function is not greater than the loss threshold, or the number of iterations of the new local model is greater than the number of iterations threshold (i.e., the iteration stop condition is met). The latest new local model is then sent to the server.

[0154] According to the image data processing method provided in the embodiments of this application, the model parameters of the new local model are constrained by the target loss function, so as to adjust the update direction and update step size of the model parameters during the model training process. The robustness of the new local model is improved by regularization term and first gradient, and the computational cost and computational complexity are reduced. In addition, the BFGS matrix estimation value allows the defense mechanism to adapt to the data distribution and model update mode of different clients, improves the generalization ability of the defense mechanism, and reduces overfitting.

[0155] like Figure 3 As shown, in some embodiments, updating the global model based on the first gradient, learning rate, and hyperparameters corresponding to the regularization term may further include:

[0156] If the first gradient is not greater than the gradient threshold, update the global model based on the first gradient, the learning rate, and the hyperparameters corresponding to the regularization term.

[0157] If the first gradient is greater than the gradient threshold, the global model is updated based on the gradient threshold, learning rate, and the hyperparameters corresponding to the regularization term.

[0158] In this implementation, the gradient threshold is a preset value used to clip the first gradient.

[0159] The specific value of the gradient threshold can be user-defined or determined based on the actual situation; this application does not impose any restrictions.

[0160] It should be noted that, with proper design of gradient threshold and regularization terms, the impact of the training process on the performance of the main task can be minimized, while maintaining the accuracy of the global model training on the main task.

[0161] In actual implementation, the first gradient can be clipped based on the following formula:

[0162]

[0163] Where r represents the number of rows in the matrix; c represents the number of columns in the matrix; η is the gradient operator; I is the identity matrix; η is the gradient operator. t,i Let be the learning rate corresponding to the (i+1)th local iteration of the global round t; q is the estimated BFGS matrix value of client k in global iteration t and local iteration i; q is the gradient threshold.

[0164] In actual implementation, the calculated first gradient can be clipped using the above formula.

[0165] If the calculated first gradient is not greater than the gradient threshold, the global model is updated based on the calculated first gradient, learning rate, and hyperparameters corresponding to the regularization term.

[0166] If the calculated first gradient is greater than the gradient threshold, the first gradient is replaced with the gradient threshold, and the global model is updated using the gradient threshold, learning rate, and hyperparameters corresponding to the regularization term.

[0167] According to the image data processing method provided in the embodiments of this application, by comparing the magnitude relationship between the first gradient and the gradient threshold, if the first gradient is greater than the gradient threshold, the first gradient is replaced with the gradient threshold. By pruning the first gradient, the search step size of the model parameter update is limited, which alleviates the problem of global model divergence caused by a large search step size during the optimization process. This reduces the impact of malicious clients on the global model through extreme updates, ensuring that the training of the global model can still remain stable and eventually converge, so that the newly trained local model can respond to and defend against attacks in real time, and improve the robustness of the new local model against backdoor attacks.

[0168] In some embodiments, step 120 may further include:

[0169] Image augmentation is performed on the local training images to obtain augmented training images;

[0170] Based on the local training image, the augmented training image, and the actual label corresponding to the local training image, a global model is trained with the goal of ensuring that the predicted label corresponding to the local training image and the loss value corresponding to the target loss function are not greater than the loss threshold, thus obtaining a new local model.

[0171] In this embodiment, image enhancement may include operations such as rotation, flipping, and color adjustment.

[0172] Enhanced training images are images obtained by enhancing local training images.

[0173] In practice, local training images can be processed using image enhancement techniques to obtain enhanced training images.

[0174] For example, rotating a cat image by 90 degrees, flipping it horizontally, or adding Gaussian noise can generate enhanced training images.

[0175] For example, an enhanced training image can be obtained by processing an image of a cat using a generative adversarial network.

[0176] In actual execution, the client can apply trigger inversion technology to restore the trigger and stick it on a clean image (without changing the actual image label) to generate enhanced training images.

[0177] During the trigger inversion process, the difficulty of generating the trigger can be quantified by the inter-class distance, that is, by quantifying the difficulty of generating the trigger by the distance between the actual label and the predicted label.

[0178] The distance between the actual label and the predicted label can be measured by calculating the size of the trigger (usually the L1 norm).

[0179] The process of applying triggers to local training images to generate augmented training images for predicted categories can be determined based on the following formula:

[0180] x′ s→t = (1-m)·x s +m·δ

[0181] Where, x s denoted as 's', representing the local training image of the actual label 's'; 'm' represents the trigger mask, a binary matrix used to determine the location on the local training image where the trigger is applied; 'δ' represents the trigger pattern, a vector with the same dimensions as the local training image, indicating the value added to the local training image at the location specified by mask 'm'; 'x'' s→t The image used for boosting training after the trigger is used to simulate an image with a backdoor, i.e., an image that should be misclassified as the predicted label t during prediction.

[0182] In actual execution, after obtaining the local training image, the augmented training image, and the actual label corresponding to the local training image, adversarial training, regularized training, transfer learning, multi-task learning, or reinforcement learning are performed based on the local training image, the augmented training image, and the actual label corresponding to the local training image. The goal is to train the global model so that the predicted label corresponding to the local training image and the loss value corresponding to the target loss function are not greater than the loss threshold, thus obtaining a new local model.

[0183] By using transfer learning training methods, a global model is applied to the training of local training images and augmented training images. By fine-tuning the global model, it can be quickly adapted to specific tasks, while the robustness of the model can be improved by utilizing augmented data.

[0184] By treating backdoor attack defense as one of the tasks, and simultaneously training other related tasks (such as classification and detection), the global model can learn more comprehensive features across multiple tasks through a multi-task learning training method, thereby improving overall performance and robustness.

[0185] Treating backdoor attack defense as a dynamic optimization problem, the global model can learn the optimal defense strategy through reinforcement learning training methods. For example, the global model can gradually learn how to identify and defend against backdoor attacks by adversarial training against attackers.

[0186] According to the image data processing method provided in the embodiments of this application, image enhancement is performed on local training images to obtain enhanced training images, thereby increasing the diversity of training images. Based on the local training images, enhanced training images, and the actual labels corresponding to the local training images, a global model is trained, enabling the global model to learn more diverse features and patterns, thus performing better when facing new and unseen data, and improving the model's generalization ability and robustness.

[0187] The image data processing method provided in this application can be executed by an image data processing device. This application uses an image data processing device executing the image data processing method as an example to illustrate the image data processing device provided in this application.

[0188] This application also provides an image data processing apparatus.

[0189] like Figure 4 As shown, the image data processing device includes: a first processing module 410, a second processing module 420, and a third processing module 430.

[0190] The first processing module 410 is used to receive the global model sent by the server.

[0191] The second processing module 420 is used to train a global model based on the local training image and the actual label corresponding to the local training image, with the goal of outputting the predicted label corresponding to the local training image and the loss value corresponding to the target loss function not being greater than the loss threshold, to obtain a new local model; the target loss function is a loss function with regularization added.

[0192] The third processing module 430 is used to send the new local model to the server. The new local model is used by the server to aggregate and generate a new global model, which is then sent to the client.

[0193] According to the image data processing apparatus provided in the embodiments of this application, by receiving a global model sent by the server, the client trains the global model using local training images and the actual labels corresponding to the local training images. During the training of the global model, the training process is optimized by adding a regularization term to the target loss function, thereby reducing the computational complexity of updating the local model and ensuring that the new local model maintains a relatively stable output, improving output accuracy, reducing the client's dependence on the server-side defense mechanism, improving the defense effect, and thus improving the reliability and security of the model.

[0194] In some embodiments, the second processing module 420 may also be used for:

[0195] Based on the actual labels and the predicted labels output by the global model, the target loss function is processed to obtain the loss value;

[0196] Based on the identity matrix, the learning rate corresponding to the global model, and the estimated value of the BFGS matrix, calculate the first gradient of the model parameters corresponding to the global model.

[0197] The global model is updated based on the hyperparameters corresponding to the first gradient, learning rate, and regularization term. Then, the steps of processing the target loss function and obtaining the loss value based on the actual label and the predicted label output by the global model are returned until the stopping iteration condition is met, resulting in a new local model.

[0198] In some embodiments, the second processing module 420 may also be used for:

[0199] If the first gradient is not greater than the gradient threshold, the global model is updated based on the first gradient, the learning rate, and the hyperparameters corresponding to the regularization term.

[0200] If the first gradient is greater than the gradient threshold, the global model is updated based on the gradient threshold, learning rate, and the hyperparameters corresponding to the regularization term.

[0201] In some embodiments, the second processing module 420 may also be used for:

[0202] Image augmentation is performed on the local training images to obtain augmented training images;

[0203] Based on the local training image, the augmented training image, and the actual label corresponding to the local training image, a global model is trained with the goal of ensuring that the predicted label corresponding to the local training image and the loss value corresponding to the target loss function are not greater than the loss threshold, thus obtaining a new local model.

[0204] This application also provides an image data processing method.

[0205] like Figure 5 As shown, the image data processing method is applied to the server side, and the method includes steps 510, 520 and 530.

[0206] The server connects to at least one client.

[0207] Step 510: Receive a local model sent by at least one client;

[0208] In this embodiment, the local model is a model trained by the client based on its local training set (i.e., local training images).

[0209] In actual execution, after each client trains the global model based on its local training set, it updates and sends the trained local model to the server, which then receives the local model sent by the client.

[0210] Step 520: Aggregate the local models to obtain a new global model;

[0211] In this step, the new global model is the model that the server processes the model parameters of one or more local models to obtain and send to the client.

[0212] In actual execution, the server can calculate the mean of the model parameters corresponding to multiple local models, and use the calculated mean as the model parameters of the global model to obtain the global model.

[0213] The server can also assign a weight to multiple clients, and then perform a weighted average of the model parameters based on the weights. The weighted average model parameters are then used as the model parameters of the global model to obtain the global model.

[0214] Of course, in actual execution, the server can also aggregate the local models in any other feasible way to obtain the global model, and this application does not limit this.

[0215] Step 530: Send the new global model to each client. The new global model is used by the clients to train based on the local training images to generate new local models.

[0216] In this step, after the server aggregates one or more received local models, it obtains a new global model. The new global model can be sent to one or more clients, allowing the clients to train new local models based on their local training images.

[0217] According to the image data processing method provided in the embodiments of this application, by receiving local models sent by one or more clients, the received one or more local models are aggregated to obtain a new global model, resulting in a more comprehensive and unified global model.

[0218] In some embodiments, the aggregation of local models to obtain a new global model may further include:

[0219] Aggregate the local models to obtain the aggregated global model;

[0220] Noise is added to the model parameters corresponding to the aggregated global model to obtain a new global model.

[0221] In this embodiment, the aggregated global model is a model obtained by aggregating the local models of one or more clients.

[0222] Compared with the aggregated global model, the new global model has lower perturbation sensitivity.

[0223] In actual execution, the server can add one or more Gaussian noise or random noise to the model parameters corresponding to the aggregated global model to obtain a new global model.

[0224] According to the image data processing method provided in the embodiments of this application, an aggregated global model is obtained by aggregating the received local models, and noise is further added to the aggregated global model to reduce the sensitivity of the aggregated global model to small perturbations and improve the generalization ability of the model.

[0225] In some embodiments, step 530 may further include:

[0226] Random noise is added to the model parameters corresponding to the aggregated global model to obtain random noise model parameters;

[0227] Construct a smooth classifier based on the parameters of the random noise model;

[0228] A new global model is obtained based on the smooth classifier.

[0229] In this embodiment, the random noise model parameters are model parameters that include random noise.

[0230] The smooth classifier is a classifier obtained by initializing or updating the original classifier in the aggregated global model based on the parameters of the random noise model.

[0231] In actual implementation, during the testing phase, smoothing operations can be performed on the model parameters, i.e., adding random noise or building a smooth classifier.

[0232] It should be noted that performing smoothing operations on model parameters introduces random noise, which can average out updates from different clients and reduce the impact of individual clients, especially malicious clients, on the new global model.

[0233] By limiting the range of model parameters and introducing smoothing operations, the decision boundaries of the global model can be made clearer, thus improving the interpretability of the model.

[0234] In actual implementation, a smooth classifier can be constructed based on the following formula:

[0235]

[0236] Among them, h s This is a smooth classifier obtained by adding random noise to the model parameters, where w represents the model parameters and x represents the model parameters. test For testing images; To find the maximum value, we need to find the probability that makes the probability... The largest category c, To test sample x under model parameters w test The probability of being classified as the predicted label c; Let μ(w) be the expectation under the probability distribution μ(w), where W is the random noise model parameter, h is the original classifier, and c represents the specific class label.

[0237] The formula above describes how to construct a robust smooth classifier using stochastic smoothing techniques. The smooth classifier predicts the test image x by adding random noise to the model parameters w and calculating the expectation. test The classification categories.

[0238] In actual execution, a smoothing operation can be performed on the server side after multiple rounds of global rounds, that is, after multiple aggregations of the local model.

[0239] For example, a parameter smoothing operation can be performed after every 5 or 10 global rounds to maintain the robustness of the model while reducing computational costs.

[0240] According to the image data processing method provided in the embodiments of this application, random noise is added to the model parameters corresponding to the aggregated global model to obtain random noise model parameters. Based on the random noise model parameters, a smooth classifier is constructed. While maintaining the model performance, the robustness of the model is increased by introducing noise, reducing its sensitivity to small changes in the input image. Thus, a new global model is obtained based on the smooth classifier, improving the robustness of the new global model to input perturbations and reducing the sensitivity of the new global model to potential backdoor attacks, thereby reducing the possibility of successful backdoor attacks.

[0241] In some embodiments, the new global model obtained based on the smooth classifier may further include:

[0242] Select test images from at least one of the acquired synthetic test images and publicly available test images;

[0243] Gaussian noise is added to the model parameters corresponding to the aggregated global model to obtain Gaussian noise model parameters;

[0244] Based on the Gaussian noise model parameters and the test image, a smooth classifier is run to obtain the classification categories output by the smooth classifier.

[0245] Based on the predicted probabilities of the classification categories corresponding to the number of targets, a smooth classifier is constrained to obtain a new global model.

[0246] In this embodiment, the synthesized test image is an image generated through computer simulation.

[0247] Publicly available test images are images released by research institutions and companies for use by researchers and developers.

[0248] The test images are selected from synthetic test images and publicly available test images, and are used to test the performance of the aggregated global model.

[0249] Gaussian noise is noise whose probability density must follow a normal distribution.

[0250] The Gaussian noise model parameters are model parameters that include Gaussian noise.

[0251] The classification category is a smooth classifier that classifies test images and outputs the predicted category.

[0252] The target number is a preset value for the number of desired categories selected from multiple categories.

[0253] The specific number of targets can be user-defined or determined based on actual circumstances. For example, the target number can be 2 or 3, and this application does not limit it.

[0254] In actual execution, the number of each category output by the smooth classifier can be counted and sorted. Based on the sorting results, a target number of categories can be selected. Based on the predicted probabilities corresponding to the target number of categories, the smooth classifier can be constrained to obtain a new global model.

[0255] In actual implementation, during server-side testing, M-order Gaussian noise can be added to the aggregated global model. Used to estimate M Monte Carlo samples, using the test image x test Parameters of each Gaussian noise model Run a smooth classifier for k∈mm and return the count vector of the classes classified by the smooth classifier. Select the two classes with the largest counts. and And calculate its corresponding prediction probability. and Finally, adjust the empirical estimate to constrain the probability that the smooth classifier returns the wrong label.

[0256] In some embodiments, the Hofding inequality can be used to calculate... and The upper bound of the constraint constrains the probability that the smooth classifier returns an incorrect label.

[0257] The predicted probability for category A can be determined based on the following formula:

[0258]

[0259] in,p A The predicted probability for category A; This is the probability estimate for class A; α is the error tolerance; and N is the total number of test samples.

[0260] The predicted probability for category B can be determined based on the following formula:

[0261]

[0262] in, The predicted probability for category B; This is the probability estimate for class B; α is the error tolerance; and N is the total number of test samples.

[0263] The probability estimate for category B can be based on the following formula:

[0264]

[0265] Calculated.

[0266] in, The probability estimate for class B; θ0 is the model parameter; σ 2 I is the covariance matrix of Gaussian noise; h θ (x) is the prediction of the input image x under the model parameters θ; b is the error category; As expected.

[0267] In actual implementation, estimation can be achieved through multiple sampling and voting processes. The value is set to be very small.

[0268] The specific formula is as follows:

[0269]

[0270] in, This is a probability estimate for the classification category B; ∈ is a very small value representing the maximum allowed probability of an incorrect label.

[0271] By adjusting the smoothing parameter σ and the number of samples N, it can be ensured that The value is very small, thus reducing the probability of incorrect labels.

[0272] Calculate using Hofding's inequality and The upper bound of the confidence level can control the probability of the aggregated global model returning incorrect labels within a very small range, which is suitable for application scenarios with strict requirements for robustness, such as autonomous driving and medical diagnosis.

[0273] Specifically, the Hofding inequality can be used to estimate the upper bound of the probability. By choosing appropriate α and N, it can be ensured that... The value is very small, thus reducing the probability of incorrect labels.

[0274] According to the image data processing method provided in the embodiments of this application, Gaussian noise model parameters are obtained by adding Gaussian noise to the model parameters. Based on the Gaussian noise model parameters and the test image, a smooth classifier is run. Based on the classification category output by the smooth classifier, the probability of the smooth classifier returning the wrong label is constrained to maintain stable prediction results and improve the reliability and security of the model in practical application scenarios.

[0275] In some embodiments, after step 520, the method may further include:

[0276] Based on the authentication radius, determine the degree of parameter perturbation corresponding to the new global model;

[0277] The degree of perturbation of output parameters.

[0278] In this embodiment, the certification radius is a quantitative measure of how much the model parameters are perturbed, and the prediction results of the new global model remain unchanged.

[0279] The parameter perturbation level is the parameter perturbation level that the new global model can withstand.

[0280] It should be noted that a larger certification radius indicates that the new global model is more robust to perturbations of model parameters.

[0281] The authentication radius can be calculated based on the following formula:

[0282]

[0283] Where RAD is the authentication radius; Let t be the time point when the backdoor attack occurs. sdv The sample variance at time t; R is the norm of the model parameters at time t; p represents the quantile of the standard normal distribution, used as the norm or perturbation of model parameters; i Update the weights of the model in the aggregation for the i-th client; γ i τ is the scaling factor for the i-th client, used to adjust the magnitude of model updates; i η is the number of local training iterations for the i-th client; i Let i be the noise level of the i-th client. Let be the backdoor attack strength of the i-th client; Let be the number of backdoor attack samples for the i-th client; T be the total number of training rounds; and Φ be the cumulative distribution function of the standard normal distribution, used to consider the pruning threshold ρ at time t.t Impact on model updates, σ t Let be the noise level at time t.

[0284] In actual implementation, the authentication radius can be calculated using the above formula. The authentication radius can be approximated as the parameter perturbation level corresponding to the new global model. The parameter perturbation level can be output so that users can understand the new global model's ability to withstand perturbations and improve the user experience.

[0285] It should be noted that by smoothing the model parameters, estimating the probability of mislabeling, using Hough's inequality to constrain the probability, and calculating the authentication radius, the server can effectively reduce the impact of malicious clients on the new global model, thereby improving the robustness and security of the model. The combined use of these steps can significantly improve the overall performance and security of the federated learning system.

[0286] According to the image data processing method provided in the embodiments of this application, by calculating the authentication radius, the degree of parameter perturbation corresponding to the new global model can be effectively quantified and represented, effectively verifying the robustness of defending against backdoor attacks in federated learning.

[0287] The image data processing method provided in this application can be executed by an image data processing device. This application uses an image data processing device executing the image data processing method as an example to illustrate the image data processing device provided in this application.

[0288] This application also provides an image data processing apparatus.

[0289] The server connects to one or more clients.

[0290] like Figure 6 As shown, the image data processing device is applied to the server side; the device includes: a fourth processing module 610, a fifth processing module 620 and a sixth processing module 630.

[0291] The fourth processing module 610 is used to receive a new local model sent by at least one client;

[0292] The fifth processing module 620 is used to aggregate the local models to obtain a new global model;

[0293] The sixth processing module 630 is used to send a new global model to each client. The new global model is used by the client to train based on the local training images to generate a new local model.

[0294] According to the image data processing apparatus provided in the embodiments of this application, by receiving local models sent by one or more clients, the received one or more local models are aggregated to obtain a new global model, resulting in a more comprehensive and unified global model.

[0295] In some embodiments, the fifth processing module 620 may also be used for:

[0296] Aggregate the local models to obtain the aggregated global model;

[0297] Noise is added to the model parameters corresponding to the aggregated global model to obtain a new global model.

[0298] In some embodiments, the fifth processing module 620 may also be used for:

[0299] Random noise is added to the model parameters corresponding to the aggregated global model to obtain random noise model parameters;

[0300] Construct a smooth classifier based on the parameters of the random noise model;

[0301] A new global model is obtained based on the smooth classifier.

[0302] In some embodiments, the fifth processing module 620 may also be used for:

[0303] Select test images from at least one of the acquired synthetic test images and publicly available test images;

[0304] Gaussian noise is added to the model parameters corresponding to the aggregated global model to obtain Gaussian noise model parameters;

[0305] Based on the Gaussian noise model parameters and the test image, a smooth classifier is run to obtain the classification categories output by the smooth classifier.

[0306] Based on the predicted probabilities of the classification categories corresponding to the number of targets, a smooth classifier is constrained to obtain a new global model.

[0307] The image data processing device in this application embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television set (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the device.

[0308] The image data processing device in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit the specific operating system used.

[0309] The image data processing device provided in this application embodiment can achieve... Figures 1 to 3 The various processes implemented in the method implementation examples will not be described again here to avoid repetition.

[0310] In some embodiments, such as Figure 7 As shown, this application embodiment also provides an electronic device 700, including a processor 701, a memory 702, and a computer program stored in the memory 702 and executable on the processor 701. When the program is executed by the processor 701, it implements the various processes of the above-described image data processing method embodiment and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0311] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0312] This application also provides a non-transitory computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the various processes of the above-described image data processing method embodiments and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0313] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0314] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described image data processing method.

[0315] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0316] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described image data processing method embodiments and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0317] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0318] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0319] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the related technology, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0320] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

[0321] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0322] Although embodiments of this application have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the claims and their equivalents.

Claims

1. An image data processing method, characterized in that, Applied to a client, the client connects to the server; the method includes: Receive the global model sent by the server; Based on the local training images and the actual labels corresponding to the local training images, the global model is trained with the goal of outputting the predicted labels corresponding to the local training images and the loss value corresponding to the target loss function not being greater than the loss threshold, to obtain a new local model; the target loss function is a loss function with regularization added. The new local model is sent to the server, where it is aggregated to generate a new global model, which is then sent to the client.

2. The image data processing method according to claim 1, characterized in that, The process of training the global model based on the local training images and their corresponding actual labels, with the objective of ensuring that the predicted labels corresponding to the local training images and the loss value corresponding to the target loss function are not greater than a loss threshold, to obtain a new local model includes: Based on the actual labels and the predicted labels output by the global model, the target loss function is processed to obtain the loss value; Based on the identity matrix, the learning rate corresponding to the global model, and the estimated value of the BFGS matrix, calculate the first gradient of the model parameters corresponding to the global model. The global model is updated based on the first gradient, the learning rate, and the hyperparameters corresponding to the regularization term. The process of processing the target loss function based on the actual label and the predicted label output by the global model to obtain the loss value continues until the iteration stop condition is met, resulting in the new local model.

3. The image data processing method according to claim 2, characterized in that, The step of updating the global model based on the first gradient, the learning rate, and the hyperparameters corresponding to the regularization term includes: If the first gradient is not greater than the gradient threshold, the global model is updated based on the first gradient, the learning rate, and the hyperparameters corresponding to the regularization term. If the first gradient is greater than the gradient threshold, the global model is updated based on the gradient threshold, the learning rate, and the hyperparameters corresponding to the regularization term.

4. The image data processing method according to any one of claims 1-3, characterized in that, The process of training the global model based on the local training images and their corresponding actual labels, with the objective of ensuring that the predicted labels corresponding to the local training images and the loss value corresponding to the target loss function are not greater than a loss threshold, to obtain a new local model includes: Image enhancement is performed on the local training images to obtain enhanced training images; Based on the local training image, the enhanced training image, and the actual label corresponding to the local training image, with the goal of outputting the predicted label corresponding to the local training image and the loss value corresponding to the target loss function not being greater than the loss threshold, the global model is trained to obtain a new local model.

5. The image data processing method according to any one of claims 1-3, characterized in that, The local training images include at least one of the following: images corresponding to different stored objects within the refrigeration equipment, medical images, and industrial scene images.

6. An image data processing method, characterized in that, The method is applied to a server-side application, wherein the server-side application is connected to at least one client; the method includes: Receive the local model sent by the at least one client; The local models described are aggregated to obtain a new global model; The new global model is sent to each of the clients, and the new global model is used by the clients to train on the local training images to generate new local models.

7. The image data processing method according to claim 6, characterized in that, The aggregation process of each local model to obtain a new global model includes: By aggregating the local models described above, we obtain the aggregated global model. Noise is added to the model parameters corresponding to the aggregated global model to obtain the new global model.

8. The image data processing method according to claim 7, characterized in that, The step of adding noise to the model parameters corresponding to the aggregated global model to obtain the new global model includes: Random noise is added to the model parameters corresponding to the aggregated global model to obtain random noise model parameters; Based on the parameters of the random noise model, a smooth classifier is constructed; The new global model is obtained based on the smooth classifier.

9. The image data processing method according to claim 8, characterized in that, The process of obtaining the new global model based on the smooth classifier includes: Select test images from at least one of the acquired synthetic test images and publicly available test images; Gaussian noise is added to the model parameters corresponding to the aggregated global model to obtain Gaussian noise model parameters; Based on the Gaussian noise model parameters and the test image, the smooth classifier is run to obtain the classification categories output by the smooth classifier; Based on the predicted probabilities of the classification categories corresponding to the target number, the smooth classifier is constrained to obtain the new global model.

10. An image data processing apparatus, characterized in that, Applied to a client, the client is connected to the server; the device includes: The first processing module is used to receive the global model sent by the server. The second processing module is used to train the global model based on the local training image and the actual label corresponding to the local training image, with the goal of outputting the predicted label corresponding to the local training image and the loss value corresponding to the target loss function not being greater than the loss threshold, to obtain a new local model; the target loss function is a loss function with regularization added. The third processing module is used to send the new local model to the server, whereby the server aggregates the new local model to generate a new global model, which is then sent to the client.

11. An image data processing apparatus, characterized in that, The device is applied to a server-side application, wherein the server-side application is connected to at least one client; the device includes: The fourth processing module is used to receive the local model sent by the at least one client; The fifth processing module is used to aggregate the local models to obtain a new global model; The sixth processing module is used to send the new global model to each of the clients, the new global model being used by the clients to train based on the local training images to generate new local models.

12. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the image data processing method as described in any one of claims 1-9.

13. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the image data processing method as described in any one of claims 1-9.

14. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the image data processing method as described in any one of claims 1-9.

Images