Image processing method and apparatus

Abstract

Embodiments of the present specification provide an image processing method and device, wherein the image processing method is applied to an image processing model, the image processing model comprises a neural network layer and a result output network layer associated with the neural network layer, and the method comprises: inputting a to-be-processed image into the image processing model, processing the to-be-processed image by using the neural network layer, and obtaining an intermediate output result output by the neural network layer; inputting the intermediate output result into the result output network layer, and obtaining a prediction result output by the result output network layer; and in a case where it is determined that the prediction result satisfies a preset result condition, determining the prediction result as a target prediction result of the to-be-processed image. Thus, when the image processing model is used for image processing, the prediction can be dynamically completed in advance, the defense against bit flip attack can be realized, and the prediction accuracy of the model can be ensured.

Description

Technical Field

[0001] The embodiments in this specification relate to the field of model training technology, and in particular to image processing methods. Background Technology

[0002] With the development of computer technology, neural network models have been widely applied in various fields, such as autonomous driving, malware detection, and medical diagnosis. However, neural network models can be maliciously attacked during runtime. For example, attacks targeting model parameters (i.e., bit-flipping attacks) can cause prediction errors and reduce accuracy because neural network models need to be loaded into computer memory to run. This is because flipping the bits of the model parameters in memory can tamper with the model's parameters. Therefore, an effective technical solution is urgently needed to address these problems. Summary of the Invention

[0003] In view of the above, embodiments of this specification provide an image processing method. One or more embodiments of this specification also relate to an image processing apparatus, a neural network model training method, a neural network model training apparatus, a computing device, a computer-readable storage medium, and a computer program, to address the technical deficiencies existing in the prior art.

[0004] According to a first aspect of the embodiments of this specification, an image processing method is provided, applied to an image processing model, the image processing model including a neural network layer and a result output network layer associated with the neural network layer, the method comprising:

[0005] The image to be processed is input into the image processing model, and the neural network layer is used to process the image to obtain the intermediate output result of the neural network layer.

[0006] The intermediate output results are input into the result output network layer to obtain the prediction results output by the result output network layer;

[0007] If the prediction result meets the preset result conditions, the prediction result is determined as the target prediction result of the image to be processed.

[0008] According to a second aspect of the embodiments of this specification, an image processing apparatus is provided, applied to an image processing model, the image processing model including a neural network layer and a result output network layer associated with the neural network layer, the apparatus comprising:

[0009] The first input module is configured to input the image to be processed into the image processing model, process the image to be processed using the neural network layer, and obtain the intermediate output result output by the neural network layer.

[0010] The second input module is configured to input the intermediate output result into the result output network layer to obtain the prediction result output by the result output network layer;

[0011] The determination module is configured to determine the prediction result as the target prediction result of the image to be processed when the prediction result meets the preset result conditions.

[0012] According to a third aspect of the embodiments of this specification, a method for training a neural network model is provided, comprising:

[0013] Determine the neural network layers of the initial neural network model, and add a result output network layer after the initial neural network layers to obtain the target neural network model;

[0014] The image training samples are input into the neural network layer to obtain the intermediate output results of the neural network layer;

[0015] The intermediate output results are input into the result output network layer to obtain the prediction results output by the result output network layer;

[0016] Using the image training labels corresponding to the image training samples and the prediction results, the output network layer is trained until the target neural network model that meets the training stopping condition is obtained.

[0017] According to a fourth aspect of the embodiments of this specification, a neural network model training apparatus is provided, comprising:

[0018] An add module is configured to determine the neural network layers of an initial neural network model and add a result output network layer after the neural network layers to obtain the target neural network model;

[0019] The first input module is configured to input image training samples into the neural network layer to obtain intermediate output results from the neural network layer;

[0020] The second input module is configured to input the intermediate output result into the result output network layer to obtain the prediction result output by the result output network layer;

[0021] The training module is configured to train the output network layer of the result using the image training labels corresponding to the image training samples and the prediction results, until a target neural network model that meets the training stopping condition is obtained.

[0022] According to a fifth aspect of the embodiments of this specification, a computing device is provided, comprising:

[0023] Memory and processor;

[0024] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the above method.

[0025] According to a sixth aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores computer-executable instructions that, when executed by a processor, implement the steps of the above-described method.

[0026] According to a seventh aspect of an embodiment of this specification, a computer program is provided, wherein when the computer program is executed in a computer, it causes the computer to perform the steps of the above-described method.

[0027] This specification provides an embodiment of an image processing method applied to an image processing model. The image processing model includes a neural network layer and a result output network layer associated with the neural network layer. An image to be processed is input into the image processing model, and the neural network layer processes the image to obtain an intermediate output result. The intermediate output result is then input into the result output network layer to obtain a prediction result. If the prediction result satisfies a preset result condition, the prediction result is determined as the target prediction result for the image to be processed.

[0028] The above method sets up a result output network layer associated with the neural network layer in the image processing model. The intermediate output result of the neural network layer is input into the result output network layer. When the prediction result output by the result output network layer meets the preset result conditions, the prediction result is output as the output result of the image processing model. This allows the image processing model to dynamically complete the prediction in advance when performing image processing, giving the model the ability to complete the prediction in advance. By randomly selecting the result output network layer to output the prediction result, the probability of being attacked by bit flip attacks is reduced. The neural network layer that is attacked by bit flip attacks is ignored in the model prediction process, thereby reducing the impact of bit flip attacks on the parameters of a certain neural network layer in the image processing model. This achieves defense against bit flip attacks, avoids model prediction errors caused by bit flip attacks, and ensures the prediction accuracy of the model. Attached Figure Description

[0029] Figure 1 This is a schematic diagram of a bit-flipping attack provided in one embodiment of this specification;

[0030] Figure 2 This is a schematic diagram illustrating an application scenario of an image processing method provided in one embodiment of this specification;

[0031] Figure 3 This is a flowchart illustrating an image processing method provided in one embodiment of this specification;

[0032] Figure 4 This is a schematic diagram of an image processing model in an image processing method provided in one embodiment of this specification;

[0033] Figure 5 This is a schematic diagram of the image processing model training process in an image processing method provided in one embodiment of this specification;

[0034] Figure 6 This is a flowchart illustrating the processing procedure of an image processing method provided in one embodiment of this specification.

[0035] Figure 7 This is a schematic diagram of the structure of an image processing apparatus provided in one embodiment of this specification;

[0036] Figure 8 This is a flowchart illustrating a neural network model training method provided in one embodiment of this specification;

[0037] Figure 9 This is a schematic diagram of the structure of a neural network model training device provided in one embodiment of this specification;

[0038] Figure 10 This is a structural block diagram of a computing device provided in one embodiment of this specification. Detailed Implementation

[0039] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0040] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0041] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0042] First, the terms and concepts used in one or more embodiments of this specification will be explained.

[0043] Bit flip attack: In computer devices, neural network models need to be loaded into memory for execution. Bit flip generally refers to modifying the bit value in memory in the hardware device (such as changing the bit value from 0 to 1 or from 1 to 0). It can manipulate the data stored in memory by other programs without the need for administrator privileges.

[0044] Weight error injection attack: Deep neural network models are mainly composed of parameters, the most important of which are weights. Common neural network models often have hundreds of millions of weights. By modifying a very small number of weights (less than 10), a neural network model can be completely destroyed, such as reducing its accuracy from over 90% to 10%, or implanting a backdoor to manipulate inference results through triggers. This type of weight error injection attack can be achieved through a bit-flipping attack, whereby an attacker can modify the corresponding bits of the key weights in the neural network model on a computer device. The attack can be accomplished by manipulating fewer than 10 bits.

[0045] In practical applications, bit-flipping attacks can be divided into targeted and non-targeted attacks. Non-targeted bit-flipping attacks aim to reduce the accuracy of the victim model to the point of being unusable. Targeted bit-flipping attacks mislead the victim model into predicting specific samples or samples embedded with special triggers as a specific class, but maintain model accuracy on other samples. Therefore, targeted attacks are more stealthy. There are three main existing bit-flipping attack methods:

[0046] TBT (Turned Bit Twist) attacks inject a backdoor into a victim model by flipping bits. The attacker's goal is to make the model maintain accuracy on benign input samples but err on samples with embedded specific triggers. The attacker activates the backdoor by embedding these special triggers into the input samples. The model then classifies all input samples with the specific triggers into a particular target category. TBT attacks only target the bits in the last layer of the victim model. First, the attacker selects several key network neurons in the last layer that have the greatest impact on the target category. Then, a special trigger is generated; when an input sample contains this trigger, the selected neurons are activated. Finally, the attacker uses an optimization algorithm to modify the key parameters corresponding to these neurons.

[0047] The ProFlip attack implants a backdoor into a neural network model by flipping bits in the network weights, misleading the model to predict all input samples with embedded triggers to a specific target class. This method can flip bits in all layers of the neural network. The attack selects the salient neurons that have the greatest impact on the model's output and then uses gradient descent to generate triggers. Finally, a retrieval algorithm is used to select parameters in the model and identify key bits among the parameters to flip.

[0048] The TA-LBF attack can misclassify specific samples into a specific target category by flipping key bits in the model parameters. This attack does not require a trigger, making it more stealthy than TBT and ProFlip attacks. Since the parameters are stored in memory as binary bits, the attacker programs the attack using binary integers. This binary integer programming problem is then further equated to a continuous optimization problem. The optimization problem is solved using the alternating direction multiplier method to determine the key bits to flip.

[0049] For targeted bit-flipping attacks, Figure 1 A schematic diagram of a bit-flipping attack according to one embodiment of this specification is shown. Figure 1As shown, a neural network model includes multiple neural network layers. Data is input into the model and passes through the first, second, ..., kth neural network layers until the last neural network layer, after which the result is output. Each neural network layer includes several model parameters; for example, the first neural network layer includes parameters P11, P12, and P1n. Since the parameters in the last neural network layer are often directly related to the model's prediction result, some bit-flipping attacks only target the last layer of the neural network model, i.e., flipping the bits of the parameters in the last neural network layer. For example, for parameter Pi in the last layer, the original bit 10001000 is flipped to 01001000. Another type of bit-flipping attack attacks any layer in the neural network model. For example, an attacker might use a certain retrieval algorithm to locate key bits in the neural network layer. These key bits can be understood as bits that can directly affect the model's prediction result. For example, when attacking the k-th neural network layer, the parameter PKn in the k-th neural network layer is flipped from the original bit 11010010 to 00010010.

[0050] Current defenses against bit-flipping attacks can be categorized into two types: integrity verification schemes and model enhancement schemes. Integrity verification-based methods detect bit-flipping attacks by verifying the integrity of model parameters during runtime. This method can detect any tampered model, thus detecting both directed and undirected bit-flipping attacks. However, these methods typically have poor scalability and require additional performance overhead and resource costs, making them unsuitable for existing commercial equipment and real-world scenarios.

[0051] Model augmentation-based methods focus on improving the robustness of the target model, thus significantly increasing the cost of bit-flipping attacks. A key metric for evaluating the cost of a bit-flipping attack is the number of bits to be flipped. Model augmentation methods typically increase the number of bits required for flipping significantly. However, such defenses severely reduce the model's accuracy, impacting its usability. Therefore, an effective technical solution is urgently needed to address these issues.

[0052] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in the embodiments of this specification are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0053] This specification provides an image processing method, and also relates to an image processing apparatus, a neural network model training method, a neural network model training apparatus, a computing device, and a computer-readable storage medium, which will be described in detail in the following embodiments.

[0054] See Figure 2 , Figure 2 A schematic diagram illustrating an application scenario of an image processing method provided according to an embodiment of this specification is shown.

[0055] Figure 2 It includes a client 202 and a server 204. The server 204 deploys the image processing model and is capable of executing the image processing method.

[0056] In practice, the user can send the image to be processed to the server 204 via client 202. After receiving the image, server 204 inputs it into the neural network layer of the image processing model and inputs the intermediate outputs of the neural network layer into the output network layer. If each output network layer outputs a prediction result, the server calculates the confidence score of each prediction result and sends the prediction result that meets the preset result conditions back to client 202 as the target prediction result. This target prediction result can be, for example, the type information of the image to be processed. This allows for the early output of the image processing model's prediction results, avoiding bit-flip attacks and ensuring the security of the model's operation.

[0057] See Figure 3 , Figure 3 A flowchart of an image processing method according to an embodiment of this specification is shown. The image processing method is applied to an image processing model, which includes a neural network layer and a result output network layer associated with the neural network layer. The image processing method specifically includes the following steps.

[0058] Step 302: Input the image to be processed into the image processing model, and use the neural network layer to process the image to obtain the intermediate output result of the neural network layer.

[0059] In this context, the image processing model can be understood as a neural network model used to process images. For example, it could be a model for predicting image types or for image segmentation. The output network layer can be understood as a neural network layer that outputs predicted results in advance; it serves as the exit point for the image processing model. This neural network layer includes convolutional layers and fully connected layers, and it processes the intermediate output results of the neural network layer. The output network layer associated with the neural network layer can be understood as the output network layer following the neural network layer; that is, the output of the neural network layer is the input of the output network layer. The neural network layer can be understood as the hidden layer in the image processing model, that is, the layers other than the input and output layers.

[0060] Based on this, the image to be processed can be input into the image processing model, and the neural network layer in the image processing model can be used to process the image to be processed to obtain the intermediate output result of the neural network layer. The intermediate output result is the input of the subsequent output network layer of the neural network layer.

[0061] In practical applications, image processing models typically include at least two neural network layers.

[0062] Accordingly, the image to be processed is processed using the neural network layer to obtain intermediate output results from the neural network layer, including:

[0063] The image to be processed is input into the i-th neural network layer to obtain the j-th intermediate output result of the i-th neural network layer, where i ∈ [1, n] and i and j both start from 1;

[0064] Determine if i is greater than or equal to n; if not, increment i by 1.

[0065] The j-th intermediate output result is input into the ith neural network layer to obtain the (j+1)-th intermediate output result of the ith neural network layer.

[0066] Determine if i is greater than or equal to n. If not, increment i by 1 and j by 1, and continue to execute the step of inputting the j-th intermediate output result into the i-th neural network layer to obtain the (j+1)-th intermediate output result of the i-th neural network layer.

[0067] In this context, i and j are both positive integers, for example, i is 1, 2, 3... and j is 1, 2, 3... N can be understood as the number of neural network layers included in the image processing model, and understandably, n is also a positive integer.

[0068] Taking an image processing model with three neural network layers as an example, where n is 3, the image to be processed is input into the first neural network layer, yielding the first intermediate output, where i is 1 and j is 1. Since i is less than n, i increments by 1, and the first intermediate output is input into the second neural network layer, yielding the second intermediate output, where i is 2 and j is 1. Since i is less than n, i increments by 1, and j increments by 1, and the second intermediate output is input into the third neural network layer, yielding the third intermediate output.

[0069] In summary, by obtaining the intermediate output results of each neural network layer, we can provide input to the output network layer, which facilitates the subsequent acquisition of the model's prediction results.

[0070] Step 304: Input the intermediate output result into the result output network layer to obtain the prediction result output by the result output network layer.

[0071] Specifically, after obtaining the intermediate output of the neural network layer, the intermediate output can be input into the result output network layer associated with the neural network layer, and the prediction result of the result output network layer can be obtained.

[0072] In practice, since the image processing model includes at least two neural network layers, the corresponding output network layers are also at least two.

[0073] Accordingly, the step of inputting the intermediate output result into the result output network layer to obtain the prediction result output by the result output network layer includes:

[0074] The j-th intermediate output result is input into the ith result output network layer to obtain the j-th prediction result output by the ith result output network layer, wherein the ith result output network layer is associated with the ith neural network layer;

[0075] Determine if i is greater than or equal to n. If not, increment i by 1 and j by 1, and continue to input the j-th intermediate output result into the i-th result output network layer to obtain the j-th prediction result output by the i-th result output network layer.

[0076] Specifically, after each neural network layer in the image processing model, an associated output network layer can be set up. The intermediate output of each neural network layer is input into its associated output network layer to obtain the prediction result output by the output network layer.

[0077] Continuing with the previous example, we can set the first output network layer after the first neural network layer, and the second output network layer after the second neural network layer. Since the output of the third neural network layer can be output through the output layer of the image processing model, there is no need to set an output network layer after the third neural network layer. In other words, for the last neural network layer in the image processing model, there is no need to set an associated output network layer.

[0078] Then, the first intermediate output of the first neural network layer is input into the first output network layer to obtain the first prediction result output by the first output network layer, and the second intermediate output of the second neural network layer is input into the second output network layer to obtain the second prediction result output by the second output network layer.

[0079] In summary, by setting a result output network layer after each neural network layer, the target prediction result can be output in advance, thereby reducing the impact of bit-flipping attacks.

[0080] Step 306: If the prediction result meets the preset result conditions, the prediction result is determined as the target prediction result of the image to be processed.

[0081] Specifically, after obtaining the prediction result output by the output network layer, it is necessary to first determine whether the prediction result meets the preset result conditions. For prediction results that meet the preset result conditions, they can be used as the final target prediction result output by the image processing model.

[0082] The preset result condition can be understood as the condition that enables the prediction result to be output from the result output network layer. For example, the preset result condition can be that the confidence of the prediction result is greater than a preset confidence threshold.

[0083] The preset result conditions can be set according to actual needs, and the embodiments in this specification are not limited here.

[0084] In specific implementation, determining the prediction result as the target prediction result of the image to be processed when the prediction result meets the preset result conditions includes:

[0085] Determine the confidence level of the first prediction result, wherein the first prediction result is one of the prediction results output by the at least two result output network layers;

[0086] If the confidence level of the first prediction result is greater than a preset confidence threshold, the target prediction result of the image to be processed, output by the image processing model, is obtained.

[0087] The confidence level of the prediction result can be calculated using a confidence algorithm.

[0088] Based on this, the confidence level of each prediction result output by each output network layer can be determined, and the prediction results with a confidence level greater than a preset confidence threshold can be used as the final target prediction results output by the image processing model.

[0089] Using the previous example, for the first and second prediction results, we can calculate the confidence level of the first prediction result and the confidence level of the second prediction result, and take the second prediction result with a confidence level greater than the preset confidence threshold as the target prediction result.

[0090] Furthermore, if the confidence level of the prediction results output by each output network layer does not meet the condition of exceeding a preset confidence threshold, the prediction result output by the last output network layer can be used as the target prediction result. This avoids the impact of some bit-flipping attacks that target only the last neural network layer in the image processing model.

[0091] In summary, by using predictions with a confidence level greater than a preset confidence threshold as the final output of the image processing model, the accuracy of the output can be guaranteed while defending against attacks.

[0092] In practical applications, to ensure model prediction efficiency and balance the accuracy and security of model predictions, a predetermined number of candidate result output layers can be determined from all output network layers. The specific implementation method is as follows:

[0093] Before determining the confidence level of the first prediction result, the method further includes:

[0094] From the at least two result output network layers, a preset number of first result output network layers are determined, and a first prediction result output by the first result output network layer is determined, wherein the first result output network layer is one of the at least two result output network layers.

[0095] In this model, the first output network layer is the candidate output network layer. The final target prediction result output by the image processing model is the prediction result output by any one of the candidate output network layers. The preset number can be understood as the hyperparameter of the image processing model. This hyperparameter can be predetermined during model training and can be dynamically adjusted as the model training progresses.

[0096] Specifically, a predetermined number of candidate output network layers can be randomly selected from at least two output network layers, and the prediction result output by each candidate output network layer can be determined. This facilitates the subsequent determination of the confidence level of each prediction result, and identifies the prediction results that meet the predetermined result conditions as the target prediction results.

[0097] For example, for an image processing model that includes 10 output network layers, 5 candidate output network layers can be randomly selected from the 10 output network layers. Then, the prediction result output by each of these 5 candidate output network layers can be calculated. Subsequently, the confidence of each prediction result can be calculated to determine the target prediction result.

[0098] In summary, by first selecting a preset number of candidate results to output to the network layer, a balance can be achieved between the model's prediction accuracy and security. While ensuring prediction accuracy, the network layer that outputs the final target prediction result is made as random as possible, thereby ensuring security.

[0099] In practical applications, in order to ensure the accuracy and efficiency of model prediction, the image processing model includes at least two neural network layers. Accordingly, the image processing model also includes at least two output network layers. Figure 4 A schematic diagram of an image processing model in an image processing method according to an embodiment of this specification is shown.

[0100] See Figure 4 , Figure 4 The image processing model shown includes a first neural network layer F1, a second neural network layer F2, ..., an Nth neural network layer FN, ... up to a last neural network layer Ff. The image to be processed, X, is used as the input to the image processing model. After passing through the first neural network layer ... up to the last neural network layer, the final output result FfX of the image processing model is obtained. Specifically, the image to be processed, X, passes through the first neural network layer F1, obtaining the first intermediate output result F1X of the first neural network layer F1. This first intermediate output result F1X is input to the second neural network layer F2, obtaining the second intermediate output result F2X of the second neural network layer F2, ... up to the Nth intermediate output result FNX of the Nth neural network layer FN, and the last intermediate output result FfX of the last neural network layer Ff.

[0101] In this image processing model, each neural network layer is followed by an output network layer, which includes convolutional layers and fully connected layers. Specifically, the first neural network layer F1 is followed by the first output network layer C1, and so on, until the Nth neural network layer FN is followed by the Nth output network layer CN. It's understandable that after processing by the last neural network layer in the image processing model, the processing result is directly output through the output layer. In other words, the output layer serves to output the result of the last neural network layer; therefore, there is no need to set up another output network layer after the last neural network layer.

[0102] The first intermediate output F1X of the first neural network layer F1 is input into the first result output network layer C1 to obtain the first prediction result C1X output by the first result output network layer C1. The second intermediate output F2X of the second neural network layer F2 is input into the second result output network layer C2 to obtain the second prediction result C2X output by the second result output network layer C2. Similarly, the Nth prediction result CNX output by the Nth result output network layer CN is obtained. Among these N result output network layers, a preset number (q) candidate result output network layers can be randomly selected, and the confidence scores of the prediction results output by these q candidate result output network layers are calculated. The prediction results with a confidence score greater than a preset confidence threshold are determined as the final target prediction results output by the image processing model. That is to say, the result output network layer that outputs the prediction results with a confidence score greater than the preset confidence threshold is used as the exit point for the image to be processed. Here, q is the hyperparameter of the image processing model, which can be preset and dynamically adjusted during the model training phase. Understandably, choosing smaller hyperparameters can make the selected exit point more random and the entropy greater, but it also increases the probability that the output network layer of these q candidate results cannot meet the exit condition of confidence greater than the preset confidence threshold, thereby reducing the prediction accuracy.

[0103] Furthermore, if no candidate result output network layer outputs a prediction result that meets the exit condition of having a confidence level greater than a preset confidence threshold, the last candidate result output network layer can be used as the exit point, and the prediction result output by the last candidate result output network layer can be used as the target prediction result.

[0104] For example, in N output network layers, C1, C2, and C3 are selected as candidate output network layers. The confidence scores of C1X, C2X, and C3X are calculated respectively. C2X, whose confidence score is greater than a preset confidence threshold, is taken as the target prediction result, and C2X is the exit point. If the confidence scores of C1X, C2X, and C3X do not meet the exit condition of having a confidence score greater than the preset confidence threshold, then the last candidate output network layer (i.e., C3) is taken as the exit point, and C3X is the target prediction result of this image processing model.

[0105] In practical applications, the training steps of the image processing model include:

[0106] Step 1: Determine the neural network layers of the initial image processing model, and add a result output network layer after the initial neural network layers to obtain the target image processing model;

[0107] Step 2: Input the image training samples into the neural network layer to obtain the intermediate output results of the neural network layer;

[0108] The neural network has at least two layers;

[0109] Accordingly, the step of inputting image training samples into the neural network layer to obtain intermediate output results from the neural network layer includes:

[0110] The image training sample is input into the i-th neural network layer to obtain the j-th intermediate output result of the i-th neural network layer, where i∈[1,n] and i and j both start from 1;

[0111] Determine if i is greater than or equal to n; if not, increment i by 1.

[0112] The j-th intermediate output result is input into the ith neural network layer to obtain the (j+1)-th intermediate output result of the ith neural network layer.

[0113] Determine if i is greater than or equal to n. If not, increment i by 1 and j by 1, and continue to execute the step of inputting the j-th intermediate output result into the i-th neural network layer to obtain the (j+1)-th intermediate output result of the i-th neural network layer.

[0114] Step 3: Input the intermediate output result into the result output network layer to obtain the prediction result output by the result output network layer;

[0115] The output network layer must have at least two layers;

[0116] The step of inputting the intermediate output result into the result output network layer to obtain the prediction result output by the result output network layer includes:

[0117] The j-th intermediate output result is input into the ith result output network layer to obtain the j-th prediction result output by the ith result output network layer, wherein the ith result output network layer is associated with the ith neural network layer;

[0118] Determine if i is greater than or equal to n. If not, increment i by 1 and j by 1, and continue to input the j-th intermediate output result into the i-th result output network layer to obtain the j-th prediction result output by the i-th result output network layer.

[0119] Step 4: Using the image training labels corresponding to the image training samples and the prediction results, train the output network layer of the results until a target image processing model that meets the training stopping condition is obtained.

[0120] The step of training the output network layer using the image training labels corresponding to the image training samples and the prediction results until a target image processing model that meets the training stopping condition is obtained includes:

[0121] Using the image training labels corresponding to the image training samples and the j-th prediction result, the j-th result output network layer is trained until the target image processing model that meets the training stopping condition is obtained.

[0122] The target image processing model can be understood as the trained image processing model.

[0123] Specifically, when training an image processing model, each output network layer in the image processing model can be trained. The specific implementation method is similar to the aforementioned image processing method, and will not be repeated here.

[0124] Furthermore, since attackers might perform bit-flipping attacks on all neural network layers in an image processing model, even though this would significantly increase the number of bits that need to be flipped, thus increasing the attack cost, the possibility of such bit-flipping attacks still exists. Therefore, robust training can be performed on the output network layer to simulate the impact of bit-flipping on the image processing model, construct flipped training samples, and train the output network layer. The specific implementation method is as follows:

[0125] After determining the neural network layers of the initial image processing model and adding a result output network layer after the neural network layers to obtain the target image processing model, the method further includes:

[0126] Based on the initial image processing model, a flipped sample providing model is constructed, wherein the flipped sample providing model includes a neural network layer;

[0127] The image training sample is input into the neural network layer to obtain the flipped training sample output by the neural network layer;

[0128] Using the image training labels corresponding to the flipped training samples and the image training samples, the output network layer of the initial image processing model is trained to obtain a target image processing model including the neural network layer of the initial image processing model and the trained output network layer.

[0129] The flipped sample model can be understood as a model that has undergone bit flipping, which can be used to provide flipped training samples for robust training of the output network layer. The target image processing model can be understood as the initial image processing model after training.

[0130] Based on this, a flipped sample provision model can be constructed according to the initial image processing model after bit flipping. The image training samples are then input into the neural network layer of the flipped sample provision model to obtain the flipped training samples output by the neural network layer. The image training labels corresponding to the flipped training samples and the image training samples are used to train the output network layer of the initial image processing model to obtain the target image processing model that meets the training stopping condition.

[0131] In summary, by using flipped training samples to train the output network layer, the output network layer can be made to adapt to the effects of bit-flipping attacks. The flipped training samples are used to simulate the output of the attacked neural network layer, and the output network layer learns this type of data to correct the prediction results in adversarial scenarios.

[0132] In practice, to ensure that the neural network layers of the initial image processing model remain unaffected, the model being attacked can be simulated by replicating the initial image processing model. The specific implementation method is as follows:

[0133] The step of constructing a flipped sample provision model based on the initial image processing model includes:

[0134] The initial image processing model is copied, and the target bits corresponding to the model parameters in the copied initial image processing model are determined. The target bits are then flipped to obtain a flipped sample provision model.

[0135] Here, the target bit can be understood as the bit that might be flipped in a bit-flipping attack. The copied initial image processing model can be understood as the model obtained after copying the initial image processing model. For example, for an initial image processing model A, copying it results in an initial image processing model A1. Subsequent target bit flipping is performed in the copied initial image processing model A1. The initial image processing model here can be understood as the initial image processing model before adding the output network layer; that is, the initial image processing model does not include the output network layer.

[0136] Based on this, the initial image processing model can be copied to obtain a copied model, which is identical to the initial image processing model. The target bits corresponding to the model parameters in the copied model are determined, and these target bits are flipped to obtain a flipped sample model.

[0137] In summary, by replicating the initial image processing model, the neural network of the initial image processing model remains unaffected, while simulating the attacked model, thereby improving the realism of the attacked model and further enhancing the realism of subsequent flipped training samples.

[0138] In practice, current bit-flipping attacks typically consider the gradient of a bit as a key factor in selecting the bit to be attacked. The gradient value of a bit reflects its importance in the model's decision-making, thus affecting the model's prediction performance. To determine the target bit that may be flipped, a pre-defined algorithm can be used to calculate the gradient value of the bit, and the target bit can be determined based on the gradient value. The specific implementation method is as follows:

[0139] The process of determining the target bits corresponding to the model parameters in the initial image processing model obtained by copying, and flipping the target bits to obtain the flipped sample providing model, includes:

[0140] Determine at least two bits corresponding to the model parameters in the initial image processing model obtained by copying;

[0141] According to a preset algorithm, the gradient value of the first bit is calculated, wherein the first bit is one of the at least two bits;

[0142] If the gradient value of the first bit is determined to be greater than a preset gradient value threshold, the first bit is determined as the target bit, and the target bit is flipped to obtain the flipped initial image processing model.

[0143] In the initial image processing model after flipping, at least two bits corresponding to the model parameters are determined, and the step of calculating the gradient value of the first bit according to the preset algorithm is continued until the preset stopping condition is reached to obtain the flipped sample providing model.

[0144] The preset algorithm can be understood as an algorithm for calculating bit gradient values, such as a vulnerability protection algorithm, the specific formula of which is as follows.

[0145]

[0146] Among them, L inf For the inference loss of bits, Let F be the cross-entropy loss, l be the true label, and F be the cross-entropy loss. final (x) represents the output of the last neural network layer in the image processing model. The preset stopping condition can be understood as reaching the preset number of iterations.

[0147] Based on this, at least two bits corresponding to the model parameters in the copied initial image processing model can be identified. According to a preset algorithm, the gradient value of each bit is calculated. Bits with gradient values ​​greater than a preset gradient threshold are identified as target bits, and these target bits are flipped to obtain the flipped initial image processing model. Then, at least two bits corresponding to the model parameters in the flipped initial image processing model are identified, and the preset algorithm is used again to calculate the gradient value of each bit. Bits with gradient values ​​greater than a preset gradient threshold are identified as target bits, and these target bits are flipped again until a preset number of iterations is reached, resulting in the final flipped sample model.

[0148] In practical applications, during each iteration, the gradient of each bit with respect to the inference loss can be calculated. Then, the vulnerability of each bit is sorted according to the absolute value of the gradient. The K largest vulnerable bits are determined based on the sorting result and are used as target bits (i.e., vulnerable bits). These K bits are then flipped. This iterative process is repeated several times to finally obtain a flipped sample model.

[0149] In summary, by identifying the target bits that may be attacked and recalculating the gradient of each bit in each iteration, the model can be adapted to the dynamic changes of the bit as it is flipped. The recalculated gradient values ​​can reflect the importance of the bit in the dynamically changing model.

[0150] Furthermore, the output network layer can be trained simultaneously using both image training samples and flipped training samples. See also Figure 5 , Figure 5 A schematic flowchart of image processing model training in an image processing method according to an embodiment of this specification is shown, and the specific steps are as follows.

[0151] Step 502: Copy the initial image processing model to obtain the copied model.

[0152] Step 504: Add a result output network layer after each neural network layer in the initial image processing model to obtain the target image processing model.

[0153] Specifically, add a result output network layer C1 after neural network layer F1, and so on, add a result output network layer CN after neural network layer FN.

[0154] Step 506: Identify the target bits that may be attacked in the replication model, and flip the target bits to obtain the flipped sample model.

[0155] Step 508: Obtain image training samples and image training labels from the training dataset.

[0156] Step 510: Input the image training samples into the target image processing model to obtain the intermediate output results of the neural network layer in the target processing model.

[0157] Specifically, we obtain the intermediate output result F1X of the output of neural network layer F1, and so on, we obtain the intermediate output result FNX of the output of neural network layer FN.

[0158] The intermediate output is used as the original training sample for the output network layer.

[0159] Step 512: Input the image training samples into the flipped sample providing model to obtain the intermediate output results of the neural network layer output in the flipped sample providing model.

[0160] Specifically, we obtain the intermediate output result F`1X of the flipped neural network layer F`1, and so on, we obtain the intermediate output result F`NX of the flipped neural network layer F`N.

[0161] The intermediate output is used as the flipped training sample for the output network layer.

[0162] Step 514: Using the original training samples, flipped training samples, and image training labels, train the output network layer until the target image processing model that meets the training conditions is obtained.

[0163] Specifically, the intermediate output F1X of neural network layer F1, the intermediate output F`1X of the flipped neural network layer F`1, and the image training labels are used to train the result output network layer C1. The training process for other result output network layers is similar to that for result output network layer C1, and will not be repeated here.

[0164] Furthermore, the original training samples and image training labels can be used to train the output network layer, and then the flipped training samples and image training labels can be used to train the output network layer again. It is understood that the training order using the original training samples and the flipped training samples can be determined according to actual needs, and this specification does not limit the specific training order.

[0165] In summary, by training the output network layer using both the original training samples and the flipped training samples, the robustness of the output network layer is improved, enabling it to adapt to the effects of bit-flipping attacks and further defend against them.

[0166] The following is in conjunction with the appendix Figure 6 The image processing method provided in this specification will be further explained using an example of its application in a neural network model including a three-layer neural network. Figure 6 A flowchart illustrating the processing steps of an image processing method according to an embodiment of this specification is shown, specifically including the following steps.

[0167] Step 602: Input the image to be processed into the first neural network layer in the image processing model to obtain the first intermediate output result of the first neural network layer.

[0168] Step 604: Input the first intermediate output result into the first result output network layer to obtain the first prediction result output by the first result output network layer.

[0169] Specifically, the first output network layer also belongs to the image processing model, and it is set after the first neural network layer.

[0170] Step 606: Input the first intermediate output result into the second neural network layer to obtain the second intermediate output result of the second neural network layer.

[0171] Step 608: Input the second intermediate output result into the second result output network layer to obtain the second prediction result output by the second result output network layer.

[0172] Step 610: Calculate the confidence level of the first prediction result, calculate the confidence level of the second prediction result, and determine the prediction result with a confidence level greater than the preset confidence threshold as the target prediction result of the image processing model.

[0173] The above method sets up a result output network layer associated with the neural network layer in the image processing model. The intermediate output result of the neural network layer is input into the result output network layer. When the prediction result output by the result output network layer meets the preset result conditions, the prediction result is output as the output result of the image processing model. This allows the image processing model to dynamically complete the prediction in advance when performing image processing, giving the model the ability to complete the prediction in advance. By randomly selecting the result output network layer to output the prediction result, the probability of being attacked by bit flip attacks is reduced. The neural network layer that is attacked by bit flip attacks is ignored in the model prediction process, thereby reducing the impact of bit flip attacks on the parameters of a certain neural network layer in the image processing model. This achieves defense against bit flip attacks, avoids model prediction errors caused by bit flip attacks, and ensures the prediction accuracy of the model.

[0174] Corresponding to the above method embodiments, this specification also provides embodiments of image processing apparatus. Figure 7 A schematic diagram of the structure of an image processing apparatus according to one embodiment of this specification is shown. Figure 7 As shown, the device is applied to an image processing model, which includes a neural network layer and a result output network layer associated with the neural network layer. The device includes:

[0175] The first input module 702 is configured to input the image to be processed into the image processing model, process the image to be processed using the neural network layer, and obtain the intermediate output result output by the neural network layer.

[0176] The second input module 704 is configured to input the intermediate output result into the result output network layer to obtain the prediction result output by the result output network layer;

[0177] The determining module 706 is configured to determine the prediction result as the target prediction result of the image to be processed when the prediction result is determined to meet the preset result conditions.

[0178] In an optional embodiment, the neural network has at least two layers; the first input module 702 is further configured to:

[0179] The image to be processed is input into the i-th neural network layer to obtain the j-th intermediate output result of the i-th neural network layer, where i ∈ [1, n] and i and j both start from 1;

[0180] Determine if i is greater than or equal to n; if not, increment i by 1.

[0181] The j-th intermediate output result is input into the ith neural network layer to obtain the (j+1)-th intermediate output result of the ith neural network layer.

[0182] Determine if i is greater than or equal to n. If not, increment i by 1 and j by 1, and continue to execute the step of inputting the j-th intermediate output result into the i-th neural network layer to obtain the (j+1)-th intermediate output result of the i-th neural network layer.

[0183] In an optional embodiment, the result output network layer has at least two layers; the second input module 704 is further configured to:

[0184] The j-th intermediate output result is input into the ith result output network layer to obtain the j-th prediction result output by the ith result output network layer, wherein the ith result output network layer is associated with the ith neural network layer;

[0185] Determine if i is greater than or equal to n. If not, increment i by 1 and j by 1, and continue to input the j-th intermediate output result into the i-th result output network layer to obtain the j-th prediction result output by the i-th result output network layer.

[0186] In an optional embodiment, the determining module 706 is further configured to:

[0187] Determine the confidence level of the first prediction result, wherein the first prediction result is one of the prediction results output by the at least two result output network layers;

[0188] If the confidence level of the first prediction result is greater than a preset confidence threshold, the target prediction result of the image to be processed, output by the image processing model, is obtained.

[0189] In an optional embodiment, the determining module 706 is further configured to:

[0190] From the at least two result output network layers, a preset number of first result output network layers are determined, and a first prediction result output by the first result output network layer is determined, wherein the first result output network layer is one of the at least two result output network layers.

[0191] In an optional embodiment, the device further includes a training module configured to:

[0192] The neural network layers of the initial image processing model are determined, and a result output network layer is added after the neural network layers to obtain the target image processing model;

[0193] The image training samples are input into the neural network layer to obtain the intermediate output results of the neural network layer;

[0194] The intermediate output results are input into the result output network layer to obtain the prediction results output by the result output network layer;

[0195] Using the image training labels corresponding to the image training samples and the prediction results, the result output network layer is trained until the target image processing model that meets the training stopping condition is obtained.

[0196] In an optional embodiment, the neural network has at least two layers; the training module is further configured to:

[0197] The image training sample is input into the i-th neural network layer to obtain the j-th intermediate output result of the i-th neural network layer, where i∈[1,n] and i and j both start from 1;

[0198] Determine if i is greater than or equal to n; if not, increment i by 1.

[0199] The j-th intermediate output result is input into the ith neural network layer to obtain the (j+1)-th intermediate output result of the ith neural network layer.

[0200] Determine if i is greater than or equal to n. If not, increment i by 1 and j by 1, and continue to execute the step of inputting the j-th intermediate output result into the i-th neural network layer to obtain the (j+1)-th intermediate output result of the i-th neural network layer.

[0201] In an optional embodiment, the output network layer has at least two layers; the training module is further configured to:

[0202] The j-th intermediate output result is input into the ith result output network layer to obtain the j-th prediction result output by the ith result output network layer, wherein the ith result output network layer is associated with the ith neural network layer;

[0203] Determine if i is greater than or equal to n. If not, increment i by 1 and j by 1, and continue to input the j-th intermediate output result into the i-th result output network layer to obtain the j-th prediction result output by the i-th result output network layer.

[0204] In an optional embodiment, the training module is further configured to:

[0205] Using the image training labels corresponding to the image training samples and the j-th prediction result, the j-th result output network layer is trained until the target image processing model that meets the training stopping condition is obtained.

[0206] In an optional embodiment, the training module is further configured to:

[0207] Based on the initial image processing model, a flipped sample providing model is constructed, wherein the flipped sample providing model includes a neural network layer;

[0208] The image training sample is input into the neural network layer to obtain the flipped training sample output by the neural network layer;

[0209] Using the image training labels corresponding to the flipped training samples and the image training samples, the output network layer of the initial image processing model is trained to obtain a target image processing model including the neural network layer of the initial image processing model and the trained output network layer.

[0210] In an optional embodiment, the training module is further configured to:

[0211] The initial image processing model is copied, and the target bits corresponding to the model parameters in the copied initial image processing model are determined. The target bits are then flipped to obtain a flipped sample provision model.

[0212] In an optional embodiment, the training module is further configured to:

[0213] Determine at least two bits corresponding to the model parameters in the initial image processing model obtained by copying;

[0214] According to a preset algorithm, the gradient value of the first bit is calculated, wherein the first bit is one of the at least two bits;

[0215] If the gradient value of the first bit is determined to be greater than a preset gradient value threshold, the first bit is determined as the target bit, and the target bit is flipped to obtain the flipped initial image processing model.

[0216] In the initial image processing model after flipping, at least two bits corresponding to the model parameters are determined, and the step of calculating the gradient value of the first bit according to the preset algorithm is continued until the preset stopping condition is reached to obtain the flipped sample providing model.

[0217] In summary, the aforementioned device sets up a result output network layer associated with the neural network layer in the image processing model. The intermediate output of the neural network layer is input into the result output network layer. When the prediction result output by the result output network layer meets preset conditions, this prediction result is output as the image processing model's output. This allows the image processing model to dynamically and proactively complete predictions during image processing, giving the model the ability to predict in advance. By randomly selecting the result output network layer to output the prediction result, the probability of being attacked by bit flip attacks is reduced. The neural network layer vulnerable to bit flip attacks is ignored during model prediction, thus reducing the impact of bit flip attacks on the parameters of a particular neural network layer in the image processing model. This achieves defense against bit flip attacks, avoids model prediction errors caused by bit flip attacks, and ensures the model's prediction accuracy.

[0218] The above is an illustrative scheme of an image processing apparatus according to this embodiment. It should be noted that the technical solution of this image processing apparatus and the technical solution of the image processing method described above belong to the same concept. For details not described in detail in the technical solution of the image processing apparatus, please refer to the description of the technical solution of the image processing method described above.

[0219] Corresponding to the above method embodiments, this specification also provides embodiments of neural network model training methods. Figure 8 A flowchart of a neural network model training method according to an embodiment of this specification is shown, and the specific steps are as follows.

[0220] Step 802: Determine the neural network layers of the initial neural network model, and add a result output network layer after the neural network layers to obtain the target neural network model;

[0221] Step 804: Input the image training samples into the neural network layer to obtain the intermediate output results of the neural network layer;

[0222] Step 806: Input the intermediate output result into the result output network layer to obtain the prediction result output by the result output network layer;

[0223] Step 808: Using the image training labels corresponding to the image training samples and the prediction results, train the output network layer of the results until a target neural network model that meets the training stopping condition is obtained.

[0224] The neural network model can be the aforementioned image processing model, in which case its training samples are image training samples. The neural network model can also be a text processing model, in which case its training samples are text training samples. The embodiments described in this specification are not limited thereto.

[0225] In summary, the above method sets up a result output network layer associated with the neural network layer in the image processing model. The intermediate output of the neural network layer is input into the result output network layer. When the prediction result output by the result output network layer meets preset conditions, this prediction result is used as the output of the image processing model. This allows the image processing model to dynamically complete predictions in advance, giving the model the ability to predict ahead of time. By randomly selecting the result output network layer to output the prediction result, the probability of being attacked by bit flip attacks is reduced. The neural network layer vulnerable to bit flip attacks is ignored during the model's prediction process, thus reducing the impact of bit flip attacks on the parameters of a particular neural network layer in the image processing model. This achieves defense against bit flip attacks, avoids model prediction errors caused by bit flip attacks, and ensures the model's prediction accuracy.

[0226] The above is an illustrative scheme of a neural network model training method according to this embodiment. It should be noted that the technical solution of this neural network model training method and the technical solution of the image processing method described above belong to the same concept. For details not described in detail in the technical solution of the neural network model training method, please refer to the description of the technical solution of the image processing method described above.

[0227] Corresponding to the above method embodiments, this specification also provides embodiments of a neural network model training device. Figure 9 This specification shows a schematic diagram of a neural network model training apparatus according to one embodiment. The apparatus includes:

[0228] Add module 902, configured to determine the neural network layers of the initial neural network model, and add a result output network layer after the neural network layers to obtain the target neural network model;

[0229] The first input module 904 is configured to input image training samples into the neural network layer to obtain intermediate output results from the neural network layer;

[0230] The second input module 906 is configured to input the intermediate output result into the result output network layer to obtain the prediction result output by the result output network layer;

[0231] The training module 908 is configured to train the output network layer of the result using the image training labels corresponding to the image training samples and the prediction results, until a target neural network model that meets the training stopping condition is obtained.

[0232] In summary, the aforementioned device sets up a result output network layer associated with the neural network layer in the image processing model. The intermediate output of the neural network layer is input into the result output network layer. When the prediction result output by the result output network layer meets preset conditions, this prediction result is output as the image processing model's output. This allows the image processing model to dynamically and proactively complete predictions during image processing, giving the model the ability to predict in advance. By randomly selecting the result output network layer to output the prediction result, the probability of being attacked by bit flip attacks is reduced. The neural network layer vulnerable to bit flip attacks is ignored during model prediction, thus reducing the impact of bit flip attacks on the parameters of a particular neural network layer in the image processing model. This achieves defense against bit flip attacks, avoids model prediction errors caused by bit flip attacks, and ensures the model's prediction accuracy.

[0233] The above is an illustrative scheme of a neural network model training device according to this embodiment. It should be noted that the technical solution of this neural network model training device and the technical solution of the neural network model training method described above belong to the same concept. For details not described in detail in the technical solution of the neural network model training device, please refer to the description of the technical solution of the neural network model training method described above.

[0234] Figure 10 A structural block diagram of a computing device 1000 according to one embodiment of this specification is shown. The components of the computing device 1000 include, but are not limited to, a memory 1010 and a processor 1020. The processor 1020 is connected to the memory 1010 via a bus 1030, and a database 1050 is used to store data.

[0235] The computing device 1000 also includes an access device 1040, which enables the computing device 1000 to communicate via one or more networks 1060. Examples of these networks include Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or combinations of communication networks such as the Internet. The access device 1040 may include one or more of any type of wired or wireless network interface (e.g., a network interface card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, a Wi-MAX (Worldwide Interoperability for Microwave Access) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a Near Field Communication (NFC) interface, and so on.

[0236] In one embodiment of this application, the aforementioned components of the computing device 1000 and Figure 10 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 10 The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this application. Those skilled in the art can add or replace other components as needed.

[0237] The computing device 1000 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). The computing device 1000 can also be a mobile or stationary server.

[0238] The processor 1020 is configured to execute the following computer-executable instructions, which, when executed by the processor, implement the steps of the above method.

[0239] The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the above method belong to the same concept, and all details not described in detail in the technical solution of the computing device can be referred to the description of the technical solution of the above method.

[0240] An embodiment of this specification also provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the above-described method.

[0241] The above is an illustrative scheme of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium and the technical solution of the method described above belong to the same concept, and all details not described in detail in the technical solution of the storage medium can be referred to the description of the technical solution of the method described above.

[0242] An embodiment of this specification also provides a computer program, wherein when the computer program is executed in a computer, it causes the computer to perform the steps of the above-described method.

[0243] The above is an illustrative example of a computer program according to this embodiment. It should be noted that the technical solution of this computer program and the technical solution of the method described above belong to the same concept. Details not described in detail in the technical solution of the computer program can be found in the description of the technical solution of the method described above.

[0244] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0245] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added to or subtracted according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0246] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments in this specification are not limited to the described order of actions, because according to the embodiments in this specification, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments in this specification.

[0247] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0248] The preferred embodiments disclosed above are merely illustrative of this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments described herein. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the embodiments, thereby enabling those skilled in the art to better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. An image processing method applied to an image processing model, the image processing model comprising at least two neural network layers and a result output network layer associated with the neural network layers, the method comprising: The image to be processed is input into the image processing model, and the image to be processed is processed by the neural network layer to obtain the intermediate output result of the neural network layer, wherein each neural network layer corresponds to an intermediate output result; The intermediate output result is input into the result output network layer to obtain the prediction result output by the result output network layer. If the neural network layer is the last layer, there is no need to set up a result output network layer, and the prediction result is directly output by the output layer of the image processing model. If at least one of the prediction results satisfies a preset result condition, the prediction result is determined as the target prediction result of the image to be processed; The image processing model is trained in the following manner: based on the initial image processing model, a flipped sample providing model is constructed, wherein the flipped sample providing model includes a neural network layer and is a model after bit flipping; image training samples are input into the neural network layer to obtain the flipped training samples output by the neural network layer; using the flipped training samples and the image training labels corresponding to the image training samples, the result output network layer of the initial image processing model is trained to obtain a target image processing model including the neural network layer of the initial image processing model and the trained result output network layer.

2. The method according to claim 1, wherein processing the image to be processed using the neural network layer to obtain an intermediate output result from the neural network layer comprises: The image to be processed is input into the i-th neural network layer to obtain the j-th intermediate output result of the i-th neural network layer, where i ∈ [1, n] and i and j both start from 1; Determine if i is greater than or equal to n; if not, increment i by 1. The j-th intermediate output result is input into the ith neural network layer to obtain the (j+1)-th intermediate output result of the ith neural network layer. Determine if i is greater than or equal to n. If not, increment i by 1 and j by 1, and continue to execute the step of inputting the j-th intermediate output result into the i-th neural network layer to obtain the (j+1)-th intermediate output result of the i-th neural network layer.

3. The method according to claim 2, wherein the output network layer has at least two layers; Accordingly, the intermediate output result is input into the result output network layer to obtain the prediction result output by the result output network layer, including: The j-th intermediate output result is input into the ith result output network layer to obtain the j-th prediction result output by the ith result output network layer, wherein the ith result output network layer is associated with the ith neural network layer; Determine if i is greater than or equal to n. If not, increment i by 1 and j by 1, and continue to input the j-th intermediate output result into the i-th result output network layer to obtain the j-th prediction result output by the i-th result output network layer.

4. The method according to claim 3, wherein, when it is determined that the at least one prediction result satisfies a preset result condition, the prediction result is determined as the target prediction result of the image to be processed, comprising: Determine the confidence level of the first prediction result, wherein the first prediction result is one of the prediction results output by at least two result output network layers; If the confidence level of the first prediction result is greater than a preset confidence threshold, the target prediction result of the image to be processed, output by the image processing model, is obtained.

5. The method according to claim 4, further comprising, before determining the confidence level of the first prediction result: From the at least two result output network layers, a preset number of first result output network layers are determined, and a first prediction result output by the first result output network layer is determined, wherein the first result output network layer is one of the at least two result output network layers.

6. The method according to any one of claims 1-5, wherein the training step of the image processing model comprises: The neural network layers of the initial image processing model are determined, and a result output network layer is added after the neural network layers to obtain the target image processing model; The image training samples are input into the neural network layer to obtain the intermediate output results of the neural network layer; The intermediate output results are input into the result output network layer to obtain the prediction results output by the result output network layer; Using the image training labels corresponding to the image training samples and the prediction results, the result output network layer is trained until the target image processing model that meets the training stopping condition is obtained.

7. The method according to claim 6, wherein the neural network layer comprises at least two layers; Accordingly, the image training samples are input into the neural network layer to obtain intermediate output results from the neural network layer, including: The image training sample is input into the i-th neural network layer to obtain the j-th intermediate output result of the i-th neural network layer, where i∈[1,n] and i and j both start from 1; Determine if i is greater than or equal to n; if not, increment i by 1. The j-th intermediate output result is input into the ith neural network layer to obtain the (j+1)-th intermediate output result of the ith neural network layer. Determine if i is greater than or equal to n. If not, increment i by 1 and j by 1, and continue to execute the step of inputting the j-th intermediate output result into the i-th neural network layer to obtain the (j+1)-th intermediate output result of the i-th neural network layer.

8. The method according to claim 7, wherein the output network layer has at least two layers; The step of inputting the intermediate output result into the result output network layer to obtain the prediction result output by the result output network layer includes: The j-th intermediate output result is input into the ith result output network layer to obtain the j-th prediction result output by the ith result output network layer, wherein the ith result output network layer is associated with the ith neural network layer; Determine if i is greater than or equal to n. If not, increment i by 1 and j by 1, and continue to input the j-th intermediate output result into the i-th result output network layer to obtain the j-th prediction result output by the i-th result output network layer.

9. The method according to claim 8, wherein training the output network layer using the image training labels corresponding to the image training samples and the prediction results until a target image processing model satisfying the training stopping condition is obtained, comprises: Using the image training labels corresponding to the image training samples and the j-th prediction result, the j-th result output network layer is trained until the target image processing model that meets the training stopping condition is obtained.

10. The method according to claim 1, wherein constructing the flipped sample provision model based on the initial image processing model comprises: The initial image processing model is copied, and the target bits corresponding to the model parameters in the copied initial image processing model are determined. The target bits are then flipped to obtain a flipped sample provision model.

11. The method according to claim 10, wherein determining the target bit corresponding to the model parameters in the copied initial image processing model, and performing a flipping process on the target bit to obtain a flipped sample providing model, comprises: Determine at least two bits corresponding to the model parameters in the initial image processing model obtained by copying; According to a preset algorithm, the gradient value of the first bit is calculated, wherein the first bit is one of the at least two bits; If the gradient value of the first bit is determined to be greater than a preset gradient value threshold, the first bit is determined as the target bit, and the target bit is flipped to obtain the flipped initial image processing model. In the initial image processing model after flipping, at least two bits corresponding to the model parameters are determined, and the step of calculating the gradient value of the first bit according to the preset algorithm is continued until the preset stopping condition is reached to obtain the flipped sample providing model.

12. A method for training a neural network model, comprising: Determine the neural network layers of the initial neural network model, and add a result output network layer after the initial neural network layers to obtain the target neural network model; Based on the initial neural network model, a flipped sample providing model is constructed, wherein the flipped sample providing model includes a neural network layer and is a model after bit flipping; The image training sample is input into the neural network layer to obtain the flipped training sample output by the neural network layer; Using the image training labels corresponding to the flipped training samples and the image training samples, the output network layer of the initial neural network model is trained to obtain a target neural network model including the neural network layer of the initial neural network model and the trained output network layer. The image training samples are input into the neural network layer to obtain the intermediate output results of the neural network layer; The intermediate output results are input into the result output network layer to obtain the prediction results output by the result output network layer; Using the image training labels corresponding to the image training samples and the prediction results, the output network layer is trained until the target neural network model that meets the training stopping condition is obtained.

13. A computing device, comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the method according to any one of claims 1 to 11 or 12.

Images