A method and device for optimizing a WTA attention mechanism neural network model

Abstract

The application discloses an optimization method and device of a WTA attention mechanism neural network model, when receiving an image training data set, performing bidirectional generative adversarial training on the image training data set, generating corresponding target training sample data, and obtaining network construction parameters, using the network construction parameters to construct an initial WTA attention mechanism network model, and performing training to generate a target WTA attention mechanism network model, when receiving target image data, extracting corresponding input feature maps, and inputting the target WTA attention mechanism network model to generate a WTA path graph, and using the WTA path graph to feed back and optimize the target WTA attention mechanism network model; and the technical problems that the existing method of using global pooling is used to change the neural network, but poor interpretability and the performance of the neural network is not obviously improved.

Description

Technical Field

[0001] This invention relates to the field of neural network construction technology, and in particular to an optimization method and apparatus for a WTA attention mechanism neural network model. Background Technology

[0002] Winner Take All (WTA) is a neural network learning algorithm. Its principle is that the inhibitory neuron is activated first and transmits activation pulse signals to the corresponding output layer neuron, activating it. At the same time, it transmits inhibition pulse signals to other output layer neurons, inhibiting their activation.

[0003] Currently, with the ever-growing size of image datasets, there is a great deal of redundant information in images. Existing neural networks struggle to focus on the important features of the input image. Furthermore, existing attention mechanisms tend to modify the neural network as a whole, such as by using global pooling, but this results in poor interpretability and minimal performance improvement. Summary of the Invention

[0004] This invention provides an optimization method and apparatus for a WTA attention mechanism neural network model, which solves the technical problems that existing neural networks have difficulty in focusing on important features of the input image, and that existing attention mechanisms focus on changing the neural network as a whole, such as using global pooling to change the neural network, but have poor interpretability and no significant improvement in neural network performance.

[0005] The first aspect of this invention provides an optimization method for a WTA attention mechanism neural network model, comprising:

[0006] When the image training dataset is received, bidirectional generative adversarial training is performed on the image training dataset to generate corresponding target training sample data and obtain network construction parameters;

[0007] The initial WTA attention mechanism network model is constructed using the aforementioned network construction parameters;

[0008] The initial WTA attention mechanism network model is trained using the target training sample data to generate the target WTA attention mechanism network model;

[0009] When target image data is received, the input feature map corresponding to the target image data is extracted;

[0010] The input feature map is used to input the target WTA attention mechanism network model to generate a WTA path map;

[0011] The target WTA attention mechanism network model is optimized using the WTA path graph feedback.

[0012] Optionally, the initial GAN ​​network model includes a generator and a recognizer. The step of performing bidirectional generative adversarial training on the image training dataset upon receiving the dataset to generate corresponding target training sample data and obtaining network construction parameters includes:

[0013] When an image training dataset is received, the first feature information corresponding to each sample training image in the image training dataset is extracted;

[0014] The generator is trained by inputting the first feature information to generate a corresponding target training image.

[0015] Extract the second feature information corresponding to the target training image;

[0016] The identifier distinguishes between the first feature information and the second feature information, and generates a discrimination result.

[0017] If the discrimination result is that the initial GAN ​​network model cannot distinguish between the first feature information and the second feature information, then a target GAN network model is generated;

[0018] The target GAN network model is input with the image training dataset to perform bidirectional generative adversarial training, generating corresponding target training sample data and obtaining network construction parameters.

[0019] Optionally, it also includes:

[0020] If the discrimination result is that the initial GAN ​​network model can distinguish the first feature information and the second feature information, the parameters of the generator and the recognizer are adjusted according to the preset gradient, and the process jumps to the step of using the recognizer to distinguish the first feature information and the second feature information and generate a discrimination result.

[0021] Optionally, the step of training the initial WTA attention mechanism network model using the target training sample data to generate the target WTA attention mechanism network model includes:

[0022] The target training sample data is input into the initial WTA attention mechanism network model to generate the corresponding training sample path graph.

[0023] Calculate the training loss value between the training sample path graph and the associated standard path graph;

[0024] Compare the training loss value with a preset training standard threshold;

[0025] If the training loss value is less than or equal to the training standard threshold, then training is stopped and the target WTA attention mechanism network model is generated.

[0026] Optionally, involving an LSTM network model, the step of using the input feature map to input the target WTA attention mechanism network model to generate a WTA path map includes:

[0027] The input feature map is used to input the target WTA attention mechanism network model to determine the corresponding winner state data;

[0028] The LSTM network model records the target attention location data associated with the winner state data and outputs the WTA path map.

[0029] The second aspect of this invention provides an optimization device for a WTA attention mechanism neural network model, comprising:

[0030] The target training sample data acquisition module is used to perform bidirectional generative adversarial training on the image training dataset when it receives the image training dataset, generate corresponding target training sample data, and acquire network construction parameters.

[0031] An initial WTA attention mechanism network model construction module is used to construct an initial WTA attention mechanism network model using the network construction parameters.

[0032] The target WTA attention mechanism network model generation module is used to train the initial WTA attention mechanism network model using the target training sample data to generate the target WTA attention mechanism network model.

[0033] The input feature map acquisition module is used to extract the input feature map corresponding to the target image data when the target image data is received.

[0034] The WTA path graph generation module is used to input the input feature map into the target WTA attention mechanism network model to generate a WTA path graph.

[0035] An optimization module is used to optimize the target WTA attention mechanism network model using the WTA path graph feedback.

[0036] Optionally, an initial GAN ​​network model is involved, the initial GAN ​​network model including a generator and a recognizer, and the target training sample data acquisition module includes:

[0037] The first feature information acquisition submodule is used to extract the first feature information corresponding to each sample training image in the image training dataset when the image training dataset is received;

[0038] The target training image acquisition submodule is used to input the first feature information into the generator for learning and to generate the corresponding target training image.

[0039] The second feature information acquisition submodule is used to extract the second feature information corresponding to the target training image;

[0040] The comparison result generation submodule is used to distinguish the first feature information and the second feature information through the recognizer and generate a discrimination result;

[0041] The image discrimination submodule is used to generate a target GAN network model if the discrimination result is that the initial GAN ​​network model cannot distinguish between the first feature information and the second feature information.

[0042] The first information processing submodule is used to input the image training dataset into the target GAN network model for bidirectional generative adversarial training, generate corresponding target training sample data, and obtain network construction parameters.

[0043] Optionally, the target training sample data acquisition module further includes:

[0044] The second information processing submodule is used to adjust the parameters of the generator and the recognizer according to a preset gradient if the discrimination result is that the initial GAN ​​network model can distinguish the first feature information and the second feature information, and then jump to the step of distinguishing the first feature information and the second feature information by the recognizer and generating a discrimination result.

[0045] Optionally, the target WTA attention mechanism network model generation module includes:

[0046] The training sample path graph acquisition submodule is used to input the target training sample data into the initial WTA attention mechanism network model and generate the corresponding training sample path graph.

[0047] The training loss value acquisition submodule is used to calculate the training loss value between the training sample path graph and the associated standard path graph.

[0048] The comparison submodule is used to compare the training loss value with a preset training standard threshold.

[0049] The third information processing submodule is used to stop training and generate the target WTA attention mechanism network model if the training loss value is less than or equal to the training standard threshold.

[0050] Optionally, the WTA path map generation module includes:

[0051] The winner state data acquisition submodule is used to input the input feature map into the target WTA attention mechanism network model to determine the corresponding winner state data;

[0052] The attention position recording submodule is used to record the target attention position data associated with the winner state data through the LSTM network model and output the WTA path map.

[0053] As can be seen from the above technical solutions, the present invention has the following advantages:

[0054] When an image training dataset is received, bidirectional generative adversarial training is performed on the dataset to generate corresponding target training sample data. Network construction parameters are then obtained, and an initial WTA attention mechanism network model is built using these parameters. This initial model is then trained using the target training sample data to generate a target WTA attention mechanism network model. When target image data is received, the corresponding input feature map is extracted and input into the target WTA attention mechanism network model to generate a WTA path map. The WTA path map is then used to optimize the target WTA attention mechanism network model. This approach addresses the difficulty of existing neural networks in focusing on important features of the input image. Furthermore, existing attention mechanisms often involve altering the neural network holistically, such as using global pooling, which results in poor interpretability and minimal performance improvements. This approach enables the target WTA attention mechanism network model to focus on important regions of the image, thereby extracting effective features more efficiently, shortening the training time, and improving accuracy. Attached Figure Description

[0055] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0056] Figure 1 A flowchart illustrating the steps of an optimization method for a WTA attention mechanism neural network model provided in Embodiment 1 of the present invention;

[0057] Figure 2 This is a flowchart illustrating the steps of an optimization method for a WTA attention mechanism neural network model provided in Embodiment 2 of the present invention.

[0058] Figure 3 This is a schematic diagram illustrating bidirectional generative adversarial training of the target GAN network model provided in Embodiment 2 of the present invention.

[0059] Figure 4 This is a structural block diagram of an optimization device for a WTA attention mechanism neural network model provided in Embodiment 3 of the present invention. Detailed Implementation

[0060] This invention provides an optimization method and apparatus for a WTA attention mechanism neural network model, which solves the problem that existing neural networks are unable to focus on important features of the input image, and that existing attention mechanisms focus on changing the neural network as a whole, such as using global pooling to change the neural network, but have poor interpretability and no significant improvement in neural network performance.

[0061] To make the objectives, features, and advantages of this invention more apparent and understandable, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described below are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0062] Please see Figure 1 , Figure 1 The flowchart illustrates the steps of an optimization method for a WTA attention mechanism neural network model provided in Embodiment 1 of the present invention.

[0063] The present invention provides an optimization method for a WTA attention mechanism neural network model, comprising:

[0064] Step 101: When the image training dataset is received, perform bidirectional generative adversarial training on the image training dataset to generate corresponding target training sample data and obtain network construction parameters.

[0065] An image training dataset refers to a set of images consisting of multiple sample training images used as inputs and training data for a model.

[0066] Target training sample data refers to the sample image data used to train the initial WTA attention mechanism network model.

[0067] Network construction parameters refer to the model parameters used to construct the initial WTA attention mechanism network model.

[0068] In this embodiment of the invention, when an image training dataset is received, bidirectional generative adversarial training is performed using the image training dataset to generate corresponding target training sample data and obtain network construction parameters.

[0069] Step 102: Construct the initial WTA attention mechanism network model using network construction parameters.

[0070] In this embodiment of the invention, an initial WTA attention mechanism network model is constructed based on the obtained network construction parameters.

[0071] Step 103: Train the initial WTA attention mechanism network model using the target training sample data to generate the target WTA attention mechanism network model.

[0072] The target WTA attention mechanism network model refers to the initial WTA attention mechanism network model after the model has been trained.

[0073] In this embodiment of the invention, the initial WTA attention mechanism network model is trained based on the obtained target training sample data to generate the corresponding target WTA attention mechanism network model.

[0074] Step 104: When the target image data is received, extract the input feature map corresponding to the target image data.

[0075] Input feature map refers to the feature map extracted from the target image data.

[0076] In this embodiment of the invention, when target image data is received, an input feature map for model input is extracted from the target image data.

[0077] Step 105: Input the input feature map into the target WTA attention mechanism network model to generate the WTA path map.

[0078] In this embodiment of the invention, the input feature map extracted from the target image data is used to input the target WTA attention mechanism network model, thereby generating the corresponding WTA path map.

[0079] Step 106: Optimize the target WTA attention mechanism network model using WTA path graph feedback.

[0080] In this embodiment of the invention, the WTA path graph output by the target WTA attention mechanism network model is fed back to optimize the target WTA attention mechanism network model.

[0081] In this embodiment of the invention, when an image training dataset is received, bidirectional generative adversarial training is performed on the image training dataset to generate corresponding target training sample data, and network construction parameters are obtained. An initial WTA attention mechanism network model is constructed using the network construction parameters. The initial WTA attention mechanism network model is trained using the target training sample data to generate a target WTA attention mechanism network model. When target image data is received, the input feature map corresponding to the target image data is extracted. The input feature map is input into the target WTA attention mechanism network model to generate a WTA path map. The WTA path map is used to optimize the target WTA attention mechanism network model. This invention solves the problem that existing neural networks are unable to focus on important features of the input image, and that existing attention mechanisms focus on changing the neural network as a whole, such as using global pooling to change the neural network, but this results in poor interpretability and insignificant improvement in neural network performance. This invention enables the target WTA attention mechanism network model to focus on important regions of the image, thereby extracting effective features more efficiently. It allows the target WTA attention mechanism network model to assign greater weight to important information while reducing the weight of unimportant information, shortening the training time of the target WTA attention mechanism network model and improving accuracy.

[0082] Please see Figure 2 , Figure 2 The flowchart illustrates the steps of an optimization method for a WTA attention mechanism neural network model provided in Embodiment 2 of the present invention.

[0083] The present invention provides an optimization method for a WTA attention mechanism neural network model, comprising:

[0084] Step 201: When the image training dataset is received, perform bidirectional generative adversarial training on the image training dataset to generate corresponding target training sample data and obtain network construction parameters.

[0085] Further, please refer to Figure 3 This involves an initial GAN ​​network model, which includes a generator and a recognizer. Step 201 may include the following sub-steps:

[0086] S11. When the image training dataset is received, extract the first feature information corresponding to each sample training image in the image training dataset.

[0087] The first feature information refers to the image feature data in the sample training images, which is used as input to the recognizer for adversarial training.

[0088] In this embodiment of the invention, when an image training dataset is received, the first feature information corresponding to each sample training image in the image training dataset is extracted.

[0089] S12. The first feature information is input into the generator for learning, and the corresponding target training image is generated.

[0090] The target training image refers to the training image generated by the generator based on the first feature information input into the generator.

[0091] In this embodiment of the invention, the generator learns based on the extracted first feature information to generate the corresponding target training image.

[0092] S13. Extract the second feature information corresponding to the target training image.

[0093] The second feature information refers to the image feature data extracted from the target training image generated by the generator, which is used as input to the recognizer for adversarial training.

[0094] In this embodiment of the invention, the corresponding second feature information is extracted from the target training image generated by the generator.

[0095] S14. The first feature information and the second feature information are distinguished by the recognizer, and the distinction result is generated.

[0096] In this embodiment of the invention, the first feature information and the second feature information are input into the recognizer for identification, thereby generating a identification result.

[0097] S15. If the initial GAN ​​network model cannot distinguish between the first feature information and the second feature information, then generate the target GAN network model.

[0098] In this embodiment of the invention, if the initial GAN ​​network model cannot distinguish between the sample training image associated with the first feature information and the target training image associated with the second feature information, that is, it cannot distinguish which image is the sample training image or the target training image, then the target GAN network model is generated.

[0099] S16. Input the image training dataset into the target GAN network model for bidirectional generative adversarial training, generate the corresponding target training sample data, and obtain the network construction parameters.

[0100] In this embodiment of the invention, an image training dataset is input into the target GAN network model for bidirectional generative adversarial training to generate corresponding target training sample data and obtain network construction parameters.

[0101] Furthermore, step 201 may also include the following sub-steps:

[0102] S17. If the initial GAN ​​network model can distinguish the first feature information and the second feature information, adjust the parameters of the generator and the recognizer according to the preset gradient, and jump to the step of distinguishing the first feature information and the second feature information through the recognizer to generate the discrimination result.

[0103] In this embodiment of the invention, if the initial GAN ​​network model can distinguish between the sample training image associated with the first feature information and the target training image associated with the second feature information, that is, it can distinguish which image is the sample training image or the target training image, then the process jumps back to the step of distinguishing the first feature information and the second feature information through the recognizer to generate the discrimination result.

[0104] In one example of this invention, the generator in the GAN network is mainly used to learn the distribution of real images so that the images it generates are more realistic and can fool the discriminator. The discriminator in the network needs to distinguish between real and fake images. During the training process, the generator tries to make the generated images more realistic, while the discriminator tries to identify the real and fake images. Over time, the generator and the discriminator are constantly competing against each other. The whole process is equivalent to a two-player game, expressed as formula (1):

[0105]

[0106] In the formula, V represents the initial GAN ​​network model, x represents the real image, z represents the noise input to the G network, G(z) represents the image generated by the G network, and D(*) represents the probability that the D network judges the image as real. For the discriminator D, the goal is for the discriminator D to correctly identify real data; this is the first half of the GAN formula, where Ex ~ pdata(x) represents the expectation that x will be obtained from Pdata. x represents real data (images), P... data (x) represents the distribution of the actual data. For the latter part, Indicates the expectation z from P z Obtained from the (z) distribution. z is random noise, P z (z) represents the distribution of the generated random noise.

[0107] Ultimately, the two networks reach a dynamic equilibrium: the images generated by the generator are close to the distribution of real images, and at this point, the generator is used to generate the target training sample data required for the initial WTA attention mechanism network model.

[0108] Step 202: Construct the initial WTA attention mechanism network model using network construction parameters.

[0109] In this embodiment of the invention, the specific implementation process of step 202 is similar to that of step 102, and will not be repeated here.

[0110] It is worth mentioning that it is necessary to define the WTA concept applicable to neural networks, including: the definition of the winner, the region / point of current network attention, the winner-takes-all path, etc.; and to define the network construction parameters required for the entire initial WTA attention mechanism network model and build the initial WTA attention mechanism network model.

[0111] The required network construction parameters include: In the neurons of a certain layer inside the neural network, for a specific input X, all neurons n in this competitive layer have an output response, and the neuron j with the largest response value is the winning neuron in the competition, which is expressed as formula (2):

[0112]

[0113] In the formula, W j W represents the current neuron weights. i This is used as the weight for the remaining neurons. A further approach is to adjust the weight vector W of the neuron that wins the competition. j The transpose T is used to conform to mathematical multiplication. n represents the total number of neurons in the competitive layer, and X represents a specific input parameter, with X in the form X∈R. H×W×C The input tensor is C, which is the number of channels in the initial WTA attention mechanism network model, and H and W are the length and width of the images in the image training dataset, respectively.

[0114] It is expressed as formula (3):

[0115] ΔW j =b(XW j )

[0116] In the formula, W j Let ΔW be the current neuron weight, and b be a constant greater than 0 and less than or equal to 1. In the initial learning phase, b has a larger value; as the neural network trains, the value of b gradually decreases, meaning the neuron weight changes gradually less. j This represents the difference in the change of the weight vector of j before and after.

[0117] Step 203: Train the initial WTA attention mechanism network model using the target training sample data to generate the target WTA attention mechanism network model.

[0118] Furthermore, step 203 may include the following sub-steps:

[0119] S21. Input the target training sample data into the initial WTA attention mechanism network model to generate the corresponding training sample path graph.

[0120] The training sample path graph refers to the path graph generated when the target training sample data is input into the initial WTA attention mechanism network model, and is used to determine whether the initial WTA attention mechanism network model has been trained.

[0121] In this embodiment of the invention, the target training sample data generated by bidirectional generative adversarial training based on the target GAN network model is input into the initial WTA attention mechanism network model to generate the corresponding training sample path graph.

[0122] S22. Calculate the training loss value between the training sample path graph and the associated standard path graph.

[0123] The standard path diagram refers to the standard path diagram corresponding to the target training sample data.

[0124] The training loss value refers to the loss value between the training sample path graph and the standard path graph.

[0125] In this embodiment of the invention, the training loss value between the training sample path graph and the associated standard path graph is calculated.

[0126] S23. Compare the training loss value with the preset training standard threshold.

[0127] The training standard threshold refers to the threshold used to determine whether the initial WTA attention mechanism network model has been trained successfully.

[0128] In this embodiment of the invention, the training loss value is compared with a preset training standard threshold.

[0129] S24. If the training loss value is less than or equal to the training standard threshold, stop training and generate the target WTA attention mechanism network model.

[0130] In this embodiment of the invention, if the training loss value is less than or equal to the training standard threshold, training is stopped and the target WTA attention mechanism network model is generated.

[0131] Furthermore, step 203 may also include the following sub-steps:

[0132] S25. If the training loss value is greater than the training standard threshold, adjust the parameters of the initial WTA attention mechanism network model according to the preset gradient, and jump to the step of inputting the target training sample data into the initial WTA attention mechanism network model to generate the corresponding training sample path graph.

[0133] In this embodiment of the invention, if the training loss value is greater than the training standard threshold, the parameters of the initial WTA attention mechanism network model are adjusted according to the preset gradient, and the process jumps to the step of inputting the target training sample data into the initial WTA attention mechanism network model to generate the corresponding training sample path graph, and retraining is performed.

[0134] Step 204: When the target image data is received, extract the input feature map corresponding to the target image data.

[0135] In this embodiment of the invention, the specific implementation process of step 204 is similar to that of step 104, and will not be repeated here.

[0136] Step 205: Input the input feature map into the target WTA attention mechanism network model to determine the corresponding winner state data.

[0137] Winner state data refers to the current flow state data of the winner determined by the competition using the input feature map input target WTA attention mechanism network model.

[0138] It is worth mentioning that when the input feature map is input into the target WTA attention mechanism network model, due to the competition rules of WTA, a winner will be generated in the current network. The target WTA attention mechanism network model marks the current winner as pi. Since the winner generated by the target WTA attention mechanism network model is a dynamic transformation process, after the WTA competition is completed, the WTA attention mechanism network will generate an attention scanning path P, which consists of {p1, p2, p3, ..., pn}.

[0139] Step 206: Record the target attention location data associated with the winner state data through the LSTM network model, and output the WTA path map.

[0140] In one example of this invention, the modified LSTM neural network model records the target attention position data associated with the winner's state data by utilizing the memory characteristics of LSTM, and the input information of the LSTM neural network model is set as X. t h t As a hidden layer, C t In cellular state;

[0141] First, in the cellular state, information is discarded through the LSTM forgetting gate, as expressed by formula (4):

[0142] f t =σ(W f ·[h t-1 ,X t ]+b f )

[0143] In the formula, f t Represents the forgetting gate information, h t-1 The hidden layer state at the previous moment, W f b is the learnable parameter matrix for the forget gate. f Let be the learnable bias parameter matrix, and let sigmoid be the activation function.

[0144] Next, the cell state is updated through the LSTM input gate, which includes receiving the previous layer's hidden state information h through the sigmoid function. t-1 and the current input information X t And hide the previous layer's state information h t-1 And the current input information X t The information is passed to the tanh function, and under the influence of the sigmoid and tanh functions, the important information will be preserved, as shown in formula (5):

[0145] i t =σ(W i [h t-1 ,X t ]+b i )

[0146]

[0147] In the formula, i t Enter the door information. The updated new report state, with Sigmoid as the activation function, W i Let X be the learnable parameter matrix of the input gate. t h t-1 b represents the hidden layer state at the previous time step. i Let W be the learnable bias parameter matrix, tanh be the activation function, and W be the bias parameter matrix. C The learnable parameter matrix for the cell renewal gate is set as X. t h t-1 b represents the hidden layer state at the previous time step. C This is a learnable bias parameter matrix.

[0148] Specifically, the term "gate" controls how much information can pass through. In this embodiment, all gates use the sigmoid activation function to map the output to the interval [0, 1]. A value of 1 indicates that all information can pass through, and a value of 0 indicates that no information passes through. The information received by both the input gate and the forget gate is X. t and h t-1 The input gate determines which information is added to C. t-1 To update C t The forget gate, on the other hand, allows the network to forget useless information. t-1 To C tThe renewal of memory cells is key to solving the problem of long-term dependency. Memory cells store useful information from various moments and continuously discard useless messages through the forget gate and receive new information through the input gate, thereby updating the cell state. By using memory cells, LSTM can capture the correlation between information from different points in a time series.

[0149] Forget Gate: Reads previous memory h0 and current input information X t Then, the sigmoid function is used to output a value between 0 and 1, where 1 represents "completely keep" and 0 represents "completely discard". Both are then processed by the sigmoid function to output a weight f between 0 and 1. t This indicates the importance of a feature, so that the feature selection in the next step can determine whether to retain it or not. The specific calculation formula is shown in formula (4).

[0150] Input gate: Input the previous cell's prior memory h0 and the current input information X. t The control signal i is obtained by updating the information through the sigmoid function. t The output i t Given a value between 0 and 1, and using the tanh function to generate a vector C. t The specific calculation formula is shown in formula (5).

[0151] Then, the cell state is calculated, the new information discovered by the LSTM neural network is updated, and a new cell state is obtained, which is expressed as formula (6):

[0152]

[0153] In the formula, C t Current cell state, C t-1 The cell state at the previous moment, i t For control signals.

[0154] In this context, all W represent convolution kernels, all b represent learned biases, and * represents convolution operations.

[0155] As can be seen, the control signal of the input gate is multiplied by the information stored in the cell to obtain the updated matrix, and the previously calculated forgotten state matrix is ​​added to obtain the cell state matrix at the current time. Finally, the state matrix C of the output information is generated. t The specific calculation formula is shown in formula (6).

[0156] In this embodiment of the invention, if the current network input is the winner pi, the modified LSTM network, utilizing the memory properties of LSTM, will combine the winner pi with the current cell state C. tIn contrast, winners pi that have not yet appeared will be allowed to be input and added to the new cell state, carried over to the next time step. This prevents attention from being permanently focused on the most active (winner) position in the saliency map and ensures the smooth generation of the WTA path graph. Winners that have appeared will be recorded in the attention scan path P. If a winner pi that has appeared is found during the traversal of P, pi can be skipped.

[0157] It is worth mentioning that in the input feature map, if a location wins the spatial competition in a certain spatial dimension, it will gain the attention of the current network and is defined as the winner of the current network's attention.

[0158] After the current network focus is suppressed, the target WTA attention mechanism network model naturally converges to the next most salient position. This process is repeated to generate an attention scanning path. Then, through the memory properties of the LSTM network, attention is prevented from being permanently concentrated on the most active (winner) position in the saliency map. After the WTA competition is completed, the attention path map of the output target WTA attention mechanism network model is obtained, and the attention flow of the neural network when extracting features can be observed.

[0159] Step 207: Optimize the target WTA attention mechanism network model using WTA path graph feedback.

[0160] In this embodiment of the invention, the target WTA attention mechanism network model is optimized based on the obtained WTA path graph.

[0161] In this embodiment of the invention, when an image training dataset is received, bidirectional generative adversarial training is performed on the image training dataset to generate corresponding target training sample data, and network construction parameters are obtained. An initial WTA attention mechanism network model is constructed using the network construction parameters. The initial WTA attention mechanism network model is trained using the target training sample data to generate a target WTA attention mechanism network model. When target image data is received, the input feature map corresponding to the target image data is extracted. The input feature map is input into the target WTA attention mechanism network model to generate a WTA path map. The WTA path map is used to optimize the target WTA attention mechanism network model. This invention solves the problem that existing neural networks are unable to focus on important features of the input image, and that existing attention mechanisms focus on changing the neural network as a whole, such as using global pooling to change the neural network, but this results in poor interpretability and insignificant improvement in neural network performance. This invention enables the target WTA attention mechanism network model to focus on important regions of the image, thereby extracting effective features more efficiently. It allows the target WTA attention mechanism network model to assign greater weight to important information while reducing the weight of unimportant information, shortening the training time of the target WTA attention mechanism network model and improving accuracy.

[0162] Please see Figure 4 , Figure 4 This is a structural block diagram of an optimization device for a WTA attention mechanism neural network model provided in Embodiment 3 of the present invention.

[0163] This invention provides an optimization device for a WTA attention mechanism neural network model, comprising:

[0164] The target training sample data acquisition module 301 is used to perform bidirectional generative adversarial training on the image training dataset when it receives the image training dataset, generate the corresponding target training sample data, and acquire the network construction parameters.

[0165] The initial WTA attention mechanism network model construction module 302 is used to construct the initial WTA attention mechanism network model using network construction parameters.

[0166] The target WTA attention mechanism network model generation module 303 is used to train the initial WTA attention mechanism network model using target training sample data to generate the target WTA attention mechanism network model.

[0167] The input feature map acquisition module 304 is used to extract the input feature map corresponding to the target image data when the target image data is received.

[0168] The WTA path graph generation module 305 is used to generate a WTA path graph by inputting the input feature map into the target WTA attention mechanism network model.

[0169] Optimization module 306 is used to optimize the target WTA attention mechanism network model using WTA path graph feedback.

[0170] Furthermore, the initial GAN ​​network model is involved, which includes a generator and a recognizer. The target training sample data acquisition module 301 includes:

[0171] The first feature information acquisition submodule is used to extract the first feature information corresponding to each sample training image in the image training dataset when the image training dataset is received.

[0172] The target training image acquisition submodule is used to learn from the first feature information input generator and generate the corresponding target training image.

[0173] The second feature information acquisition submodule is used to extract the second feature information corresponding to the target training image.

[0174] The comparison result generation submodule is used to distinguish between the first feature information and the second feature information through the recognizer and generate the discrimination result.

[0175] The image discrimination submodule is used to generate a target GAN network model if the initial GAN ​​network model cannot distinguish between the first feature information and the second feature information.

[0176] The first information processing submodule is used to input the image training dataset into the target GAN network model for bidirectional generative adversarial training, generate corresponding target training sample data, and obtain network construction parameters.

[0177] Furthermore, the target training sample data acquisition module 301 also includes:

[0178] The second information processing submodule is used to adjust the parameters of the generator and the recognizer according to the preset gradient if the initial GAN ​​network model can distinguish the first feature information and the second feature information, and then jump to the step of distinguishing the first feature information and the second feature information through the recognizer to generate the discrimination result.

[0179] Furthermore, the target WTA attention mechanism network model generation module 303 includes:

[0180] The training sample path graph acquisition submodule is used to input the target training sample data into the initial WTA attention mechanism network model and generate the corresponding training sample path graph.

[0181] The training loss value acquisition submodule is used to calculate the training loss value between the training sample path graph and the associated standard path graph.

[0182] The comparison submodule is used to compare the training loss value with the preset training standard threshold.

[0183] The third information processing submodule is used to stop training and generate the target WTA attention mechanism network model if the training loss value is less than or equal to the training standard threshold.

[0184] Furthermore, the WTA path map generation module 305 includes:

[0185] The winner state data acquisition submodule is used to input the input feature map into the target WTA attention mechanism network model to determine the corresponding winner state data.

[0186] The attention location recording submodule is used to record the target attention location data associated with the winner state data through the LSTM network model and output the WTA path map.

[0187] In this embodiment of the invention, when an image training dataset is received, bidirectional generative adversarial training is performed on the image training dataset to generate corresponding target training sample data, and network construction parameters are obtained. An initial WTA attention mechanism network model is constructed using the network construction parameters. The initial WTA attention mechanism network model is trained using the target training sample data to generate a target WTA attention mechanism network model. When target image data is received, the input feature map corresponding to the target image data is extracted. The input feature map is input into the target WTA attention mechanism network model to generate a WTA path map. The WTA path map is used to optimize the target WTA attention mechanism network model. This invention solves the problem that existing neural networks are unable to focus on important features of the input image, and that existing attention mechanisms focus on changing the neural network as a whole, such as using global pooling to change the neural network, but this results in poor interpretability and insignificant improvement in neural network performance. This invention enables the target WTA attention mechanism network model to focus on important regions of the image, thereby extracting effective features more efficiently. It allows the target WTA attention mechanism network model to assign greater weight to important information while reducing the weight of unimportant information, shortening the training time of the target WTA attention mechanism network model and improving accuracy.

[0188] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0189] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.

[0190] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0191] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0192] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0193] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An optimization method for a WTA attention mechanism neural network model, characterized in that, include: When the image training dataset is received, bidirectional generative adversarial training is performed on the image training dataset to generate corresponding target training sample data and obtain network construction parameters; The initial WTA attention mechanism network model is constructed using the aforementioned network construction parameters; The initial WTA attention mechanism network model is trained using the target training sample data to generate the target WTA attention mechanism network model; When target image data is received, the input feature map corresponding to the target image data is extracted; The input feature map is used to input the target WTA attention mechanism network model to generate a WTA path map; The target WTA attention mechanism network model is optimized using the WTA path graph feedback. The step of using the input feature map to input the target WTA attention mechanism network model to generate a WTA path map includes: The input feature map is used to input the target WTA attention mechanism network model to determine the corresponding winner state data; The target attention location data associated with the winner state data is recorded using an LSTM network model, and the WTA path map is output.

2. The optimization method for the WTA attention mechanism neural network model according to claim 1, characterized in that, The document relates to an initial GAN ​​network model, which includes a generator and a recognizer. The steps involved, upon receiving an image training dataset, performing bidirectional generative adversarial training on the image training dataset to generate corresponding target training sample data and obtaining network construction parameters, include: When an image training dataset is received, the first feature information corresponding to each sample training image in the image training dataset is extracted; The generator is trained by inputting the first feature information to generate a corresponding target training image. Extract the second feature information corresponding to the target training image; The identifier distinguishes between the first feature information and the second feature information, and generates a discrimination result. If the discrimination result is that the initial GAN ​​network model cannot distinguish between the first feature information and the second feature information, then a target GAN network model is generated; The target GAN network model is input with the image training dataset to perform bidirectional generative adversarial training, generating corresponding target training sample data and obtaining network construction parameters.

3. The optimization method for the WTA attention mechanism neural network model according to claim 2, characterized in that, Also includes: If the discrimination result is that the initial GAN ​​network model can distinguish the first feature information and the second feature information, the parameters of the generator and the recognizer are adjusted according to the preset gradient, and the process jumps to the step of using the recognizer to distinguish the first feature information and the second feature information and generate a discrimination result.

4. The optimization method for the WTA attention mechanism neural network model according to claim 1, characterized in that, The step of training the initial WTA attention mechanism network model using the target training sample data to generate the target WTA attention mechanism network model includes: The target training sample data is input into the initial WTA attention mechanism network model to generate the corresponding training sample path graph. Calculate the training loss value between the training sample path graph and the associated standard path graph; Compare the training loss value with a preset training standard threshold; If the training loss value is less than or equal to the training standard threshold, then training is stopped and the target WTA attention mechanism network model is generated.

5. An optimization device for a WTA attention mechanism neural network model, characterized in that, include: The target training sample data acquisition module is used to perform bidirectional generative adversarial training on the image training dataset when it receives the image training dataset, generate corresponding target training sample data, and acquire network construction parameters. An initial WTA attention mechanism network model construction module is used to construct an initial WTA attention mechanism network model using the network construction parameters. The target WTA attention mechanism network model generation module is used to train the initial WTA attention mechanism network model using the target training sample data to generate the target WTA attention mechanism network model. The input feature map acquisition module is used to extract the input feature map corresponding to the target image data when the target image data is received. The WTA path graph generation module is used to input the input feature map into the target WTA attention mechanism network model to generate a WTA path graph. The optimization module is used to optimize the target WTA attention mechanism network model using the WTA path graph feedback. The WTA path map generation module includes: The winner state data acquisition submodule is used to input the input feature map into the target WTA attention mechanism network model to determine the corresponding winner state data; The attention position recording submodule is used to record the target attention position data associated with the winner state data through an LSTM network model and output the WTA path map.

6. The optimization device for the WTA attention mechanism neural network model according to claim 5, characterized in that, Involving an initial GAN ​​network model, the initial GAN ​​network model includes a generator and a recognizer, and the target training sample data acquisition module includes: The first feature information acquisition submodule is used to extract the first feature information corresponding to each sample training image in the image training dataset when the image training dataset is received; The target training image acquisition submodule is used to input the first feature information into the generator for learning and to generate the corresponding target training image. The second feature information acquisition submodule is used to extract the second feature information corresponding to the target training image; The comparison result generation submodule is used to distinguish the first feature information and the second feature information through the recognizer and generate a discrimination result; The image discrimination submodule is used to generate a target GAN network model if the discrimination result is that the initial GAN ​​network model cannot distinguish between the first feature information and the second feature information. The first information processing submodule is used to input the image training dataset into the target GAN network model for bidirectional generative adversarial training, generate corresponding target training sample data, and obtain network construction parameters.

7. The optimization device for the WTA attention mechanism neural network model according to claim 6, characterized in that, The target training sample data acquisition module further includes: The second information processing submodule is used to adjust the parameters of the generator and the recognizer according to a preset gradient if the discrimination result is that the initial GAN ​​network model can distinguish the first feature information and the second feature information, and then jump to the step of distinguishing the first feature information and the second feature information by the recognizer and generating a discrimination result.

8. The optimization device for the WTA attention mechanism neural network model according to claim 5, characterized in that, The target WTA attention mechanism network model generation module includes: The training sample path graph acquisition submodule is used to input the target training sample data into the initial WTA attention mechanism network model and generate the corresponding training sample path graph. The training loss value acquisition submodule is used to calculate the training loss value between the training sample path graph and the associated standard path graph. The comparison submodule is used to compare the training loss value with a preset training standard threshold. The third information processing submodule is used to stop training and generate the target WTA attention mechanism network model if the training loss value is less than or equal to the training standard threshold.

Images