Method and apparatus for training neural network and image recognition method

Abstract

A method and apparatus for training a neural network and an image recognition method are disclosed. The neural network includes a first model and a second model. The method for training the neural network includes obtaining a second image from a first image, the second image having a lower quality than the first image; inputting the first image to the first model of the neural network and inputting the second image to the second model of the neural network; calculating an attention map and a gradient map for each of the first model and the second model; constructing a loss function based on a matrix of a dot product of the gradient map and the attention map of the first model and a matrix of a dot product of the gradient map and the attention map of the second model; and training the neural network by minimizing the loss function.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of neural networks, and in particular to training of neural networks and image recognition. BACKGROUND

[0002] In recent years, face recognition technology has made great progress in both academic and industrial fields and has been widely applied. In practical applications, the recognition performance of low-quality face images is still limited. This is mainly due to the fact that low-quality face images can damage some face features, thereby causing the quality of face features extracted by a model for face recognition to degrade and thus causing the face recognition performance to decrease. The scenario of recognizing low-quality face images can be a video monitoring scenario, a portable terminal, and the like. SUMMARY

[0003] A brief overview of the present disclosure is presented below in order to provide a basic understanding of some aspects of the present disclosure. It should be appreciated that this overview is not an exhaustive overview of the present disclosure. It is not intended to determine the key or important parts of the present disclosure, nor is it intended to limit the scope of the present disclosure. Its purpose is merely to give some concepts in a simplified form as a prelude to the more detailed description discussed later.

[0004] According to one aspect of the present invention, there is provided a method of training a neural network, the neural network comprising a first model and a second model, the method comprising the steps of: obtaining a second image from a first image, the second image having a lower quality than the first image; inputting the first image into the first model of the neural network and inputting the second image into the second model of the neural network; computing an attention map and a gradient map for each of the first model and the second model; constructing a loss function based on a matrix of a dot product of the gradient map and the attention map of the first model and a matrix of a dot product of the gradient map and the attention map of the second model; and training the neural network by minimizing the loss function.

[0005] Preferably, the loss function is a square of a difference between the matrix of the dot product of the gradient map and the attention map of the first model and the matrix of the dot product of the gradient map and the attention map of the second model.

[0006] Preferably, the method further comprises softening the attention map for each of the first model and the second model after computing the attention map, and the loss function is a square of a difference between a matrix of a dot product of the gradient map and the softened attention map of the first model and a matrix of a dot product of the gradient map and the softened attention map of the second model.

[0007] Preferably, the first model and the second model are two symmetric branches of the neural network, and each of the first model and the second model comprises one or more convolutional layers and one or more fully connected layers.

[0008] Preferably, the method further comprises training the neural network using the constructed loss function while also using a knowledge distillation loss function and a classification loss function.

[0009] Preferably, the matrix of the dot product of the gradient map and the attention map is a Gram matrix.

[0010] Preferably, the first image and the second image comprise a human face.

[0011] According to another aspect of the present application, there is provided an image recognition method, which performs image recognition by inputting an image to be recognized into a second model of a neural network trained by the method of training a neural network according to the present application.

[0012] According to yet another aspect of the present application, there is provided an apparatus for training a neural network, the neural network comprising a first model and a second model, the apparatus comprising: an obtaining device configured to obtain a second image from a first image, the second image having a lower quality than the first image; an input device configured to input the first image into the first model of the neural network and to input the second image into the second model of the neural network; a calculating device configured to calculate an attention map and a gradient map for each of the first model and the second model; and a constructing device configured to construct a loss function based on a matrix of the dot product of the gradient map and the attention map of the first model and a matrix of the dot product of the gradient map and the attention map of the second model, wherein the neural network is trained by minimizing the loss function.

[0013] According to other aspects of the present application, there are also provided corresponding computer program codes, computer readable storage media and computer program products.

[0014] By the method and apparatus for training a neural network according to the present application, it is possible to improve the recognition performance for low-quality images.

[0015] These and other aspects of the present application will become apparent from the following detailed description of preferred embodiments thereof, taken in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0016] To further illustrate and describe the above and other advantages and features of the present disclosure, a detailed description of specific embodiments of the present disclosure will be rendered by reference to the appended drawings, which are depicted with reference to the following detailed description. Like reference numerals in the drawings and the description below indicate like elements. It should be understood that the drawings and detailed description are not intended to limit the scope of the present disclosure. In the drawings:

[0017] Figure 1 A neural network architecture capable of implementing the solution of the present application is schematically shown.

[0018] Figure 2 is a flowchart of a method for training a neural network according to an embodiment of the present application;

[0019] Figure 3 a comparison of the texture information of the attention map and the gradient map of the high-quality image and the low-quality image is schematically shown;

[0020] Figure 4 is a block diagram of a device for training a neural network according to an embodiment of the present application;

[0021] Figure 5 is a block diagram of an exemplary structure of a general-purpose personal computer in which the method and / or device according to an embodiment of the present application can be implemented. DETAILED DESCRIPTION

[0022] Exemplary embodiments of the present disclosure will be described hereinafter with reference to the accompanying drawings. In the description, all specific details imparted herein are intended to be examples that do not limit the present disclosure. Thus, it should be understood that various embodiments of the disclosure can be practiced with modification and alteration, and that the disclosure be limited only by the scope of the claims. In addition, it should be understood that the drawings are not to scale, emphasis has been placed on illustrating the principles of the present disclosure and the preferred embodiments, and the disclosure should not be limited by the specific illustrated embodiments.

[0023] It is also to be noted that, in order to avoid obscuring the present disclosure due to unnecessary details, only the device structures and / or processing steps closely related to the scheme according to the present disclosure are shown in the drawings, and other details less related to the present disclosure are omitted.

[0024] As described above, at present, the recognition performance of low-quality images, especially human face images, is low. By analyzing the low-quality recognition model and the high-quality recognition model and the features extracted therefrom, it is found that in the existing deep learning model, the attention map and the gradient map learned by the high-quality image are better than those learned by the low-quality image. In particular, the gradient map texture of the high-quality image is clearer.

[0025] Therefore, in order to solve the problems in the prior art, the present application proposes an effective low-quality image, especially human face image, recognition method based on attention map and gradient map.

[0026] First, a two-branch neural network model is constructed to extract features from high-quality images and low-quality images, respectively. Then, a gradient texture-guided learning method based on softened attention is proposed, enabling the low-quality model to learn information from the attention map and gradient map of the high-quality model, making them as consistent as possible, thereby improving the recognition performance of low-quality images.

[0027] Figure 1 A neural network architecture capable of implementing the present invention is illustrated schematically. For example... Figure 1 As shown, the neural network 100 includes two symmetrical branches: a low-quality model 3 and a high-quality model 4. Each branch includes an N-level convolutional layer C and two fully connected layers FC1 and FC2. The high-quality image 1 is input into the high-quality model 4, while the low-quality image 2, converted from the high-quality image 1, is input into the low-quality model 3.

[0028] The following is combined Figure 1 and Figure 2 Hereinafter, we will describe a method 200 for training a neural network according to an embodiment of the present invention.

[0029] First, in step 201, a second image is obtained from the first image, the second image having a lower quality than the first image. Specifically, in this embodiment, the first image is, for example, Figure 1 The high-quality image 1 is shown in the image, while the second image is, for example, a high-quality image 1. Figure 1 Low-quality image 2.

[0030] It should be understood that any suitable existing technical method can be used to degrade the quality of the high-quality image 1 to obtain the low-quality image 2. Further details of this invention are omitted here.

[0031] Next, in step 202, the first image is input into the first model of the neural network, and the second image is input into the second model of the neural network. Specifically, in this embodiment, the first model is, for example, a... Figure 1 The high-quality model 4 in the middle, while the second model is, for example, Figure 1 Low-quality model 3 in the middle.

[0032] Next, in step 203, the attention maps and gradient maps of the first and second models are calculated respectively. Specifically, in this embodiment, the attention maps 6 and gradient maps of the high-quality model 4 and the low-quality model 3 are calculated respectively. Figure 5 .

[0033] It should be understood that any suitable existing technical method can be used to compute the attention map and gradient map. For example, the attention map can be computed in the following ways.

[0034] First, the importance of each point in the feature maps of the high-quality model 4 and the low-quality model 3 is calculated using the following equation (1):

[0035]

[0036] wherein, represents the importance of each feature map k for the target class c, is the gradient value for the target class c, and is the gradient of each feature map, wherein i, j, k are the width, height, and channel number of the feature map, respectively.

[0037] Second, the attention map of the high-quality model 4 and the low-quality model 3 is calculated using the following equation (2):

[0038]

[0039] wherein, ReLU represents a rectified linear unit, and A k represents the feature map, wherein k represents the channel number. It should be understood that the rectified linear unit is a commonly used activation function in artificial neural networks.

[0040] Preferably, in step 203’, the attention map of the first model and the second model is softened. Specifically, in the present embodiment, the attention map of the high-quality model 4 and the low-quality model 3 is softened, respectively. For example, the attention map can be softened using the following equation (3):

[0041]

[0042] wherein, T is a temperature value representing a softening parameter, which can take a value from 0 to positive infinity. Preferably, in the present application, T can take a value from 1 to 10, preferably 5. It should be noted that when T takes a value of positive infinity, the above equation (3) degenerates to the original texture guided (TG) loss function, while when T takes a value of 1, the attention map is not softened.

[0043] It should be understood that the above-described attention map softening equation is only an example. Any suitable prior art method can be used to soften the attention map.

[0044] Next, in step 204, a loss function is constructed based on the matrix of the dot product of the gradient map of the first model and the (softened) attention map and the matrix of the dot product of the gradient map of the second model and the (softened) attention map. Specifically, in the present embodiment, the SA-TG loss function is constructed based on the matrix of the dot product of the gradient map of the high-quality model 4 and the (softened) attention map.

[0045] It should be noted that the loss function can be constructed using, for example, a Gram matrix, but the present application is not limited thereto, but any suitable matrix can be used to construct the loss function. For example, a texture-guided (SA-TG) loss function for the soft attention mechanism can be constructed using the following equation (4):

[0046]

[0047] where f H '(x i ) is the gradient map of the high-quality image, f L '(x i ') is the gradient map of the low-quality image, and G() represents the Gram matrix.

[0048] Finally, in step 205, the neural network is trained by minimizing the loss function. Specifically, in the present embodiment, the neural network 100 is trained by minimizing the SA-TG loss function as in equation (4) above.

[0049] An example of an algorithm for training the neural network using the SA-TG loss function is given below. It should be understood that the method 200 of training the neural network according to the embodiment of the present application is not limited to the following algorithm, but any suitable algorithm can be used to implement it.

[0050]

[0051]

[0052] Preferably, as shown in Figure 1 , the neural network 100 can also be trained simultaneously using, for example, an identification (ID) loss function and a knowledge distillation (KD) loss function. Training the neural network using the ID loss function and the KD loss function is known in the prior art, and the present application will not be described here.

[0053] Figure 3 Soft attention-based gradient maps of low-quality images and high-quality images are shown. From Figure 3 it can be seen that there is a clear difference in the texture information of the attention maps and gradient maps of the high-quality images and the low-quality images. This difference can be made up by the method 200 of training the neural network described above, i.e. by using the SA-TG loss function to enable the low-quality model to learn the information of the high-quality model, in particular the texture information.

[0054] The beneficial effects of the method for training neural networks in this invention are illustrated in Table 1 below. In the experiments in Table 1, ResNet34 was used as the neural network model, trained using the WebFace training set, and tested using the IVS ID card face recognition test set. To simulate low-quality face recognition, JPEG compression was randomly added to the WebFace training set to simulate low-quality images. For the test set, two cases were considered: one where the ID card photo was JPEG compressed while the on-site photo was not compressed, and another where both the ID card photo and the on-site photo were compressed.

[0055]

[0056] Table 1

[0057] As can be seen from Table 1, compared with the mainstream KD loss function, the SA-TG loss function of this invention can greatly improve the performance of low-quality face recognition.

[0058] The above combination Figure 1 and Figure 3 A method for training a neural network according to an embodiment of the present invention has been described in detail. As can be seen from the above description, the recognition performance for low-quality images, especially face images, has been improved by the method according to each embodiment.

[0059] The methods discussed above can be implemented entirely by a computer-executable program, or partially or entirely using hardware and / or firmware. When implemented in hardware and / or firmware, or when a computer-executable program is loaded into a hardware device capable of running the program, a device for processing transactions, as described below, is implemented. Hereinafter, an overview of these devices is given without repeating some details already discussed above; however, it should be noted that while these devices can perform the methods described above, the methods may not necessarily employ or be performed by those components of the described device.

[0060] Figure 4 An apparatus 400 for training a neural network according to one embodiment is shown, comprising an acquisition device 401, an input device 402, a computing device 403, and a construction device 404. The acquisition device 401 is used to acquire a second image from a first image, the second image being of lower quality than the first image. The input device 402 is used to input the first image into a first model of the neural network and the second image into a second model of the neural network. The computing device 403 is used to compute the attention map and gradient map of each of the first and second models. The construction device 404 is used to construct a loss function based on the matrix of the dot product of the gradient map and attention map of the first model and the matrix of the dot product of the gradient map and attention map of the second model, wherein the neural network is trained by minimizing this loss function.

[0061] Preferably, the loss function is the squared difference between the matrix of the dot product of the gradient map and attention map of the first model and the matrix of the dot product of the gradient map and attention map of the second model.

[0062] Preferably, the device 400 further includes a softening device configured to soften the attention maps of the first model and the second model after the attention maps are computed, wherein the loss function is the square of the difference between the matrix of the dot product of the gradient map of the first model and the softened attention map and the matrix of the dot product of the gradient map of the second model and the softened attention map.

[0063] Preferably, the first model and the second model are two symmetrical branches of a neural network, and each of the first model and the second model includes one or more convolutional layers and one or more fully connected layers.

[0064] Figure 4 The device 400 shown for training the neural network corresponds to Figure 2 The method 200 for training a neural network is shown. Therefore, details regarding the various devices in the apparatus 400 for training a neural network have already been provided in [the document / document / etc.]. Figure 2 The method for training neural networks is described in detail in section 200, and will not be repeated here.

[0065] Each component module and unit in the above-described device can be configured via software, firmware, hardware, or a combination thereof. Specific means or methods of configuration are well known to those skilled in the art and will not be elaborated upon here. When implemented via software or firmware, data can be transferred from a storage medium or network to a computer with a dedicated hardware architecture (e.g., ...). Figure 5 The general-purpose computer 500 shown is equipped with the programs that constitute the software, and when various programs are installed, the computer is able to perform various functions, etc.

[0066] The present invention also proposes an image recognition method, which includes inputting an image to be recognized into a low-quality model in a neural network trained by a method for training a neural network according to an embodiment of the present invention, so as to recognize the image.

[0067] Figure 5 This is a block diagram illustrating an exemplary structure of a general-purpose personal computer in which the methods and / or apparatuses according to embodiments of the present invention can be implemented. For example... Figure 5As shown, a central processing unit (CPU) 501 performs various processes in accordance with a program stored in a read only memory (ROM) 502 or a program loaded from a storage section 508 to a random access memory (RAM) 503. In the RAM 503, data required when the CPU 501 performs various processes and the like is also stored as necessary. The CPU 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output interface 505 is also connected to the bus 504.

[0068] The following components are connected to the input / output interface 505: an input section 506 (including a keyboard, a mouse, and the like), an output section 507 (including a display such as a cathode ray tube (CRT), a liquid crystal display (LCD), and the like, and a speaker, and the like), the storage section 508 (including a hard disk and the like), and a communication section 509 (including a network interface card such as a LAN card, a modem, and the like). The communication section 509 performs communication processing via a network such as the Internet. As necessary, a drive 510 can also be connected to the input / output interface 505. A removable medium 511 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, and the like is mounted on the drive 510 as necessary, so that a computer program read therefrom is installed in the storage section 508 as necessary.

[0069] In a case where the above-described series of processes are implemented by software, a program constituting the software is installed from a network such as the Internet or a storage medium such as the removable medium 511.

[0070] Those skilled in the art will appreciate that such storage media are not limited to the Figure 5 The removable medium 511 shown therein has a program stored therein, which is distributed separately from the apparatus to provide the program to the user. Examples of the removable medium 511 include a magnetic disk (including a floppy disk (registered trademark)), an optical disk (including a compact disc read only memory (CD-ROM) and a digital versatile disk (DVD)), a magneto-optical disk (including a mini disk (MD) (registered trademark)), and a semiconductor memory. Alternatively, the storage medium can be the ROM 502, a hard disk contained in the storage section 508, and the like, which has the program stored therein and is distributed to the user together with the apparatus containing them.

[0071] The present application also proposes corresponding computer program code, a computer program product having machine-readable instruction code stored therein. The instruction code is read and executed by a machine to perform the above-described method according to the embodiments of the present application.

[0072] Correspondingly, a storage medium configured to carry the above-described program product having machine-readable instruction code stored therein is also included in the disclosure of the present application. The storage medium includes, but is not limited to, a floppy disk, an optical disk, a magneto-optical disk, a memory card, a memory stick, and the like.

[0073] Through the above description, the embodiments of the present disclosure provide the following technical solutions, but are not limited thereto.

[0074] Note 1. A method of training a neural network, the neural network comprising a first model and a second model, the method comprising the steps of:

[0075] obtaining a second image from a first image, the second image having a lower quality than the first image;

[0076] inputting the first image into the first model of the neural network and inputting the second image into the second model of the neural network;

[0077] computing an attention map and a gradient map for each of the first model and the second model;

[0078] constructing a loss function based on a matrix of dot products of the gradient map and the attention map of the first model and a matrix of dot products of the gradient map and the attention map of the second model; and

[0079] training the neural network by minimizing the loss function.

[0080] Note 2. The method according to Note 1, wherein the loss function is a square of a difference between the matrix of dot products of the gradient map and the attention map of the first model and the matrix of dot products of the gradient map and the attention map of the second model.

[0081] Note 3. The method according to Note 1, further comprising, after computing the attention map, softening the attention map for each of the first model and the second model.

[0082] Note 4. The method according to Note 3, wherein the loss function is a square of a difference between the matrix of dot products of the gradient map and the softened attention map of the first model and the matrix of dot products of the gradient map and the softened attention map of the second model.

[0083] Note 5. The method according to any one of Notes 1 to 4, wherein the first model and the second model are two symmetric branches of the neural network.

[0084] Note 6. The method according to Note 5, wherein each of the first model and the second model comprises one or more convolutional layers and one or more fully connected layers.

[0085] Note 7. The method according to any one of Notes 1 to 4, wherein the matrix is a Gram matrix.

[0086] Comment 8. The method according to any one of Comments 1 to 4, further comprising training the neural network using the loss function, the knowledge distillation loss function and the classification loss function simultaneously.

[0087] Comment 9. The method according to any one of Comments 1 to 4, wherein the first image and the second image comprise a human face.

[0088] Comment 10. An image recognition method, comprising:

[0089] inputting an image to be recognized into the second model of the neural network trained according to the method of any one of Comments 1 to 9 for recognition.

[0090] Comment 11. An apparatus for training a neural network, the neural network comprising a first model and a second model, the apparatus comprising:

[0091] an obtaining means configured to obtain a second image from a first image, the second image having a lower quality than the first image;

[0092] an inputting means configured to input the first image into the first model of the neural network and to input the second image into the second model of the neural network;

[0093] a computing means configured to compute an attention map and a gradient map for each of the first model and the second model; and

[0094] a building means configured to build a loss function based on a matrix of a dot product of the gradient map and the attention map of the first model and a matrix of a dot product of the gradient map and the attention map of the second model,

[0095] wherein the neural network is trained by minimizing the loss function.

[0096] Comment 12. The apparatus according to Comment 11, wherein the loss function is a square of a difference between the matrix of the dot product of the gradient map and the attention map of the first model and the matrix of the dot product of the gradient map and the attention map of the second model.

[0097] Comment 13. The apparatus according to Comment 11, further comprising a softening means configured to soften the attention map for each of the first model and the second model after computing the attention map.

[0098] Comment 14. The apparatus according to Comment 13, wherein the loss function is a square of a difference between a matrix of a dot product of the gradient map and the softened attention map of the first model and a matrix of a dot product of the gradient map and the softened attention map of the second model.

[0099] Note 15. The device according to any one of Notes 11 to 14, wherein the first model and the second model are two symmetrical branches of the neural network.

[0100] Note 16. The device according to Note 15, wherein the first model and the second model each comprise one or more convolutional layers and one or more fully connected layers.

[0101] Note 17. An apparatus according to any one of Notes 11 to 14, wherein the matrix is ​​a Gram matrix.

[0102] Note 18. The device according to any one of Notes 1 to 4 further includes training the neural network using the loss function, the knowledge distillation loss function, and the classification loss function simultaneously.

[0103] Note 19. The device according to any one of Notes 11 to 14, wherein the first image and the second image include a human face.

[0104] Appendix 20. A computer-readable storage medium storing a program that can be executed by a processor to perform the following operations:

[0105] A second image is obtained from a first image, wherein the quality of the second image is lower than that of the first image;

[0106] The first image is input into the first model of the neural network, and the second image is input into the second model of the neural network;

[0107] Calculate the attention map and gradient map of the first model and the second model respectively;

[0108] The loss function is constructed based on the dot product matrix of the gradient map and attention map of the first model and the dot product matrix of the gradient map and attention map of the second model; and

[0109] The neural network is trained by minimizing the loss function.

[0110] Finally, it should be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Furthermore, unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0111] Although the embodiments of the present application have been described in detail above with reference to the accompanying drawings, it should be understood that the above-described embodiments are merely illustrative of the present application and do not limit the present application. Various modifications and changes can be made by those skilled in the art without departing from the spirit and scope of the present application. Therefore, the scope of the present application should be defined only by the appended claims and their equivalents.

Claims

1. A method of training a neural network, the neural network comprising a first model and a second model, the method comprising the steps of: obtaining a second image from a first image, the second image having a lower quality than the first image; inputting the first image into the first model of the neural network and inputting the second image into the second model of the neural network; computing a first attention map and a first gradient map of the first model, and a second attention map and a second gradient map of the second model; softening the computed first attention map and the computed second attention map; constructing a loss function based on a matrix of a dot product of the first gradient map and the softened first attention map and a square of a difference of a matrix of a dot product of the second gradient map and the softened second attention map; and training the neural network by minimizing the loss function, wherein the softened first attention map is a ratio of a first exponential function of the first attention map and a temperature value representing a softening parameter to a double summation of the first exponential function over a width and a height of a first feature map of the first model, and the softened second attention map is a ratio of a second exponential function of the second attention map and the temperature value representing the softening parameter to a double summation of the second exponential function over a width and a height of a second feature map of the second model. The first model and the second model are two symmetric branches of the neural network. The first model and the second model each comprise one or more convolutional layers and one or more fully connected layers.

4. The method of any one of claims 1 to 3, further comprising training the neural network using the loss function, a knowledge distillation loss function and a classification loss function simultaneously. The matrix is a Gram matrix.

6. An image recognition method, comprising: inputting an image to be recognized into the second model of the neural network trained by the method of any one of claims 1 to 5 to perform recognition.

7. An apparatus for training a neural network, the neural network comprising a first model and a second model, the apparatus comprising: obtaining means configured to obtain a second image from a first image, the second image having a lower quality than the first image; inputting means configured to input the first image into the first model of the neural network and input the second image into the second model of the neural network; computing means configured to compute a first attention map and a first gradient map of the first model, and a second attention map and a second gradient map of the second model; softening means configured to soften the computed first attention map and the computed second attention map; and constructing means configured to construct a loss function based on a matrix of a dot product of the first gradient map and the softened first attention map and a square of a difference of a matrix of a dot product of the second gradient map and the softened second attention map, wherein the neural network is trained by minimizing the loss function, wherein the softened first attention map is a ratio of a first exponential function of the first attention map and a temperature value representing a softening parameter to a double summation of the first exponential function over a width and a height of a first feature map of the first model, and the softened second attention map is a ratio of a second exponential function of the second attention map and the temperature value representing the softening parameter to a double summation of the second exponential function over a width and a height of a second feature map of the second model. ​ 2. The method of claim 1, wherein, ​ 3. The method of claim 2, wherein, ​ ​ 5. The method of any one of claims 1 to 3, wherein, ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ wherein the softened first attention map is a ratio of a first exponential function of the first attention map and a temperature value representing a softening parameter to a ratio of a double summation of the first exponential function over a width and a height of a first feature map of the first model, and the softened second attention map is a ratio of a second exponential function of the second attention map and the temperature value representing the softening parameter to a ratio of a double summation of the second exponential function over a width and a height of a second feature map of the second model.

Images