Image recognition method and system based on generative adversarial network

Abstract

The invention discloses an image recognition method and system based on a generative adversarial network, and the method comprises the steps: collecting a to-be-recognized initial image, inputting the initial image into an image enhancement model obtained through the training of the generative adversarial network, and obtaining a data enhancement image; inputting the data enhanced image into a corresponding image recognition model to obtain a recognition result about the initial image; wherein the generative adversarial network comprises a generator and a discriminator, the generator comprises an encoder and a decoder, a connection path is constructed between the encoder and the decoder through jump connection, the encoder comprises a convolutional layer, a sparse attention module and a maximum pooling layer which are connected in sequence, and a memory network is arranged between the convolutional layer and the sparse attention module. The sparse attention module is used for reducing unnecessary global calculation and reserving key features, and the memory network is used for providing context information related to input features and performing adaptive fusion. According to the invention, the problem of low image recognition precision in the prior art is solved.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and in particular to an image recognition method and system based on a generative adversarial network. Background Art

[0002] As the wave of digitalization sweeps the globe, image recognition technology, a key branch of artificial intelligence, is widely used in numerous scenarios, including security monitoring, medical diagnosis, and autonomous driving. However, image quality, constrained by image acquisition device performance and environmental factors, has become a key bottleneck hindering improvements in image recognition accuracy.

[0003] On the one hand, image acquisition equipment has hardware limitations. Some low-resolution cameras and older sensors are prone to pixel blur and loss of detail when capturing images. When shooting in low-light environments, portable devices such as mobile phones introduce a large amount of noise, compromising the integrity of the image. On the other hand, complex environmental factors can also severely impact image quality. For example, in low-light indoor environments or at night, image brightness is extremely low, obscuring critical information. In strong direct sunlight or backlighting, images can be overexposed and shadowed, resulting in information loss in certain areas. Inclement weather conditions such as haze and dust can reduce image contrast, resulting in a blurred or hazy effect.

[0004] In the existing technology, although filtering-based methods can remove noise to a certain extent, they are prone to over-smoothing, resulting in the loss of image edges and texture details. Grayscale transformation methods such as histogram equalization can enhance image contrast, but it is difficult to accurately control the degree of enhancement, often resulting in insufficient or over-brightening, and the effect on improving color saturation is limited. Some simple convolutional neural network methods based on deep learning, when processing complex scene images, lack effective modeling of global and local image features. As a result, the generated images have poor color reproduction and detail enhancement effects, suffer from problems such as insufficient color saturation and hue shift, and cannot meet the requirements of high-precision image recognition. Summary of the Invention

[0005] In view of this, the purpose of the present invention is to provide an image recognition method and system based on a generative adversarial network, aiming to solve the problem of low image recognition accuracy in the prior art.

[0006] The embodiment of the present invention is implemented as follows:

[0007] An image recognition method based on a generative adversarial network, the method comprising:

[0008] Collect the initial image to be recognized, and input the initial image into the image enhancement model trained by the generative adversarial network to obtain a data-enhanced image;

[0009] Input the data-enhanced image into the corresponding image recognition model to obtain the corresponding recognition result of the initial image using the data-enhanced image;

[0010] Among them, the generative adversarial network includes a generator and a discriminator. The generator includes an encoder and a decoder. The connection path between the encoder and the decoder is constructed through jump connections. The encoder includes a convolutional layer, a sparse attention module and a maximum pooling layer connected in sequence. A memory network is set between the convolutional layer and the sparse module. The sparse attention module is used to reduce unnecessary global calculations while retaining key features. The memory network is used to provide contextual information related to the input features and perform adaptive fusion. The decoder includes a deconvolution layer and an upsampling layer.

[0011] Furthermore, in the above-mentioned image recognition method based on generative adversarial network, the calculation process of the sparse attention module is:

[0012] Q=W q X, K = W k X, V = W v X;

[0013]

[0014] Among them, W q 、W k 、W v is a learnable weight matrix used to map the input feature X to the query, key, and value spaces respectively, S ik It is a binary matrix used to control which positions need to calculate the attention score. If S ik =1, the attention score between position i and position k will be calculated, otherwise it will be blocked, Q i is the query vector at position i, K k is the query vector at position k, d is the dimension of the query and key, A ij is the attention weight, which indicates the degree of attention of position i to position j, V j is the eigenvector of the j-th position of the value matrix, Y i is the i-th position of the output feature.

[0015] Furthermore, in the above-mentioned image recognition method based on generative adversarial network, the step of the memory network being used to provide contextual information related to the input features and perform adaptive fusion includes:

[0016] Obtain the output features of the convolutional layer output, and extract key information from the output features of the convolutional layer output to generate a memory query vector;

[0017] The similarity score is calculated using the query vector and the memory unit in the memory network, and then converted into attention weight through the softmax function;

[0018] The memory units are weighted and summed according to the calculated attention weights to obtain the corresponding context information, and the gating mechanism is used to control the adaptive fusion of the context information and the output features of the convolutional layer.

[0019] Furthermore, in the above-mentioned image recognition method based on generative adversarial network, the calculation formula of the memory query vector is:

[0020] Q=mean(F,dim=[2,3])∈R B×C ;

[0021] The similarity score is calculated as:

[0022] S=Q·M T ∈R B×N ;

[0023] The calculation formula of context information is:

[0024] C=α·M∈R B×C ;

[0025] Among them, F is the feature output of the convolutional layer, B is the batch size, C is the number of channels, M is the memory unit of the memory network, and N is the number of memory units.

[0026] Furthermore, in the above-mentioned image recognition method based on generative adversarial network, the calculation formula of adaptive fusion is:

[0027] F1=G·F+(1-G)⊙C1∈R B×C×H×W ;

[0028] G=sigmoid(W g [F,C1])∈W B×C×H×W ;

[0029] Among them, F is the feature output by the convolution layer, B is the batch size, C is the number of channels, H and W are the height and width of the output feature respectively, M is the memory unit of the memory network, N is the number of memory units, C1 is to expand the context information to the same spatial dimension as the output feature, W g is the learnable weight matrix.

[0030] Furthermore, the above-mentioned image recognition method based on generative adversarial network further includes:

[0031] A dual attention module is set on the jump connection path, which is used to perform attention calculation on the features output by the encoder and input them into the decoder.

[0032] Furthermore, in the above-mentioned image recognition method based on generative adversarial network, the step of the dual attention module for performing attention calculation on the features output by the encoder includes:

[0033] The dual attention module performs global average pooling and global maximum pooling on the received features to obtain the corresponding global average pooling and global maximum pooling results;

[0034] The global average pooling and global maximum pooling results are input into the shared fully connected layer for feature compression and linear transformation. The corresponding outputs are added together, and the channel attention weight is obtained through the activation function.

[0035] Multiply the channel attention weight and the received feature element by element to obtain the channel attention feature;

[0036] The features after the channel attention mechanism are average pooled and maximum pooled in the channel dimension to obtain the channel average pooling and channel maximum pooling results;

[0037] The channel average pooling and channel maximum pooling results are spliced ​​in the channel dimension, the spliced ​​features are convolved, and the corresponding spatial attention weights are obtained through the activation function. The spatial attention weights are multiplied by the channel attention features to obtain the final features processed by the dual attention module.

[0038] Another object of the present invention is to provide an image recognition system based on a generative adversarial network, the system comprising:

[0039] An acquisition module is used to acquire an initial image to be recognized and input the initial image into an image enhancement model trained by a generative adversarial network to obtain a data-enhanced image;

[0040] The recognition module inputs the data-enhanced image into the corresponding image recognition model to obtain the corresponding recognition result of the initial image using the data-enhanced image;

[0041] Among them, the generative adversarial network includes a generator and a discriminator. The generator includes an encoder and a decoder. The connection path between the encoder and the decoder is constructed through jump connections. The encoder includes a convolutional layer, a sparse attention module and a maximum pooling layer connected in sequence. A memory network is set between the convolutional layer and the sparse module. The sparse attention module is used to reduce unnecessary global calculations while retaining key features. The memory network is used to provide contextual information related to the input features and perform adaptive fusion. The decoder includes a deconvolution layer and an upsampling layer.

[0042] Another object of an embodiment of the present invention is to provide a readable storage medium having a computer program stored thereon, wherein the program implements the steps of the above method when executed by a processor.

[0043] Another object of an embodiment of the present invention is to provide an electronic device, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor implements the steps of the above method when executing the program.

[0044] The present invention collects an initial image to be recognized, inputs the initial image into an image enhancement model trained by a generative adversarial network to obtain a data-enhanced image; inputs the data-enhanced image into a corresponding image recognition model to obtain a corresponding recognition result of the initial image recognized by the data-enhanced image; wherein the generative adversarial network includes a generator and a discriminator, the generator includes an encoder and a decoder, a connection path is constructed between the encoder and the decoder through a jump connection, the encoder includes a convolutional layer, a sparse attention module and a maximum pooling layer connected in sequence, a memory network is provided between the convolutional layer and the sparse module, the sparse attention module is used to reduce unnecessary global calculations while retaining key features, and the memory network is used to provide contextual information related to the input features and perform adaptive fusion. The encoder of the generator extracts image features through the convolutional layer, uses the sparse attention module to reduce unnecessary global calculations and retain key features, and the memory network provides contextual information related to the input features and performs adaptive fusion, which helps to better capture useful information in the image and extract key features as much as possible even in the case of poor image quality. Through the adversarial game between the generator and the discriminator, the generator is forced to continuously learn the distribution characteristics of real high-quality images, thereby generating enhanced images that are closer to real high-quality images. This can, to a certain extent, restore image details, enhance brightness and color saturation, and improve image quality. This solves the problem of low image recognition accuracy in existing technologies. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 A flowchart of image recognition based on a generative adversarial network provided in Example 1 of the present invention;

[0046] Figure 2 This is a structural block diagram of an image recognition system based on a generative adversarial network in the third embodiment of the present invention.

[0047] The following specific embodiments will further illustrate the present invention in conjunction with the above-mentioned drawings. DETAILED DESCRIPTION

[0048] To facilitate understanding of the present invention, the present invention will be described more fully below with reference to the accompanying drawings. The drawings illustrate several embodiments of the present invention. However, the present invention may be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and comprehensive understanding of the present invention.

[0049] It should be noted that when an element is referred to as being "fixed to" another element, it may be directly on the other element or there may be an intermediate element. When an element is referred to as being "connected to" another element, it may be directly connected to the other element or there may be an intermediate element. The terms "vertical," "horizontal," "left," "right," and similar expressions used herein are for illustrative purposes only.

[0050] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this invention pertains. The terms used herein in the specification of the present invention are for the purpose of describing specific embodiments only and are not intended to limit the present invention. The term "and / or" as used herein includes any and all combinations of one or more of the relevant listed types.

[0051] Example 1

[0052] See also Figure 1 , shown is an image recognition method based on a generative adversarial network proposed in the first embodiment of the present invention, and the method includes steps S10 to S12.

[0053] Step S10: Acquire an initial image to be identified, and input the initial image into an image enhancement model trained by a generative adversarial network to obtain a data enhanced image.

[0054] Among them, the original image data to be recognized is first collected, and then these original images are input into an image enhancement model that has been pre-trained through a generative adversarial network. The model will process the input image and output enhanced image data; the generative adversarial network consists of a generator and a discriminator. The generator is responsible for learning the mapping relationship from the original image to the enhanced image, and the discriminator forces the generator to generate more realistic enhancement results through adversarial training. The final output data-enhanced image will have significantly improved visual quality or recognition performance of specific tasks (such as target detection and classification) compared to the original image.

[0055] In step S11 , the data-enhanced image is input into a corresponding image recognition model to obtain a corresponding recognition result of the initial image using the data-enhanced image.

[0056] Among them, the image data enhanced by the generative adversarial network is directly input into the image recognition model that matches the task (such as a convolutional neural network classifier or a target detection framework). The recognition model will automatically extract multi-level semantic information based on the clearer structural features, richer texture details or more optimized contrast in the enhanced image, and output the corresponding prediction results related to the original image through the fully connected layer or detection head.

[0057] Specifically, the generative adversarial network includes a generator and a discriminator. The generator includes an encoder and a decoder. The encoder and decoder are connected via a jump connection. The encoder includes a sequentially connected convolutional layer, a sparse attention module, and a maximum pooling layer. A memory network is set between the convolutional layer and the sparse module. The sparse attention module is used to reduce unnecessary global calculations while retaining key features. The memory network is used to provide contextual information related to the input features and perform adaptive fusion. The decoder includes a deconvolution layer and an upsampling layer. In specific implementation, the entire network is composed of a generator and a discriminator to form an adversarial training framework. The generator adopts the classic encoder-decoder architecture and establishes a cross-layer information highway through jump connections, allowing low-level visual features to be directly passed to the decoding stage to alleviate the problem of information loss.

[0058] The encoder consists of a sequential stack of convolutional layers, sparse attention modules, and max pooling layers. A memory network unit is embedded between the convolutional layers and the sparse attention module. The convolutional layers, sparse attention modules, max pooling layers, and memory network units can be connected as a group or as multiple identical groups. The memory network stores and provides contextual information related to the input features. It learns long-term dependencies and common patterns in the image. When the model processes a feature, the memory network provides relevant prior knowledge, helping it better understand and represent the feature. The sparse attention mechanism assigns weights to different locations in the feature map, allowing the model to focus on key parts of the image. This reduces redundant global computation while precisely focusing on key features, maximizing the preservation of structural information crucial for image reconstruction. This improves image clarity and detail. The memory network, placed before the sparse attention module, selects memory information relevant to the input feature. Memory units with higher attention weights are considered more relevant and important, while those with lower weights are deemphasized. This removes irrelevant or noisy information while retaining features valuable for subsequent processing. These filtered, enhanced, and supplemented features are then fed to the subsequent sparse attention module. These features now contain richer, more valuable information, helping the sparse attention module more accurately calculate attention weights. The synergistic effect of these two methods improves the effectiveness of the attention mechanism, ultimately contributing to higher-quality image generation.

[0059] The decoder side is composed of alternating deconvolution layers and upsampling layers, which is responsible for gradually restoring the compressed feature representation into a high-resolution image. The generator cooperates with the adversarial training pressure of the discriminator, and finally the generator can output data-enhanced images with both realism and detail fidelity.

[0060] For example, the calculation process of the sparse attention module is:

[0061] Q=X q x, K = W k W, V = W v X;

[0062]

[0063] Among them, W q 、W k 、W v is a learnable weight matrix used to map the input feature X to the query, key, and value spaces respectively, S ik It is a binary matrix used to control which positions need to calculate the attention score. If S ik =1, the attention score between position i and position k will be calculated, otherwise it will be blocked, Q i is the query vector at position i, K k is the query vector at position k, d is the dimension of the query and key, A ij is the attention weight, which indicates the degree of attention of position i to position j, V j is the eigenvector of the j-th position of the value matrix, Y i is the i-th position of the output feature.

[0064] Among them, the spatial sparsification of attention calculation is achieved through binary matrices, which significantly reduces the computational complexity of traditional dense attention and improves the efficiency of long sequence processing; the dynamic and learnable shielding mechanism enables the model to automatically focus on the local area most relevant to the current task, avoiding noise interference from irrelevant areas; and combined with query-key dimensional scaling, it effectively alleviates the gradient vanishing problem and enhances training stability.

[0065] In addition, in some optional embodiments of the present invention, the step of the memory network being used to provide context information related to the input features and perform adaptive fusion includes:

[0066] Obtain the output features of the convolutional layer output, and extract key information from the output features of the convolutional layer output to generate a memory query vector;

[0067] The similarity score is calculated using the query vector and the memory unit in the memory network, and then converted into attention weight through the softmax function;

[0068] The memory units are weighted and summed according to the calculated attention weights to obtain the corresponding context information, and the gating mechanism is used to control the adaptive fusion of the context information and the output features of the convolutional layer.

[0069] Among them, the memory network receives the features output by the convolutional layer, extracts key semantic information from it through a feature selection mechanism, and encodes it into a memory query vector; then, it calculates the cosine similarity or dot product similarity between the query vector and the pre-stored memory units in the memory network (usually a learnable parameter matrix) to generate the original similarity score, and then converts these scores into attention weight distribution through the softmax function; based on the attention weight, a weighted sum operation is performed on the memory units to aggregate the context information vector that is most relevant to the current input; finally, a gating mechanism is introduced to dynamically adjust the fusion ratio of context information and original convolution features through a learnable gating coefficient to achieve adaptive feature enhancement.

[0070] Exemplarily, the calculation formula for the memory query vector is:

[0071] Q=mean(F,dim=[2,3])∈R B×C ;

[0072] The similarity score is calculated as:

[0073] S=Q·M T ∈R B×N ;

[0074] The calculation formula of context information is:

[0075] C=α·M∈R B×C ;

[0076] Among them, F is the feature output of the convolutional layer, B is the batch size, C is the number of channels, M is the memory unit of the memory network, and N is the number of memory units.

[0077] The calculation formula for adaptive fusion is:

[0078] F1=G·F+(1-G)⊙C1∈R B×C×H×W ;

[0079] G=sigmoid(W g [F,C1])∈R B×C×H×W ;

[0080] Among them, F is the feature output by the convolution layer, B is the batch size, C is the number of channels, H and W are the height and width of the output feature respectively, M is the memory unit of the memory network, N is the number of memory units, C1 is to expand the context information to the same spatial dimension as the output feature, W g is the learnable weight matrix.

[0081] In summary, the above-mentioned embodiment of the present invention provides an image recognition method based on a generative adversarial network, which collects an initial image to be recognized, inputs the initial image into an image enhancement model trained by a generative adversarial network to obtain a data-enhanced image; and inputs the data-enhanced image into a corresponding image recognition model to obtain a corresponding recognition result of the initial image recognized by the data-enhanced image; wherein the generative adversarial network includes a generator and a discriminator, the generator includes an encoder and a decoder, the encoder and the decoder are connected by a jump connection to construct a connection path, the encoder includes a convolutional layer, a sparse attention module and a maximum pooling layer connected in sequence, a memory network is provided between the convolutional layer and the sparse module, the sparse attention module is used to reduce unnecessary global calculations while retaining key features, and the memory network is used to provide contextual information related to the input features and perform adaptive fusion. The encoder of the generator extracts image features through the convolutional layer, uses the sparse attention module to reduce unnecessary global calculations and retain key features, and the memory network provides contextual information related to the input features and performs adaptive fusion, which helps to better capture useful information in the image and extract key features as much as possible even in the case of poor image quality. Through the adversarial game between the generator and the discriminator, the generator is forced to continuously learn the distribution characteristics of real high-quality images, thereby generating enhanced images that are closer to real high-quality images. This can, to a certain extent, restore image details, enhance brightness and color saturation, and improve image quality. This solves the problem of low image recognition accuracy in existing technologies.

[0082] Example 2

[0083] This embodiment also proposes an image recognition method based on a generative adversarial network. The difference between the image recognition method based on a generative adversarial network in this embodiment and the image recognition method based on a generative adversarial network proposed in Example 1 is that:

[0084] The method further comprises:

[0085] A dual attention module is set on the jump connection path, which is used to perform attention calculation on the features output by the encoder and input them into the decoder.

[0086] Specifically, the dual attention module performs global average pooling and global maximum pooling on the received features to obtain the corresponding global average pooling and global maximum pooling results; the global average pooling and global maximum pooling results are input into the shared fully connected layer for feature compression and linear transformation, and the corresponding outputs are added, and the channel attention weight is obtained through the activation function; the channel attention weight and the received features are multiplied element by element to obtain the channel attention feature; the features after the channel attention mechanism are average pooled and maximum pooled in the channel dimension to obtain the channel average pooling and channel maximum pooling results; the channel average pooling and channel maximum pooling results are spliced ​​in the channel dimension, the spliced ​​features are convolved, and the corresponding spatial attention weight is obtained through the activation function, and the spatial attention weight is multiplied by the channel attention feature to obtain the final feature processed by the dual attention module.

[0087] Among them, the feature transfer process between the encoder and decoder is optimized through a dual attention module: a dual attention module is set up on the jump connection path from the encoder to the decoder. This module uses a series structure of channel attention and spatial attention to refine the input features. First, the module performs global average pooling and global maximum pooling operations on the received feature maps to extract global statistical information in the channel dimension respectively. The two pooling results are then input into a shared fully connected layer network, added after dimensionality reduction and linear transformation, and then activated by a sigmoid function to generate a channel attention weight matrix. This weight matrix is ​​multiplied element-by-element with the original input features to obtain a recalibrated feature representation in the channel dimension.

[0088] Next, the feature map after channel attention processing is average pooled and maximum pooled in the channel dimension to generate statistical features in the spatial dimension. The two pooling results are spliced ​​in the channel dimension and the spatial information is fused through a convolutional layer. Finally, the sigmoid function is used to generate the spatial attention weight matrix; finally, the spatial attention weight is multiplied element-by-element with the channel attention feature to obtain a feature map that is weighted by both channel and spatial dimensions, and then passed to the decoder.

[0089] It can be understood that channel attention enhances the response of important feature channels; spatial attention focuses on key spatial areas and suppresses background noise; the cascade use of dual attention mechanisms realizes the adaptive reshaping of features in the channel-space dimension, ultimately improving the model's ability to correspond to image enhancement.

[0090] In summary, the above-mentioned embodiment of the present invention provides an image recognition method based on a generative adversarial network, which collects an initial image to be recognized, inputs the initial image into an image enhancement model trained by a generative adversarial network to obtain a data-enhanced image; and inputs the data-enhanced image into a corresponding image recognition model to obtain a corresponding recognition result of the initial image recognized by the data-enhanced image; wherein the generative adversarial network includes a generator and a discriminator, the generator includes an encoder and a decoder, the encoder and the decoder are connected by a jump connection to construct a connection path, the encoder includes a convolutional layer, a sparse attention module and a maximum pooling layer connected in sequence, a memory network is provided between the convolutional layer and the sparse module, the sparse attention module is used to reduce unnecessary global calculations while retaining key features, and the memory network is used to provide contextual information related to the input features and perform adaptive fusion. The encoder of the generator extracts image features through the convolutional layer, uses the sparse attention module to reduce unnecessary global calculations and retain key features, and the memory network provides contextual information related to the input features and performs adaptive fusion, which helps to better capture useful information in the image and extract key features as much as possible even in the case of poor image quality. Through the adversarial game between the generator and the discriminator, the generator is forced to continuously learn the distribution characteristics of real high-quality images, thereby generating enhanced images that are closer to real high-quality images. This can, to a certain extent, restore image details, enhance brightness and color saturation, and improve image quality. This solves the problem of low image recognition accuracy in existing technologies.

[0091] Example 3

[0092] See also Figure 2 , shown is an image recognition system based on a generative adversarial network proposed in the third embodiment of the present invention, the system comprising:

[0093] The acquisition module 100 is used to acquire an initial image to be recognized and input the initial image into an image enhancement model trained by a generative adversarial network to obtain a data-enhanced image;

[0094] The recognition module 200 inputs the data-enhanced image into a corresponding image recognition model to obtain a corresponding recognition result of the initial image using the data-enhanced image;

[0095] Among them, the generative adversarial network includes a generator and a discriminator. The generator includes an encoder and a decoder. The connection path between the encoder and the decoder is constructed through jump connections. The encoder includes a convolutional layer, a sparse attention module and a maximum pooling layer connected in sequence. A memory network is set between the convolutional layer and the sparse module. The sparse attention module is used to reduce unnecessary global calculations while retaining key features. The memory network is used to provide contextual information related to the input features and perform adaptive fusion. The decoder includes a deconvolution layer and an upsampling layer.

[0096] The functions or operation steps implemented when the above modules are executed are substantially the same as those in the above method embodiments and will not be repeated here.

[0097] Example 4

[0098] Another aspect of the present invention further provides a readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in any one of the above-mentioned embodiments 1 to 2.

[0099] Example 5

[0100] On the other hand, the present invention also provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the program, the steps of the method described in any one of the above-mentioned embodiments 1 to 2 are implemented.

[0101] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0102] Those skilled in the art will appreciate that the logic and / or steps represented in the flowcharts or otherwise described herein, for example, may be considered as a sequenced list of executable instructions for implementing the logical functions, and may be embodied in any storage medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "storage medium" may be any device that can contain, store, communicate, propagate, or transmit a program for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0103] More specific examples of storage media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk cartridge (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the storage medium may even be paper or other suitable medium on which the program is printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering, or processing in another suitable manner as necessary, and then stored in a computer memory.

[0104] It should be understood that various parts of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement the hardware: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0105] Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "examples," "specific examples," or "some examples" means that a specific feature, structure, material, or characteristic described in conjunction with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0106] The above-described embodiments merely illustrate several implementations of the present invention, and while their descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art would be able to make numerous variations and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

Claims

1. An image recognition method based on generative adversarial network, characterized in that: The method comprises: Collect the initial image to be recognized, and input the initial image into the image enhancement model trained by the generative adversarial network to obtain a data-enhanced image; Input the data-enhanced image into the corresponding image recognition model to obtain the corresponding recognition result of the initial image using the data-enhanced image; Among them, the generative adversarial network includes a generator and a discriminator. The generator includes an encoder and a decoder. The connection path between the encoder and the decoder is constructed through jump connections. The encoder includes a convolutional layer, a sparse attention module and a maximum pooling layer connected in sequence. A memory network is set between the convolutional layer and the sparse module. The sparse attention module is used to reduce unnecessary global calculations while retaining key features. The memory network is used to provide contextual information related to the input features and perform adaptive fusion. The decoder includes a deconvolution layer and an upsampling layer.

2. The image recognition method based on generative adversarial network according to claim 1, characterized in that: The calculation process of the sparse attention module is: Q=W q X,K=W k X,V=W v X; Among them, W q 、W k 、W v is a learnable weight matrix used to map the input feature X to the query, key, and value spaces respectively, S ik It is a binary matrix used to control which positions need to calculate the attention score. If S ik =1, the attention score between position i and position k will be calculated, otherwise it will be blocked, Q i is the query vector at position i, K k is the query vector at position k, d is the dimension of the query and key, A ij is the attention weight, which indicates the degree of attention of position i to position j, V j is the eigenvector of the j-th position of the value matrix, Y i is the i-th position of the output feature.

3. The image recognition method based on generative adversarial network according to claim 2, characterized in that: The steps of the memory network providing context information related to the input features and performing adaptive fusion include: Obtain the output features of the convolutional layer output, and extract key information from the output features of the convolutional layer output to generate a memory query vector; The similarity score is calculated using the query vector and the memory unit in the memory network, and then converted into attention weight through the softmax function; The memory units are weighted and summed according to the calculated attention weights to obtain the corresponding context information, and the gating mechanism is used to control the adaptive fusion of the context information and the output features of the convolutional layer.

4. The image recognition method based on generative adversarial network according to claim 3, characterized in that: The calculation formula of the memory query vector is: Q=mean(F,dim=[2,3])∈R B×C ; The similarity score is calculated as: S=Q·M T ∈R B×N ; The calculation formula of context information is: C=α·M∈R B×C ; Among them, F is the feature output of the convolutional layer, B is the batch size, C is the number of channels, M is the memory unit of the memory network, and N is the number of memory units.

5. The image recognition method based on generative adversarial network according to claim 4, characterized in that: The calculation formula for adaptive fusion is: F1=G·F+(1-G)⊙C1∈R B×C×H×W ; G=sigmoid(W g ·[F,C1])∈R B×C×H×W ; Among them, F is the feature output by the convolution layer, B is the batch size, C is the number of channels, H and W are the height and width of the output feature respectively, M is the memory unit of the memory network, N is the number of memory units, C1 is to expand the context information to the same spatial dimension as the output feature, W g is the learnable weight matrix.

6. The image recognition method based on generative adversarial network according to claim 5, characterized in that: The method further comprises: A dual attention module is set on the jump connection path, which is used to perform attention calculation on the features output by the encoder and input them into the decoder.

7. The image recognition method based on generative adversarial network according to claim 6, characterized in that: The step of the dual attention module performing attention calculation on the features output by the encoder includes: The dual attention module performs global average pooling and global maximum pooling on the received features to obtain the corresponding global average pooling and global maximum pooling results; The global average pooling and global maximum pooling results are input into the shared fully connected layer for feature compression and linear transformation. The corresponding outputs are added together, and the channel attention weight is obtained through the activation function. Multiply the channel attention weight and the received feature element by element to obtain the channel attention feature; The features after the channel attention mechanism are average pooled and maximum pooled in the channel dimension to obtain the channel average pooling and channel maximum pooling results; The channel average pooling and channel maximum pooling results are spliced ​​in the channel dimension, the spliced ​​features are convolved, and the corresponding spatial attention weights are obtained through the activation function. The spatial attention weights are multiplied by the channel attention features to obtain the final features processed by the dual attention module.

8. An image recognition system based on a generative adversarial network, characterized in that: The system comprises: An acquisition module is used to acquire an initial image to be recognized and input the initial image into an image enhancement model trained by a generative adversarial network to obtain a data-enhanced image; The recognition module inputs the data-enhanced image into the corresponding image recognition model to obtain the corresponding recognition result of the initial image using the data-enhanced image; Among them, the generative adversarial network includes a generator and a discriminator. The generator includes an encoder and a decoder. The connection path between the encoder and the decoder is constructed through jump connections. The encoder includes a convolutional layer, a sparse attention module and a maximum pooling layer connected in sequence. A memory network is set between the convolutional layer and the sparse module. The sparse attention module is used to reduce unnecessary global calculations while retaining key features. The memory network is used to provide contextual information related to the input features and perform adaptive fusion. The decoder includes a deconvolution layer and an upsampling layer.

9. A readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

10. An electronic device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the steps of the method according to any one of claims 1 to 7 are implemented when the processor executes the program.

Images