A feature domain optimized small sample image conversion method based on representation enhancement

Abstract

The present invention belongs to the field of image processing technology, and specifically relates to a method for converting small sample images based on feature domain optimization based on representation enhancement. The present invention enhances the image contrast information and edge information by introducing the histogram equalization algorithm and the prior knowledge of the Canny algorithm. On this basis, the network feature extraction capability is improved by the channel attention mechanism based on sub-pixel convolution, and the image representation is enhanced to solve the problem that the subtle features of the image are not prominent. The present invention proposes a conversion mechanism based on the feature domain optimization algorithm, which adaptively divides the feature domain and the content domain by introducing the adversarial idea, and reduces the parameter space by using a variety of rich images in the source domain. The addition of a noise strategy makes the network not limited to the generation of a single sample, and alleviates the problem of mode collapse; according to the characteristics of the source domain and the target domain, a reconstruction strategy is constructed, and the weakened cyclic semantic consistency is used to complete the small sample image conversion task, and a converted image with better visual effect is obtained.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to a feature domain optimization small sample image conversion method based on representation enhancement. Background Art

[0002] As an important carrier of environmental perception and knowledge acquisition, images are of great significance for scientific exploration, research and information mining. The task of image conversion is to use deep learning algorithms to learn good network mapping based on a given data set to generate images that have both the characteristics of the target data domain and the content of the source data domain. Research on efficient and reliable image conversion algorithms has important theoretical value and practical significance. Scholars at home and abroad have conducted in-depth research on image conversion and achieved important results. Among them, the most famous and effective image conversion methods in the existing literature mainly include: 1. Supervised image conversion method based on generative adversarial: In 2016, Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, et al. Image-to-image translation with conditional adversarial networks. Proceedings of the IEEE conference on computer vision and pattern recognition. Honolulu, Hawaii, The United States of America, 2017: 1125-1134. A method of introducing conditional generative adversarial networks to solve large-scale supervised image conversion tasks was proposed. 2. Unsupervised image conversion method based on generative adversarial: In 2017, Jun-Yan Zhu, Taesung Park, Phillip Isola, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks. Proceedings of the IEEE international conference on computer vision. Venice, Italy, 2017: 2223-2232. An unsupervised image conversion method based on cycle semantic consistency was proposed, which, to a certain extent, utilized semantic constraints to meet the requirement of content invariance during image conversion.3. Unsupervised image conversion method based on shared domain: 2017, Ming-Yu Liu, Thomas Breuel, Jan Kautz. Unsupervised image-to-image translation networks. Advances in neural information processing systems. California, The United States of America, 2017: 700-708. A conversion method involving two image domains sharing the same content domain and the only difference is the representation between the domains was proposed, which achieved better conversion effect.

[0003] When image conversion involves small samples and unsupervised domains, supervised methods will be completely untrainable. In addition, solutions such as cycle consistency require a large image data set, which is not applicable to small sample domains. At present, scholars at home and abroad have conducted in-depth research on image conversion algorithms based on small sample domains, mainly including: 1. Small sample image generation method based on data enhancement: In 2018, Hang Gao, Zheng Shou, Alireza Zareian, et al. Low-shot learning via covariance-preserving adversaria laugmentation networks. arXiv preprint arXiv:1810.11730, 2018:1-13. It is proposed to use generative adversarial networks to generate synthetic similar data from large data sets to alleviate the problem of missing data in small sample target domains and obtain better results. 2. Small sample image classification method based on regularized meta-learning: In 2018, Yabin Zhang, Hui Tang, Kui Jia. Fine-grained visual categorization using meta-learning optimization with sample selection of auxiliary data. Proceedings of the european conference on computer vision (ECCV). Munich, Germany, 2018: 233-248. It is proposed to reduce the parameter space by using parameter sharing, that is, sharing the common semantics of the network front end, and using a small sample data set to retrain the network back end and task-related parts to alleviate the data missing problem and obtain better results. 3. Unsupervised image conversion method based on small samples: In 2019, Ming-Yu Liu, Xun Huang, Arun Mallya, et al. Few-shot unsupervised image-to-image translation. Proceedings of the IEEE / CVF International Conference on Computer Vision. Seoul, Korea, 2019: 10551-10560. A method of using source domain images to train small sample image conversion was proposed and good image conversion effects were obtained. Summary of the invention

[0004] The purpose of the present invention is to provide a feature domain optimized small sample image conversion method based on representation enhancement, which can retain more image detail information and generate richer image information.

[0005] A method for small sample image conversion based on feature domain optimization of representation enhancement, comprising the following steps:

[0006] Step 1: Get a small sample image dataset;

[0007] Step 2: Train the image conversion model;

[0008] Step 2.1: Initialize parameters and set the maximum number of iterations;

[0009] Step 2.2: Enhance the image contrast information and edge information;

[0010] Step 2.3: Use the compression and excitation operations of the channel attention mechanism to perform feature processing;

[0011] Step 2.4: Combine sub-pixel convolution to perform upsampling operation on the feature set of the feature-processed image;

[0012] Step 2.5: The conversion mechanism based on the feature domain optimization algorithm completes the conversion of small sample images with rich details;

[0013] Step 2.5.1: Introduce the adversarial idea, adaptively divide the feature domain and content domain, and use the rich multi-category images in the source domain to reduce the parameter space;

[0014] By connecting the feature information to the new discriminator, a new discriminant process is constructed. The idea of ​​the unchanged content of the two conversions is combined in the discriminant process, that is, the image feature information extracted from the converted image is similar to the image feature information extracted from the given sample after re-extraction. In order to avoid the loss being too large due to the simple application of image differences, which will make the loss function unbalanced, the adversarial idea is used to complete this process, and the image feature information extracted twice is fitted to the same feature distribution. The loss function forces the parameters of the generator in the network to be further optimized to complete a better feature extraction process, making the division of the feature domain more accurate, so as to achieve the purpose of optimizing the division of the feature domain and the content domain.

[0015] By adding a new discriminator of category features, the features of the reconstructed image are used as pseudo labels, and the features of the real image are used as real labels to complete the relevant judgment. The added loss function is:

[0016]

[0017] Among them, x represents the input image, l represents the feature category image, D represents the discriminator, and G represents the generator. express expectations;

[0018] Step 2.5.2: Add noise strategy;

[0019] The noise information redundancy of the decoder module in the generator network is introduced to increase random information, specifically:

[0020]

[0021] new vec =Cov(Cov(Cov(concat(co vec ,z),z),z),z)

[0022] Among them, x img represents the input image; vec represents the content vector; concat(·,·) represents the noise addition operation; Cov(·,·) represents the convolution process; represents the extraction process of the content vector; z represents the noise information; new vec Represents the new content vector;

[0023] Step 2.5.3: Based on the characteristics of the source domain and the target domain, a reconstruction strategy is constructed to complete the image conversion task using the weakened cyclic semantic consistency;

[0024] The cycle consistency reconstruction loss is specifically:

[0025]

[0026] in, Indicates weakened cyclic semantic consistency; represents forward loss consistency; represents backward loss consistency;

[0027] The forward loss consistency is specifically:

[0028]

[0029] Among them, Conv(·) represents the feature extraction process; λ1 represents the hyperparameter of the forward conversion and A img , A′ img , B img and C img Respectively represent images; Dec(·,·) represents image decoding based on given image category information and image content information; φ(·) represents the image category extraction process; Represents the process of extracting image content feature information;

[0030] The backward loss consistency is:

[0031]

[0032] Among them, λ2 represents the hyperparameter of the backward conversion and B′ img and C′ img Respectively represent images;

[0033] Step 2.6: Determine whether the maximum number of iterations has been reached. If not, return to step 2.2; otherwise, output the trained image conversion model;

[0034] Step 3: Input the image to be converted into the trained image conversion model to obtain a converted image with better visual effect.

[0035] Furthermore, the method for enhancing the image contrast information and edge information in step 2.2 is specifically as follows:

[0036] Step 2.2.1: Use the histogram equalization algorithm to enhance the contrast information of the image, and the processed image satisfies a uniform probability density distribution;

[0037] Step 2.2.2: Use the grayscale image enhanced by the histogram equalization algorithm as the input of the Canny algorithm and perform Gaussian smoothing; perform gradient calculation on the smoothed image, use non-maximum suppression and double threshold detection to complete the acquisition of edge information features, and add them to the image feature set;

[0038] The smoothed image is:

[0039]

[0040] Where f(x,y) represents the input grayscale image; Represents the convolution process; represents Gaussian function; σ represents standard deviation;

[0041] The gradient of the smoothed image is solved as follows:

[0042]

[0043]

[0044]

[0045]

[0046] Among them, g x represents the gradient in the x direction; g y represents the gradient in the y direction; M(x,y) represents the image gradient; α(x,y) represents the image direction.

[0047] Furthermore, the method of performing feature processing using the compression operation and excitation operation of the channel attention mechanism in step 2.3 is specifically as follows:

[0048] The compression operations are as follows:

[0049]

[0050] Among them, z c Represents the statistic of channel length; H and W are the spatial dimensions of the feature map; u c represents the feature map; F sq Indicates compression operation;

[0051] The excitation operation applies the gating module and the Sigmoid activation function to learn the nonlinear description information between channels, which is formally expressed as:

[0052] s=F ex (z,W)=σ(W2δ(W1z))

[0053] Among them, σ represents the Sigmoid function, represents a variable, and e represents a natural constant; C represents the number of channels; r represents the ratio of dimensionality reduction; δ represents the ReLU function, Indicates variable; F ex Indicates an incentive operation.

[0054] Furthermore, the method of performing upsampling operation on the image feature set after feature processing in combination with sub-pixel convolution in step 2.4 is specifically as follows:

[0055]

[0056] Among them, I out Indicates the output value; I input represents the input value; f represents the network; L represents the Lth layer; b L represents the bias of the Lth layer; represents the convolution operation; W L represents the parameters of the Lth layer; PS(·) represents the pixel adjustment function, which is:

[0057]

[0058] Where T represents the input feature set; represents the rounding down operation; x represents the feature length after expansion; r represents the upsampling multiple; y represents the feature width after expansion; C represents the number of feature channels before expansion; mod(·,·) represents the modulus operation; c represents the number of feature channels after expansion.

[0059] The beneficial effects of the present invention are:

[0060] The present invention enhances the image contrast information and edge information by introducing the prior knowledge of the histogram equalization algorithm and the Canny algorithm. On this basis, the present invention proposes a channel attention mechanism based on sub-pixel convolution to improve the network feature extraction capability, enhance the image representation, solve the problem that the subtle features of the image are not prominent, and is conducive to subsequent conversion tasks. . Due to the limited number of samples, the parameter space optimization is insufficient and a single sample is generated. The present invention proposes a conversion mechanism based on the feature domain optimization algorithm. By introducing the adversarial idea to adaptively divide the feature domain and the content domain, the parameter space is reduced by using the rich images of multiple categories in the source domain. The addition of the noise strategy makes the network not limited to the generation of a single sample, and the mode collapse problem is alleviated. According to the characteristics of the source domain and the target domain, a reconstruction strategy is constructed, and the weakened cyclic semantic consistency is used to complete the small sample image conversion task, and a conversion image with better visual effect is obtained. The present invention has a good conversion effect on the conversion of small sample images, and can retain more image detail information, generate richer conversion images, and has certain effectiveness. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 It is a flow chart of the present invention.

[0062] Figure 2(a) to Figure 2(b) Figure 2 is an example result diagram of a comparative experiment of 10,000 rounds of training on a small sample dataset of underwater sonar. Figure 2(a) is an example result diagram of the original FUNIT network experiment. Figure 2(b) is an example result diagram of the original FUNIT network experiment after contrast information and edge information enhancement.

[0063] Figure 3(a) to Figure 3(b) Figure 3 is an example result diagram of the comparative experiment of the Oxford flower optical small sample image dataset after 50,000 rounds of training. Figure 3(a) is an example result diagram of the experiment of the original FUNIT network. Figure 3(b) is an example result diagram of the experiment of the original FUNIT network after contrast information and edge information enhancement.

[0064] Figure 4 This is a schematic diagram of the network structure of the channel attention mechanism based on sub-pixel convolution proposed in the present invention.

[0065] Figure 5(a) to Figure 5(b)Figure 5(a) is an example result diagram of the comparative experiment of 10,000 rounds of training on the underwater sonar small sample data set (underwater targets with complex shapes). Figure 5(b) is an example result diagram of the original FUNIT network experiment (selecting underwater targets with complex shapes).

[0066] Figure 6(a) to Figure 6(b) Figure 6(a) is an example result of the comparative experiment of the Oxford flower optical small sample image dataset after 57,500 rounds of training (targets with complex shapes). Figure 6(b) is an example result of the original FUNIT network experiment (boxing targets with complex shapes).

[0067] Figure 7 It is a schematic diagram of the network structure that introduces adversarial thinking to adaptively divide the feature domain and content domain.

[0068] Figure 8 This is a schematic diagram of the network structure added by the noise strategy.

[0069] Fig. 9 It is a schematic diagram of the network structure of the image reconstruction strategy.

[0070] Figure 10(a) to Figure 10(b) Figure 10(a) is an experimental example result diagram of 10,000 rounds of training with the adversarial idea introduced into the underwater sonar small sample data set. Figure 10(b) is an experimental example result diagram of the original FUNIT network (selecting an underwater target with a single shape), and Figure 10(a) is an experimental example result diagram of the original FUNIT network after the adversarial idea is introduced.

[0071] Figure 11(a) to Figure 11(b) Figure 11 is a comparative experimental example result diagram of 10,000 rounds of training for the reconstruction strategy of the underwater sonar small sample data set. Figure 11(a) is an experimental example result diagram of the original FUNIT network (selecting an underwater target with a large deformation), and Figure 11(b) is an experimental example result diagram of the original FUNIT network after the reconstruction strategy is constructed.

[0072] Figure 12(a) to Figure 12(b) Figure 12(a) is an example of the comparative experimental result of the Oxford flower optical small sample image dataset after 50,000 rounds of training with the adversarial idea. Figure 12(b) is an example of the experimental result of the original FUNIT network (selecting targets with rich detail information).

[0073] Figure 13(a) to Figure 13(b)Figure 13(a) is a comparative experimental result of the reconstruction strategy constructed by Oxford flower optical small sample image dataset for 50,000 rounds of training. Figure 13(b) is an experimental result of the original FUNIT network (selecting the target with large deformation, i.e., the optical target converted from a single plant to multiple plants). Figure 13(a) is an experimental result of the original FUNIT network after the reconstruction strategy is constructed.

[0074] Fig.14 It is a schematic diagram of the network structure of the feature domain optimization small sample image conversion method based on representation enhancement proposed in the present invention.

[0075] Figure 15(a) to Figure 15(b) Figure 15(a) is a comparative experimental result diagram of the small sample image conversion method based on representation enhancement and optimization in the feature domain proposed in the present invention after 10,000 rounds of training (underwater sonar small sample data set), Figure 15(b) is a comparative experimental result diagram of the small sample image conversion method based on representation enhancement and optimization in the feature domain proposed in the present invention after 10,000 rounds of training (underwater sonar small sample data set), Figure 15(a) is a comparative experimental result diagram of the original FUNIT network (underwater target with fewer texture features selected), and Figure 15(b) is a comparative experimental result diagram of the small sample image conversion method based on representation enhancement and optimization in the feature domain proposed in the present invention.

[0076] Figure 16(a) to Figure 16(b) FIG16( a) is an example result diagram of a comparative experiment of the small sample image conversion method based on representation enhancement and optimized in feature domain after 50,000 training rounds (Oxford flower optical small sample image dataset), FIG16( a) is an example result diagram of an experiment of the original FUNIT network (boxing a target with fewer texture features), and FIG16( b) is an example result diagram of an experiment of the small sample image conversion method based on representation enhancement and optimized in feature domain proposed in the present invention. DETAILED DESCRIPTION

[0077] The present invention is further described below in conjunction with the accompanying drawings.

[0078] The present invention provides a method for converting small sample images based on feature domain optimization based on representation enhancement. The method comprises the following steps: (1) enhancing the image contrast information and edge information according to the characteristics of the small sample images in the data set; (2) proposing a channel attention mechanism based on sub-pixel convolution to improve the network feature extraction capability for representation enhancement; (3) proposing a conversion mechanism based on feature domain optimization algorithm to complete the conversion of small sample images with rich details. In order to achieve better small sample image conversion effect, the present invention proposes a method for converting small sample images based on feature domain optimization based on representation enhancement. That is, in order to solve the problem of blurred contrast information and edge information of small sample images due to the acquisition method, the image contrast information and edge information are enhanced; on this basis, a channel attention mechanism based on sub-pixel convolution is proposed to improve the network feature extraction ability, enhance image representation, and solve the problem of subtle features of the image not being prominent; due to the limited number of samples, it will lead to insufficient optimization of parameter space and the generation of single samples. A conversion mechanism based on the feature domain optimization algorithm is proposed. By introducing the adversarial idea to adaptively divide the feature domain and content domain, the parameter space is reduced by using multiple rich images in the source domain. The addition of the noise strategy makes the network not limited to the generation of a single sample, slowing down the mode collapse problem. According to the characteristics of the source domain and the target domain, a reconstruction strategy is constructed, and the weakened cyclic semantic consistency is used to complete the small sample image conversion task, and a conversion image with better visual effect is obtained. The feature domain optimization small sample image conversion method based on representation enhancement proposed in the present invention has a good conversion effect on small sample image conversion, and can retain more image detail information, generate richer conversion images, and has certain effectiveness.

[0079] A method for small sample image conversion based on feature domain optimization of representation enhancement, comprising the following steps:

[0080] Step 1: Get a small sample image dataset;

[0081] Step 2: Train the image conversion model;

[0082] Step 2.1: Initialize parameters and set the maximum number of iterations;

[0083] Step 2.2: Enhance the image contrast information and edge information;

[0084] Step 2.2.1: Use the histogram equalization algorithm to enhance the contrast information of the image, and the processed image satisfies a uniform probability density distribution;

[0085] Step 2.2.2: Use the grayscale image enhanced by the histogram equalization algorithm as the input of the Canny algorithm and perform Gaussian smoothing; perform gradient calculation on the smoothed image, use non-maximum suppression and double threshold detection to complete the acquisition of edge information features, and add them to the image feature set;

[0086] The smoothed image is:

[0087]

[0088] Where f(x,y) represents the input grayscale image; Represents the convolution process; represents Gaussian function; σ represents standard deviation;

[0089] The gradient of the smoothed image is solved as follows:

[0090]

[0091]

[0092]

[0093]

[0094] Among them, g x represents the gradient in the x direction; g y represents the gradient in the y direction; M(x,y) represents the image gradient; α(x,y) represents the image direction;

[0095] Step 2.3: Use the compression and excitation operations of the channel attention mechanism to perform feature processing;

[0096] The compression operations are as follows:

[0097]

[0098] Among them, z c Represents the statistic of channel length; H and W are the spatial dimensions of the feature map; u c represents the feature map; F sq Indicates compression operation;

[0099] The excitation operation applies the gating module and the Sigmoid activation function to learn the nonlinear description information between channels, which is formally expressed as:

[0100] s=F ex (z,W)=σ(W2δ(W1z))

[0101] Among them, σ represents the Sigmoid function, represents a variable, and e represents a natural constant; C represents the number of channels; r represents the ratio of dimensionality reduction; δ represents the ReLU function, Indicates variable; F ex Indicates incentive operation;

[0102] Step 2.4: Combine sub-pixel convolution to perform upsampling operation on the feature set of the feature-processed image;

[0103]

[0104] Among them, I out Indicates the output value; I input represents the input value; f represents the network; L represents the Lth layer; b L represents the bias of the Lth layer; represents the convolution operation; W L represents the parameters of the Lth layer; PS(·) represents the pixel adjustment function, which is:

[0105]

[0106] Where T represents the input feature set; represents the rounding operation; x represents the feature length after expansion; r represents the upsampling multiple; y represents the feature width after expansion; C represents the number of feature channels before expansion; mod(·,·) represents the modulus operation; c represents the number of feature channels after expansion;

[0107] Step 2.5: The conversion mechanism based on the feature domain optimization algorithm completes the conversion of small sample images with rich details;

[0108] Step 2.5.1: Introduce the adversarial idea, adaptively divide the feature domain and content domain, and use the rich multi-category images in the source domain to reduce the parameter space;

[0109] By connecting the feature information to the new discriminator, a new discriminant process is constructed. The idea of ​​the unchanged content of the two conversions is combined in the discriminant process, that is, the image feature information extracted from the converted image is similar to the image feature information extracted from the given sample after re-extraction. In order to avoid the loss being too large due to the simple application of image differences, which will make the loss function unbalanced, the adversarial idea is used to complete this process, and the image feature information extracted twice is fitted to the same feature distribution. The loss function forces the parameters of the generator in the network to be further optimized to complete a better feature extraction process, making the division of the feature domain more accurate, so as to achieve the purpose of optimizing the division of the feature domain and the content domain.

[0110] By adding a new discriminator of category features, the features of the reconstructed image are used as pseudo labels, and the features of the real image are used as real labels to complete the relevant judgment. The added loss function is:

[0111]

[0112] Among them, x represents the input image, l represents the feature category image, D represents the discriminator, and G represents the generator. express expectations;

[0113] Step 2.5.2: Add noise strategy;

[0114] The noise information redundancy of the decoder module in the generator network is introduced to increase random information, specifically:

[0115]

[0116] new vec =Cov(Cov(Cov(concat(co vec ,z),z),z),z)

[0117] Among them, x img represents the input image; vec represents the content vector; concat(·,·) represents the noise addition operation; Cov(·,·) represents the convolution process; represents the extraction process of the content vector; z represents the noise information; new vec Represents the new content vector;

[0118] Step 2.5.3: Based on the characteristics of the source domain and the target domain, a reconstruction strategy is constructed to complete the image conversion task using the weakened cyclic semantic consistency;

[0119] The cycle consistency reconstruction loss is specifically:

[0120]

[0121] in, Indicates weakened cyclic semantic consistency; represents forward loss consistency; represents backward loss consistency;

[0122] The forward loss consistency is specifically:

[0123]

[0124] Among them, Conv(·) represents the feature extraction process; λ1 represents the hyperparameter of the forward conversion and A img , A′ img , B img and C img Respectively represent images; Dec(·,·) represents image decoding based on given image category information and image content information; φ(·) represents the image category extraction process; Represents the process of extracting image content feature information;

[0125] The backward loss consistency is:

[0126]

[0127] Among them, λ2 represents the hyperparameter of the backward conversion and B′ img and C′ img Respectively represent images;

[0128] Step 2.6: Determine whether the maximum number of iterations has been reached. If not, return to step 2.2; otherwise, output the trained image conversion model;

[0129] Step 3: Input the image to be converted into the trained image conversion model to obtain a converted image with better visual effect.

[0130] Embodiment 1:

[0131] The object of the present invention is to provide a method for converting small sample images by optimizing feature domain based on representation enhancement, which can retain more image detail information and generate richer image information.

[0132] The present invention comprises the following steps in implementation:

[0133] (1) Enhance the image contrast information and edge information: ① Use the histogram equalization algorithm to process the image contrast information; ② Use the Canny algorithm to obtain the image edge information;

[0134] (2) A channel attention mechanism based on sub-pixel convolution is proposed to improve the network feature extraction capability for representation enhancement: ① The compression and excitation operations of the channel attention mechanism are used for feature processing; ② The processed image feature set is upsampled by a factor of 2 in combination with sub-pixel convolution;

[0135] (3) A conversion mechanism based on feature domain optimization algorithm is proposed to complete the conversion of small sample images with rich details: ① By introducing the adversarial idea to adaptively divide the feature domain and content domain, the parameter space is reduced by using the rich multi-category images in the source domain; ② The addition of noise strategy makes the network not limited to the generation of a single sample, which alleviates the problem of mode collapse; ③ According to the characteristics of the source domain and the target domain, a reconstruction strategy is constructed to complete the image conversion task by using the weakened cyclic semantic consistency, thus obtaining a small sample conversion image with better visual effect.

[0136] The present invention may also include:

[0137] 1. The probability density function of the small sample image in step (1) is r represents the converted grayscale image, s represents the output grayscale image, and p r (r) represents the probability density function of the random variable r. The conversion function is w is the pseudo-integral variable, p r (w) represents the probability density function of the random variable w, and L represents the maximum grayscale value.

[0138] 2. In the step (1), the grayscale image enhanced by the histogram equalization algorithm is used as the input of the Canny algorithm, and Gaussian smoothing, gradient solution, non-maximum suppression and double threshold detection are performed to complete the acquisition of edge information features, which are then added to the image feature set.

[0139] 3. In step (2), the channel-wise statistical information is generated through the global pooling operation. After compressing the global spatial information, the network is provided with a compression operation of the overall perspective, which alleviates the limitation of the local perspective. On this basis, the gating module and activation function are introduced to perform the excitation operation by more fully capturing the channel-related dependency information, and the nonlinear description information between channels is learned.

[0140] 4. In step (2), sub-pixel convolution is introduced to perform an upsampling operation on the image feature set with a magnification of 2, specifically: I out Indicates the output value, I input represents the input value, f represents the network, L represents the Lth layer, b L represents the bias of the Lth layer, represents the convolution operation, W L represents the parameters of the Lth layer, and PS(·) represents the adjustment function of the pixel.

[0141] 5. The pixel adjustment function introduced in step (2) is specifically: T represents the input feature set, represents the rounding-down operation, x represents the feature length after expansion, r represents the upsampling multiple, y represents the feature width after expansion, C represents the number of feature channels before expansion, mod(·,·) represents the modulus operation, and c represents the number of feature channels after expansion.

[0142] 6. In step (3), by adding a new discriminator of category features, the features of the reconstructed image are used as pseudo labels and the features of the real image are used as real labels to complete the relevant judgment and add the loss function x represents the input image, l represents the feature category image, D represents the discriminator, and G represents the generator. Express expectations.

[0143] 7. In step (3), random information is added by introducing redundant noise information into the decoder module in the generator network, specifically: and new vec =Cov(Cov(Cov(concat(covec,z),z),z) ,z ), x img represents the input image, co vec represents the content vector, concat(·,·) represents the noise addition operation, Cov(·,·) represents the convolution process, represents the extraction process of the content vector, z represents the noise information, and new vec Represents the new content vector.

[0144] 8. In step (3), the pixel level error of the image is weakened into a feature level error by adding a convolutional layer, specifically: indicates a weakened cyclic semantic consistency, represents the forward loss consistency, It represents the backward loss consistency and realizes the conversion mechanism based on the feature domain optimization algorithm through weakened cyclic semantic consistency to obtain small sample image conversion with rich details.

[0145] 9. Forward loss consistency in step (3) Conv(·) represents the feature extraction process, λ1 represents the hyperparameter of the forward conversion, and A img , A′ img , B img and C img denote images respectively, Dec(·,·) denotes image decoding according to given image category information and image content information, φ(·) denotes the image category extraction process, Represents the process of extracting image content feature information.

[0146] 10. Backward loss consistency in step (3) λ2 represents the hyperparameter of the backward transformation and B′ img and C′ img Represent images respectively.

[0147] Compared with the prior art, the advantages of the present invention are as follows: a. The traditional image conversion method uses a large amount of source domain and target domain data to perform feature extraction to complete the image conversion task. However, when the number of samples is limited, that is, a small sample data set, the traditional image conversion method cannot fully train the image feature extraction process, which affects the effectiveness of feature extraction. In order to make the small sample image conversion have a better conversion effect and retain more image detail information, the present invention proposes a small sample image conversion method based on feature domain optimization based on representation enhancement; b. Small sample images have the problem of blurred contrast information and edge information due to their acquisition methods. The present invention enhances the image contrast information and edge information by introducing the histogram equalization algorithm and the Canny algorithm prior knowledge; c. In the image contrast information and On the basis of enhancing the edge information, the present invention proposes a channel attention mechanism based on sub-pixel convolution to improve the network feature extraction capability, enhance the image representation, solve the problem that the subtle features of the image are not prominent, and is conducive to subsequent conversion tasks; d. Due to the limited number of samples, it will lead to insufficient optimization of the parameter space and the generation of a single sample. The present invention proposes a conversion mechanism based on the feature domain optimization algorithm. By introducing the adversarial idea to adaptively divide the feature domain and the content domain, the parameter space is reduced by using a variety of rich images in the source domain. The addition of a noise strategy makes the network not limited to the generation of a single sample, which alleviates the problem of mode collapse. According to the characteristics of the source domain and the target domain, a reconstruction strategy is constructed, and the weakened cyclic semantic consistency is used to complete the small sample image conversion task and obtain a converted image with better visual effect.

[0148] The feature domain optimization small sample image conversion method based on representation enhancement proposed in the present invention has a good conversion effect on small sample image conversion, can retain more image detail information, and generate richer converted images, which has certain effectiveness.

[0149] Embodiment 2:

[0150] Combination Figure 1 , the specific steps of the present invention are as follows:

[0151] (1) Enhance image contrast and edge information

[0152] The image contrast information enhancement process uses the histogram equalization algorithm, and the probability density function is:

[0153]

[0154] In the formula, r represents the converted grayscale image, s represents the output grayscale image, and p r (r) represents the probability density function of the random variable r.

[0155] The conversion function is:

[0156]

[0157] In the formula, w is the pseudo-integral variable, p r (w) represents the probability density function of the random variable w, and L represents the maximum grayscale value.

[0158] From the above two formulas, we can get:

[0159]

[0160] In the formula, the value range of s is 0≤s≤L-1.

[0161] The Canny algorithm is used to enhance the image edge information. The grayscale image enhanced by the histogram equalization algorithm is used as the input of the Canny algorithm and Gaussian smoothing is performed. The smoothed image is:

[0162]

[0163] Where f(x,y) represents the input grayscale image, represents the convolution process, represents the Gaussian function, and σ represents the standard deviation.

[0164] The gradient of the smoothed image is solved as follows:

[0165]

[0166]

[0167]

[0168]

[0169] In the formula, g x represents the gradient in the x direction, g y represents the gradient in the y direction, M(x,y) represents the image gradient, and α(x,y) represents the image direction.

[0170] The edge information features are acquired by using non-maximum suppression and double threshold detection, and are added to the image feature set.

[0171] Since the current evaluation indicators of image conversion are not perfect, this paper follows the common practice in this research field and compares algorithms based on image visual effects. In the field of small sample image conversion, the conversion effect of the FUNIT network is relatively the best. This paper will use the FUNIT network as the original network for comparative experimental verification.

[0172] In order to verify the effectiveness of the enhancement of contrast information and edge information of the small sample image dataset of the present invention, Figure 2 is a comparative experimental example result diagram of 10,000 rounds of training of the underwater sonar small sample dataset, wherein Figure 2(a) is an example result diagram of the original FUNIT network experiment, and Figure 2(b) is an example result diagram of the original FUNIT network experiment after contrast information and edge information enhancement. By comparing the representative image generation results selected in Figures 2(a) and 2(b), due to the further guidance of the contrast information and edge information on the network gradient, the image after contrast information and edge information enhancement processing is more complete and clear.

[0173] Figure 3 is an example of the comparative experimental results of the Oxford flower optical small sample image dataset after 50,000 rounds of training, where Figure 3(a) is the experimental example result of the original FUNIT network, and Figure 3(b) is the experimental example result of the original FUNIT network after contrast information and edge information enhancement. Although the generated images are random, by comparing the representative image generation results selected in Figure 3(a) and Figure 3(b), the details such as the petal veins in the original FUNIT network after contrast information and edge information enhancement are clearer than those in the original FUNIT network.

[0174] (2) Propose a channel attention mechanism based on sub-pixel convolution to improve the network feature extraction capability for representation enhancement

[0175] ① Compression operation and incentive operation of channel attention mechanism

[0176] The compression operation provides the network with an overall perspective by compressing the global spatial information, that is, generating channel-wise statistical information through the global pooling operation. This statistical information alleviates the limitation that the feature map can only use the local perspective and cannot use network information outside the local perspective, which is conducive to the network to extract features more fully. The compression operation is as follows:

[0177]

[0178] In the formula, z c represents the statistic of channel length, H and W are the spatial dimensions of the feature map, and u c represents the feature map, F sq Indicates a compression operation.

[0179] On this basis, in order to make fuller use of the information extracted after the compression operation, the network performs an excitation operation, which can more fully capture the channel-related dependency information. The excitation operation applies the gating module and the Sigmoid activation function to learn the nonlinear description information between channels. Its formal expression is:

[0180] s=Fex (z,W)=σ(W2δ(W1z))

[0181] In the formula, σ represents the Sigmoid function, C represents the number of channels, r represents the ratio of dimensionality reduction, δ represents the ReLU function, and F ex Indicates an incentive operation.

[0182] The formal representation of the Sigmoid function is:

[0183]

[0184] In the formula, represents a variable, and e represents a natural constant.

[0185] The formal representation of the ReLU function is:

[0186]

[0187] In the formula, Represents a variable.

[0188] ②Introduce sub-pixel convolution to upsample the image feature set

[0189] The present invention introduces sub-pixel convolution to perform an upsampling operation on the image feature set with a magnification of 2, specifically:

[0190]

[0191] In the formula, I out Indicates the output value, I input represents the input value, f represents the network, L represents the Lth layer, b L represents the bias of the Lth layer, represents the convolution operation, W L represents the parameters of the Lth layer, and PS(·) represents the adjustment function of the pixel.

[0192] Introduce pixel adjustment function, specifically:

[0193]

[0194] In the formula, T represents the input feature set, represents the rounding-down operation, x represents the feature length after expansion, r represents the upsampling multiple, y represents the feature width after expansion, C represents the number of feature channels before expansion, mod(·,·) represents the modulus operation, and c represents the number of feature channels after expansion.

[0195] Figure 4This is a schematic diagram of the network structure of the channel attention mechanism based on sub-pixel convolution proposed in the present invention. The cooperation of the channel attention mechanism and the sub-pixel convolution makes the scale of the image constantly change, which brings a certain degree of sparse representation advantage to the network. The sparse representation is used to add a regularization effect to the network to prevent the network from having a memory effect on small sample images and causing overfitting. Therefore, by utilizing the attention to image feature information brought by the channel attention mechanism and the image feature integration brought by sub-pixel convolution and the advantages of partial sparse representation, the network feature extraction capability can be improved, image representation enhancement can be performed, the problem of subtle features of the image not being prominent can be solved, and subsequent conversion tasks can be performed.

[0196] On the basis of Figures 2 and 3, in order to further verify the effectiveness of the channel attention mechanism based on sub-pixel convolution proposed in the present invention to improve the network feature extraction ability for representation enhancement, Figure 5 is an example result diagram of the comparative experiment of 10,000 rounds of underwater sonar small sample data set training (underwater targets with complex shapes), where Figure 5 (a) is an example result diagram of the original FUNIT network experiment (selecting underwater targets with complex shapes), and Figure 5 (b) is an example result diagram of the original FUNIT network experiment after representation enhancement. By comparing the image generation results of the representative complex underwater target images selected in Figures 5 (a) and 5 (b), it can be seen that the image after representation enhancement has a more complete outline and richer target information, but it should be noted that the representation enhancement based on the channel attention mechanism based on sub-pixel convolution requires sufficient training, otherwise it will have a negative impact on the quality of the generated image. Figure 6 is an example result of the comparative experiment of the Oxford flower optical small sample image dataset after 57,500 rounds of training (targets with complex shapes), where Figure 6(a) is an example result of the original FUNIT network experiment (targets with complex shapes selected), and Figure 6(b) is an example result of the original FUNIT network experiment after representation enhancement. Although the generated images are random, by comparing the generation results of the representative complex target images selected in Figure 6(a) and Figure 6(b), it can be seen that the shadow information on the petals of the flowers in the enhanced representation image is clearer, the image is more complete and rich, and the subtle features are more obvious, but it should also be noted that sufficient training is required, otherwise it will still have a negative impact on the quality of the generated images.

[0197] (3) Propose a conversion mechanism based on feature domain optimization algorithm to complete the conversion of small sample images with rich details

[0198] Since the limited number of samples may lead to insufficient optimization of the parameter space and generation of a single sample, the present invention proposes a conversion mechanism based on a feature domain optimization algorithm. By introducing the adversarial idea, the feature domain and the content domain are adaptively divided, and the parameter space is reduced by using a variety of rich images in the source domain. The addition of a noise strategy makes the network not limited to the generation of a single sample, alleviating the problem of mode collapse. According to the characteristics of the source domain and the target domain, a reconstruction strategy is constructed, and the weakened cyclic semantic consistency is used to complete the small sample image conversion task.

[0199] ① Adaptively divide the feature domain and content domain by introducing adversarial thinking

[0200] By connecting the feature information to the new discriminator, a new discrimination process is constructed. The idea of ​​keeping the content of the two conversions unchanged is combined in the discrimination process. That is, the image feature information after re-extraction of the converted image is similar to the image feature information extracted from the given sample. In order to avoid the simple application of image differences resulting in excessive loss and thus imbalance of the loss function, it is proposed to use the adversarial idea to complete this process, fit the image feature information extracted twice to the same feature distribution, and force the parameters of the generator in the network to be further optimized through the loss function to complete a better feature extraction process, so that the division of the feature domain is more accurate, thereby achieving the purpose of optimizing the division of the feature domain and the content domain. Figure 7 This is a schematic diagram of the network structure proposed by the present invention to introduce the adversarial idea to adaptively divide the feature domain and the content domain. The division of the feature domain and the content domain allows the network to use sufficient source domain images to alleviate the problem of limited number of target domain samples, that is, to use the rich multi-category images in the source domain to reduce the parameter space. By adding a new discriminator of category features, the features of the reconstructed image are used as pseudo labels, and the features of the real image are used as real labels to complete the relevant judgment, and the added loss function is:

[0201]

[0202] In the formula, x represents the input image, l represents the feature category image, D represents the discriminator, G represents the generator, Express expectations.

[0203] ②Addition of noise strategy

[0204] The addition of noise strategy is to use the random information contained in the noise to enrich the content information of the image, making the image conversion results more diverse. However, the noise information is directly added as additional information to the network input layer. Under the guidance of the gradient of the loss function, the network often ignores the added noise information after training iterations, making it impossible for the converted image to enhance its performance by using random information. The present invention introduces redundant noise information in the decoder module of the generator network to increase the retention of random information, so that the network is not limited to the generation of a single sample, and the mode collapse problem is alleviated. Figure 8This is a schematic diagram of the network structure added by the noise strategy.

[0205] The formal description of the noise addition strategy is:

[0206]

[0207] new vec =Cov(Cov(Cov(concat(co vec ,z),z),z),z)

[0208] In the formula, x img represents the input image, co vec represents the content vector, concat(·,·) represents the noise addition operation, Cov(·,·) represents the convolution process, represents the extraction process of the content vector, z represents the noise information, and new vec Represents the new content vector.

[0209] ③Build a reconstruction strategy to weaken the cyclic semantic consistency operation

[0210] The traditional cycle consistency loss function requires that the geometric semantics of the original image cannot be changed during the conversion process. In fact, if the original image visual effect is forced to remain unchanged, it will inevitably limit the scope of application of the image conversion process, so that some image domains with large changes in geometric semantics cannot be effectively converted. Therefore, the present invention weakens the pixel-level error of the image into a feature-level error by adding a convolutional layer to take into account the relaxation of geometric constraints and the adaptation of inter-domain content. At the same time, the reconstruction loss commonly used in codecs is used to construct an image reconstruction strategy. Fig. 9 It is a schematic diagram of the network structure of the image reconstruction strategy. The new cycle consistency reconstruction loss is specifically:

[0211]

[0212] In the formula, indicates a weakened cyclic semantic consistency, represents the forward loss consistency, Denotes backward loss consistency.

[0213] According to the characteristics of the source domain and the target domain, a reconstruction strategy is constructed, and the weakened cycle semantic consistency is used to complete the image conversion task and obtain a small sample conversion image with better visual effect.

[0214] The forward loss consistency is:

[0215]

[0216] In the formula, Conv(·) represents the feature extraction process, λ1 represents the hyperparameter of the forward conversion, and A img , A i ' mg , B img and C img denote images respectively, Dec(·,·) denotes image decoding according to given image category information and image content information, φ(·) denotes the image category extraction process, Represents the process of extracting image content feature information.

[0217] The backward loss consistency is:

[0218]

[0219] Where λ2 represents the hyperparameter of the backward conversion and B′ img and C′ img Represent images respectively.

[0220] In order to verify the effectiveness of the proposed conversion mechanism based on the feature domain optimization algorithm, we first experimentally verify the adaptive division of the feature domain and the content domain by introducing the adversarial idea, and the construction of the reconstruction strategy to weaken the cyclic semantic consistency operation. At the same time, since the addition of the noise strategy in the proposed conversion mechanism based on the feature domain optimization algorithm is more about the consideration of image richness, we choose to verify its effectiveness together with the overall image conversion method. Figure 10 is an experimental example result diagram of 10,000 rounds of training with the adversarial idea introduced in the underwater sonar small sample data set, where Figure 10 (a) is an experimental example result diagram of the original FUNIT network (selecting a single underwater target with a single shape), and Figure 10 (b) is an experimental example result diagram of the original FUNIT network after the adversarial idea is introduced. By comparing the image generation results of the representative single underwater target with a single shape selected in Figure 10 (a) and Figure 10 (b), it can be seen that by introducing the adversarial idea, that is, adding a new category discriminator, the category information is easier to extract, and then expressed in the converted image, so that the color distribution of the converted image is closer to the target domain image, and the information of part of the content domain is well preserved. Figure 11 is a comparative experimental example result diagram of 10,000 rounds of training of reconstruction strategies for underwater sonar small sample datasets, where Figure 11(a) is an experimental example result diagram of the original FUNIT network (with a large deformation of the underwater target selected), and Figure 11(b) is an experimental example result diagram of the original FUNIT network after the reconstruction strategy is constructed. By comparing the generation results of the representative large deformation underwater target images selected in Figure 11(a) and Figure 11(b), it can be seen that it is helpful for images with large deformations, while the original network is more like a copy of the target domain image when facing target objects with large differences.

[0221] FIG12 is a comparative experimental example result diagram of the Oxford flower optical small sample image dataset, which introduces the adversarial idea and trains for 50,000 rounds. FIG12(a) is an experimental example result diagram of the original FUNIT network (boxing a target with rich detail information), and FIG12(b) is an experimental example result diagram of the original FUNIT network after introducing the adversarial idea. By comparing the generation results of the representative target image with rich detail information boxed in FIG12(a) and FIG12(b), it can be seen that due to the role of the pseudo-label, the content domain is also relatively completely preserved, and the content of the converted image is more inclined to the target domain image. FIG13 is a comparative experimental example result diagram of the Oxford flower optical small sample image dataset, which builds a reconstruction strategy and trains for 50,000 rounds. FIG13(a) is an experimental example result diagram of the original FUNIT network (boxing a target with large deformation, i.e., an optical target converted from a single plant to multiple plants), and FIG13(b) is an experimental example result diagram of the original FUNIT network after building the reconstruction strategy. By comparing the target image generation results of the representative targets with large deformations, i.e., single plant to multiple plants, selected in Figure 13(a) and Figure 13(b), it can be seen that the original FUNIT network after constructing the reconstruction strategy makes the conversion from single flower to multiple flowers more natural.

[0222] Fig.14 1 is a schematic diagram of the network structure of the feature domain optimization small sample image conversion method based on representation enhancement proposed in the present invention, and FIG15 is a comparative experimental example result diagram of the feature domain optimization small sample image conversion method based on representation enhancement proposed in the present invention after 10,000 rounds of training (underwater sonar small sample data set), wherein FIG15(a) is an experimental example result diagram of the original FUNIT network (underwater target with fewer texture features), and FIG15(b) is an experimental example result diagram of the feature domain optimization small sample image conversion method based on representation enhancement proposed in the present invention. By comparing the image generation results of the underwater target with fewer representative texture features selected in FIG15(a) and FIG15(b), it can be seen that the method proposed in the present invention has a better conversion effect on small sample image conversion, and can retain more image detail information and generate richer converted images.

[0223] FIG16 is a comparative experimental example result diagram of the feature domain optimization small sample image conversion method based on representation enhancement proposed by the present invention after 50,000 rounds of training (Oxford flower optical small sample image dataset), wherein FIG16(a) is an experimental example result diagram of the original FUNIT network (boxing a target with fewer texture features), and FIG16(b) is an experimental example result diagram of the feature domain optimization small sample image conversion method based on representation enhancement proposed by the present invention. By comparing the generated results of the target image with fewer representative texture features boxed in FIG16(a) and FIG16(b), it can be seen that the method proposed by the present invention automatically adds corresponding veins to the petals, thereby improving the richness of the image and generating a richer converted image.

[0224] The experimental results of the small sample image conversion method based on representation enhancement and feature domain optimization proposed in the present invention were analyzed on the underwater sonar small sample image dataset and the Oxford flower optical small sample image dataset, which verified that the proposed method has a good conversion effect on small sample image conversion, can retain more image detail information, and generate richer converted images, which has certain effectiveness.

[0225] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for small sample image conversion based on feature domain optimization based on representation enhancement, characterized in that: The following steps are involved: Step 1: Get a small sample image dataset; Step 2: Train the image conversion model; Step 2.1: Initialize parameters and set the maximum number of iterations; Step 2.2: Enhance the image contrast information and edge information; Step 2.3: Use the compression and excitation operations of the channel attention mechanism to perform feature processing; Step 2.4: Combine sub-pixel convolution to perform upsampling operation on the feature set of the feature-processed image; Step 2.5: The conversion mechanism based on the feature domain optimization algorithm completes the conversion of small sample images with rich details; Step 2.5.1: Introduce the adversarial idea, adaptively divide the feature domain and content domain, and use the rich multi-category images in the source domain to reduce the parameter space; By connecting the feature information to the new discriminator, a new discriminant process is constructed. The idea of ​​keeping the content of the two transformations unchanged is combined in the discriminant process, that is, the image feature information extracted from the transformed image is similar to the image feature information extracted from the given sample. In order to avoid the loss being too large due to the simple application of image differences, which in turn makes the loss function unbalanced, the adversarial idea is used to complete this process, and the image feature information extracted twice is fitted to the same feature distribution. The loss function forces the parameters of the generator in the network to be further optimized to complete a better feature extraction process, making the division of the feature domain more accurate, and achieving the purpose of optimizing the division of the feature domain and the content domain. By adding a new discriminator of category features, the features of the reconstructed image are used as pseudo labels, and the features of the real image are used as real labels to complete the relevant judgment. The added loss function is: Among them, x represents the input image, l represents the feature category image, D represents the discriminator, and G represents the generator. express expectations; Step 2.5.2: Add noise strategy; The noise information redundancy of the decoder module in the generator network is introduced to increase random information, specifically: new vec =Cov(Cov(Cov(concat(co vec ,z),z),z),z) Among them, x img represents the input image; vec represents the content vector; concat(·,·) represents the noise addition operation; Cov(·,·) represents the convolution process; represents the extraction process of the content vector; z represents the noise information; new vec Represents the new content vector; Step 2.5.3: Based on the characteristics of the source domain and the target domain, a reconstruction strategy is constructed to complete the image conversion task using the weakened cyclic semantic consistency; The cycle consistency reconstruction loss is specifically: in, Indicates weakened cyclic semantic consistency; represents forward loss consistency; represents backward loss consistency; The forward loss consistency is specifically: Among them, Conv(·) represents the feature extraction process; λ1 represents the hyperparameter of the forward conversion and A img , A i ' mg , B img and C img Respectively represent images; Dec(·,·) represents image decoding based on given image category information and image content information; φ(·) represents the image category extraction process; Represents the process of extracting image content feature information; The backward loss consistency is: Among them, λ2 represents the hyperparameter of the backward conversion and B′ img and C′ img Respectively represent images; Step 2.6: Determine whether the maximum number of iterations has been reached. If not, return to step 2.2; otherwise, output the trained image conversion model; Step 3: Input the image to be converted into the trained image conversion model to obtain a converted image with better visual effect.

2. The method for small sample image conversion based on feature domain optimization based on representation enhancement according to claim 1, characterized in that: The method for enhancing the image contrast information and edge information in step 2.2 is specifically as follows: Step 2.2.1: Use the histogram equalization algorithm to enhance the contrast information of the image, and the processed image satisfies a uniform probability density distribution; Step 2.2.2: Use the grayscale image enhanced by the histogram equalization algorithm as the input of the Canny algorithm and perform Gaussian smoothing; perform gradient calculation on the smoothed image, use non-maximum suppression and double threshold detection to complete the acquisition of edge information features, and add them to the image feature set; The smoothed image is: Where f(x,y) represents the input grayscale image; Represents the convolution process; represents Gaussian function; σ represents standard deviation; The gradient of the smoothed image is solved as follows: Among them, g x represents the gradient in the x direction; g y represents the gradient in the y direction; M(x,y) represents the image gradient; α(x,y) represents the image direction.

3. The method for small sample image conversion based on feature domain optimization based on representation enhancement according to claim 1, characterized in that: The method for performing feature processing using the compression operation and excitation operation of the channel attention mechanism in step 2.3 is specifically as follows: The compression operations are as follows: Among them, z c Represents the statistic of channel length; H and W are the spatial dimensions of the feature map; u c represents the feature map; F sq Indicates compression operation; The excitation operation applies the gating module and the Sigmoid activation function to learn the nonlinear description information between channels, which is formally expressed as: s=F ex (z,W)=σ(W2δ(W1z)) Among them, σ represents the Sigmoid function, represents a variable, and e represents a natural constant; C represents the number of channels; r represents the ratio of dimensionality reduction; δ represents the ReLU function, Indicates variable; F ex Indicates an incentive operation.

4. The method for small sample image conversion based on feature domain optimization based on representation enhancement according to claim 1, characterized in that: The method of performing upsampling operation on the image feature set after feature processing in combination with sub-pixel convolution in step 2.4 is specifically as follows: Among them, I out Indicates the output value; I input represents the input value; f represents the network; L represents the Lth layer; b L represents the bias of the Lth layer; represents the convolution operation; W L represents the parameters of the Lth layer; PS(·) represents the pixel adjustment function, which is: Where T represents the input feature set; represents the rounding down operation; x represents the feature length after expansion; r represents the upsampling multiple; y represents the feature width after expansion; C represents the number of feature channels before expansion; mod(·,·) represents the modulus operation; c represents the number of feature channels after expansion.

Images