Image processing method and system based on artificial intelligence

Abstract

The invention discloses an image processing method and system based on artificial intelligence. The method comprises the steps of data acquisition, design of a self-adaptive standardized network, design of an enhancement style network and establishment of an image enhancement network. The invention belongs to the technical field of image processing, and particularly relates to an artificial intelligence-based image processing method and system, and the method comprises the steps: carrying out the standardization through adaptive stretching feature distribution by using a trainable percentile maximum value block and a percentile minimum value block, and carrying out the normalization according to the distribution of each feature; an image enhancement network is established based on K + 1 enhancement pattern networks, two adaptive standardized networks and an independent convolutional layer, and reasonable color rendition performance and contrast enhancement are realized while original image color information is reserved based on adaptive standardized network processing and enhancement pattern network processing.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image processing method and system based on artificial intelligence. Background Art

[0002] The image processing method based on artificial intelligence uses artificial intelligence technologies such as machine learning and deep learning to denoise, analyze and understand images, and is thus applied to many fields such as image generation and style transfer, medical image analysis, driverless cars and intelligent security. However, the general image normalization process is largely affected by outliers. Carefully selecting the maximum and minimum percentile values ​​can produce fully enhanced results, but it is difficult to select the best percentile value; general image enhancement models are not sufficient to restore the original color of the image and improve the contrast. Summary of the invention

[0003] In view of the above situation, in order to overcome the defects of the prior art, the present invention provides an image processing method and system based on artificial intelligence. In view of the problem that the general image normalization process is greatly affected by outliers, carefully selecting the maximum and minimum percentiles can produce fully enhanced results, but it is difficult to select the optimal percentile value. The scheme uses trainable maximum and minimum percentile blocks, standardizes by adaptively stretching the feature distribution, and normalizes according to the distribution of each feature; in view of the problem that the general image enhancement model is insufficient to restore the original color of the image and improve the contrast, the scheme establishes an image enhancement network based on K+1 enhanced style networks, two adaptive normalization networks and an independent convolutional layer, and achieves reasonable color restoration performance and contrast enhancement while retaining the original image color information based on adaptive normalization network processing and enhanced style network processing.

[0004] The technical solution adopted by the present invention is as follows: The image processing method based on artificial intelligence provided by the present invention comprises the following steps: Step S1: data collection;

[0005] Step S2: designing an adaptive normalization network;

[0006] Step S3: design an enhanced style network;

[0007] Step S4: Establish an image enhancement network.

[0008] Furthermore, in step S1, the data acquisition is to acquire image data, and perform resampling and denoising processing on the acquired image data.

[0009] Further, in step S2, the design of the adaptive normalization network is to introduce an adaptive normalization network using two SE blocks of maximum pooling and minimum pooling; the adaptive normalization network uses trainable percentile maximum and percentile minimum blocks to standardize by stretching the feature distribution; the role of the adaptive normalization network is to use two weights obtained based on the MaSE block and the MiSE block and , adaptively normalize the given input feature image; specifically includes the following steps: Step S21: Maximum pooling, for a given input feature image X C , perform maximum pooling in the MaSE block as follows: ; In the formula, is the maximum value of the feature image obtained based on maximum pooling, i and j are the pixel coordinate indexes of the feature image;

[0010] Step S22: Generate the channel weights of the MaSE block. The MaSE block generates channel weights after passing through a fully connected layer, a ReLU function, another fully connected layer, and a sigmoid function. , as follows: ; In the formula, F ex (·) is the external feature function, W is the weight parameter of the fully connected layer, W1 and W2 are the weight parameters of the two fully connected layers, σ is the sigmoid activation function, g is the ReLU activation function, δ is the derivative of the ReLU activation function, , , and is the channel weight of different channels in the MaSE block; z max is the vector of maximum values; ; and It is the maximum value obtained based on maximum pooling for different input feature images;

[0011] Step S23: Minimum pooling, by applying minimum pooling in the MiSE block, the channel weights of the MiSE block are obtained , as follows: ; ; In the formula, , , and is the channel weight of different channels in MiSE block, z min is the vector of minimum values; ; , and It is the minimum value obtained based on minimum pooling for different input feature images;

[0012] Step S24: Output the feature vector, the formula used is as follows: ; Where N is the output feature vector and X is the input feature image.

[0013] Furthermore, in step S3, the designed enhanced style network consists of a convolution layer, a batch normalization module, and a sigmoid activation function; the number of input channels of the first enhanced style network is 3; the number of input channels of other enhanced style networks is the same as the number of output channels, and its value is C;

[0014] Step S31: Calculate the convolution output. For the input feature X=[x1, x2, ..., x C ], C is the number of input features; and the learned filter kernel set V = [v1, v2, ..., v C' ], C' is the number of filter kernels; the convolution output U=[u1,u2,...,u C' ] obtained by: u C' =Conv(x C )=v C' *X; In the formula, Conv(·) is the convolution function, * is the convolution operation, and v C' is the parameter of the C'th convolution filter, u C' is the C'th output;

[0015] Step S32: Batch normalization, for each output u C' , both perform batch normalization operations to generate feature maps b C' , the process is as follows: ; In the formula, α C' and β C' is the learning parameter for scaling and offsetting the normalized features, b C' (x, y) and u C' (x, y) are b C' and u C' The eigenvalue at the spatial position (x, y), μ(·) is the mean, Var(·) is the variance, and ε is the minimum value to avoid the denominator being zero;

[0016] Step S33: Calculate the enhanced style network output. The output S of the enhanced style network is obtained by the sigmoid activation function. The formula used is as follows: S = σ(B); Where B is the total number of feature maps b C' A set of , σ is the sigmoid activation function.

[0017] Further, in step S4, the established image enhancement network includes K+1 enhanced style networks, two adaptive normalization networks and an independent convolutional layer; when applied to a given image I, the enhanced style network and the adaptive normalization network are combined to generate an enhanced image R, specifically including the following steps: Step S41: Enhanced style network processing. For a given feature X, the enhanced style network processing is as follows: S=f S (X); In the formula, f S (·) is the feature representation of the enhanced style network, S is the normalized feature with 128 channels;

[0018] Step S42: Adaptive normalized network processing. For a given feature X, the adaptive normalized network processing is as follows: N=f N (X); Where N is the normalized feature, f N (·) is the function representation of the adaptive normalized network;

[0019] Step S43: Extract features. For image I, K consecutive enhanced style networks are placed after the first adaptive normalization network to extract features and make the distribution of features consistent. The specific process is as follows: ; In the formula, ∘ is the combination operation, N E is the final feature of the restored enhanced image;

[0020] Step S44: Obtain an enhanced image. In the final stage, an enhanced image R is obtained based on a convolutional layer and a sigmoid activation function. The convolution operation ensures that the restored image has three channels. The specific process is as follows: R = σ(Conv(N E )).

[0021] The artificial intelligence-based image processing system provided by the present invention comprises a data acquisition module, an adaptive standardized network design module, an enhanced style network design module and an image enhancement network establishment module;

[0022] The data acquisition module acquires image data, performs resampling and denoising on the acquired image data, and sends the data to the adaptive standardized network design module;

[0023] The adaptive normalization network design module uses trainable percentile maximum and percentile minimum blocks to normalize by stretching the feature distribution and sends the data to the enhanced style network design module;

[0024] The enhanced style network design module calculates the enhanced style network output based on convolution operation and batch processing, and sends the data to the image enhancement network establishment module;

[0025] The image enhancement network building module includes K+1 enhanced style networks, two adaptive normalization networks and an independent convolutional layer. When applied to a given image I, the enhanced style network and the adaptive normalization network are combined to produce an enhanced image R.

[0026] The beneficial effects achieved by the present invention using the above scheme are as follows: (1) In view of the fact that the general image normalization process is greatly affected by outliers, carefully selecting the maximum and minimum percentiles can produce sufficiently enhanced results, but it is difficult to select the optimal percentile value. In this scheme, trainable maximum and minimum percentile blocks are used to standardize by adaptively stretching the feature distribution and normalizing according to the distribution of each feature.

[0027] (2) To address the problem that general image enhancement models are insufficient to restore the original color of the image and improve the contrast, this scheme establishes an image enhancement network based on K+1 enhanced style networks, two adaptive normalization networks, and an independent convolutional layer. Based on the adaptive normalization network processing and the enhanced style network processing, reasonable color restoration performance and contrast enhancement are achieved while retaining the original image color information. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 A schematic diagram of the process of the image processing method based on artificial intelligence provided by the present invention;

[0029] Figure 2 A schematic diagram of an image processing system based on artificial intelligence provided by the present invention;

[0030] Figure 3 is a schematic flow chart of step S2;

[0031] Figure 4 is a schematic flow chart of step S3;

[0032] Figure 5 It is a schematic diagram of the process of step S4.

[0033] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention. DETAILED DESCRIPTION

[0034] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0035] In the description of the present invention, it should be understood that terms such as “upper”, “lower”, “front”, “back”, “left”, “right”, “top”, “bottom”, “inside” and “outside” indicating directions or positional relationships are based on the directions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore should not be understood as limiting the present invention.

[0036] Example 1, see Figure 1 The present invention provides an artificial intelligence-based image processing method, which comprises the following steps: Step S1: data collection;

[0037] Step S2: designing an adaptive normalization network;

[0038] Step S3: design an enhanced style network;

[0039] Step S4: Establish an image enhancement network.

[0040] Example 2, see Figure 1 This embodiment is based on the above embodiment. In step S1, data acquisition is to acquire image data, and resample and denoise the acquired image data.

[0041] Example 3, see Figure 1 and Figure 3 This embodiment is based on the above embodiment. In step S2, the design of the adaptive normalization network is to introduce an adaptive normalization network using two SE blocks of maximum pooling and minimum pooling; the adaptive normalization network uses trainable percentile maximum and percentile minimum blocks to standardize by stretching the feature distribution; the role of the adaptive normalization network is to use two weights obtained based on the MaSE block and the MiSE block and , adaptively normalize the given input feature image; specifically includes the following steps: Step S21: Maximum pooling, for a given input feature image X C , perform maximum pooling in the MaSE block as follows: ; In the formula, is the maximum value of the feature image obtained based on maximum pooling, i and j are the pixel coordinate indexes of the feature image;

[0042] Step S22: Generate the channel weights of the MaSE block. The MaSE block generates channel weights after passing through a fully connected layer, a ReLU function, another fully connected layer, and a sigmoid function. , as follows: ; In the formula, F ex (·) is the external feature function, W is the weight parameter of the fully connected layer, W1 and W2 are the weight parameters of the two fully connected layers, σ is the sigmoid activation function, g is the ReLU activation function, δ is the derivative of the ReLU activation function, , , and is the channel weight of different channels in the MaSE block; z max is the vector of maximum values; ; and It is the maximum value obtained based on maximum pooling for different input feature images;

[0043] Step S23: Minimum pooling, by applying minimum pooling in the MiSE block, the channel weights of the MiSE block are obtained , as follows: ; ; In the formula, , , and is the channel weight of different channels in MiSE block, z min is the vector of minimum values; ; , and It is the minimum value obtained based on minimum pooling for different input feature images;

[0044] Step S24: Output the feature vector, the formula used is as follows: ; Where N is the output feature vector and X is the input feature image.

[0045] By performing the above operations, the general image normalization process is greatly affected by outliers. Carefully selecting the maximum and minimum percentile values ​​can produce sufficiently enhanced results, but it is difficult to select the optimal percentile values. In this scheme, trainable maximum and minimum percentile blocks are used to standardize by adaptively stretching the feature distribution, and normalizing according to the distribution of each feature.

[0046] Example 4, see Figure 1 and Figure 4 , this embodiment is based on the above embodiment. In step S3, the designed enhanced style network consists of a convolution layer, a batch normalization module and a sigmoid activation function; the number of input channels of the first enhanced style network is 3; the number of input channels of other enhanced style networks is the same as the number of output channels, and its value is C;

[0047] Step S31: Calculate the convolution output. For the input feature X=[x1, x2, ..., x C ], C is the number of input features; and the learned filter kernel set V = [v1, v2, ..., v C' ], C' is the number of filter kernels; the convolution output U=[u1,u2,...,u C' ] obtained by: u C' =Conv(x C )=v C' *X; In the formula, Conv(·) is the convolution function, * is the convolution operation, and v C' is the parameter of the C'th convolution filter, u C' is the C'th output;

[0048] Step S32: Batch normalization, for each output u C' , both perform batch normalization operations to generate feature maps b C' , the process is as follows: ; In the formula, α C' and β C' is the learning parameter for scaling and offsetting the normalized features, b C' (x, y) and u C' (x, y) are b C' and u C' The eigenvalue at the spatial position (x, y), μ(·) is the mean, Var(·) is the variance, and ε is the minimum value to avoid the denominator being zero;

[0049] Step S33: Calculate the enhanced style network output. The output S of the enhanced style network is obtained by the sigmoid activation function. The formula used is as follows: S = σ(B); Where B is the total number of feature maps b C' is a set of , σ is the sigmoid activation function.

[0050] Example 5, see Figure 1 and Figure 5 This embodiment is based on the above embodiment. In step S4, the established image enhancement network includes K+1 enhanced style networks, two adaptive normalization networks and an independent convolutional layer. When applied to a given image I, the enhanced style network and the adaptive normalization network are combined to generate an enhanced image R. Specifically, the following steps are included: Step S41: Enhanced style network processing. For a given feature X, the enhanced style network processing is as follows: S=f S (X); In the formula, f S (·) is the feature representation of the enhanced style network, S is the normalized feature with 128 channels;

[0051] Step S42: Adaptive normalized network processing. For a given feature X, the adaptive normalized network processing is as follows: N=f N (X); Where N is the normalized feature, f N (·) is the function representation of the adaptive normalized network;

[0052] Step S43: Extract features. For image I, K consecutive enhanced style networks are placed after the first adaptive normalization network to extract features and make the distribution of features consistent. The specific process is as follows: ; In the formula, ∘ is the combination operation, N E is the final feature of the restored enhanced image;

[0053] Step S44: Obtain an enhanced image. In the final stage, an enhanced image R is obtained based on a convolutional layer and a sigmoid activation function. The convolution operation ensures that the restored image has three channels. The specific process is as follows: R = σ(Conv(N E )).

[0054] By performing the above operations, in order to address the problem that general image enhancement models are not sufficient to restore the original color of the image and improve the contrast, this scheme establishes an image enhancement network based on K+1 enhanced style networks, two adaptive normalization networks and an independent convolutional layer. Based on adaptive normalization network processing and enhanced style network processing, reasonable color restoration performance and contrast enhancement are achieved while retaining the original image color information.

[0055] Example 6, see Figure 2 , this embodiment is based on the above embodiment, the artificial intelligence-based image processing system provided by the present invention includes a data acquisition module, an adaptive standardized network design module, an enhanced style network design module and an image enhancement network establishment module;

[0056] The data acquisition module acquires image data, performs resampling and denoising on the acquired image data, and sends the data to the adaptive standardized network design module;

[0057] The adaptive normalization network design module uses trainable percentile maximum and percentile minimum blocks to normalize by stretching the feature distribution and sends the data to the enhanced style network design module;

[0058] The enhanced style network design module calculates the enhanced style network output based on convolution operation and batch processing, and sends the data to the image enhancement network establishment module;

[0059] The image enhancement network building module includes K+1 enhanced style networks, two adaptive normalization networks and an independent convolutional layer. When applied to a given image I, the enhanced style network and the adaptive normalization network are combined to produce an enhanced image R.

[0060] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0061] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

[0062] The present invention and its embodiments are described above, and such description is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. In short, if ordinary technicians in the field are inspired by it, without departing from the purpose of the invention, they can design a structure and embodiment similar to the technical solution without creativity, which should belong to the protection scope of the present invention.

Claims

1. An image processing method based on artificial intelligence, characterized in that: The method comprises the following steps: Step S1: data collection; Step S2: designing an adaptive normalization network; Step S3: design an enhanced style network; Step S4: establishing an image enhancement network; In step S2, the design of the adaptive normalization network is to introduce an adaptive normalization network using two SE blocks of maximum pooling and minimum pooling; the adaptive normalization network uses trainable percentile maximum and percentile minimum blocks to standardize by stretching the feature distribution; the role of the adaptive normalization network is to use two weights obtained based on the MaSE block and the MiSE block and , adaptively normalize the given input feature image; specifically includes the following steps: Step S21: Maximum pooling, for a given input feature image X C , perform maximum pooling in the MaSE block as follows: ； In the formula, is the maximum value of the feature image obtained based on maximum pooling, i and j are the pixel coordinate indexes of the feature image; Step S22: Generate the channel weights of the MaSE block. The MaSE block generates channel weights after passing through a fully connected layer, a ReLU function, another fully connected layer, and a sigmoid function. , as follows: ； In the formula, F ex (·) is the external feature function, W is the weight parameter of the fully connected layer, W1 and W2 are the weight parameters of the two fully connected layers, σ is the sigmoid activation function, g is the ReLU activation function, δ is the derivative of the ReLU activation function, , , and is the channel weight of different channels in the MaSE block; z max is the vector of maximum values; ; and It is the maximum value obtained based on maximum pooling for different input feature images; Step S23: Minimum pooling, by applying minimum pooling in the MiSE block, the channel weights of the MiSE block are obtained , as follows: ； ； In the formula, , , and is the channel weight of different channels in MiSE block, z min is the vector of minimum values; ; , and It is the minimum value obtained based on minimum pooling for different input feature images; Step S24: Output the feature vector, the formula used is as follows: ； Where N is the output feature vector and X is the input feature image.

2. The image processing method based on artificial intelligence according to claim 1, characterized in that: In step S3, the designed enhanced style network consists of a convolutional layer, a batch normalization module, and a sigmoid activation function; the number of input channels of the first enhanced style network is 3; the number of input channels of other enhanced style networks is the same as the number of output channels, and its value is C; Step S31: Calculate the convolution output. For the input feature X=[x1, x2, ..., x C ], C is the number of input features; and the learned filter kernel set V = [v1, v2, ..., v C' ], C' is the number of filter kernels; the convolution output U=[u1,u2,...,u C' ] obtained by: u C' =Conv(x C )=v C' *X; In the formula, Conv(·) is the convolution function, * is the convolution operation, and v C' is the parameter of the C'th convolution filter, u C' is the C'th output; Step S32: Batch normalization, for each output u C' , both perform batch normalization operations to generate feature maps b C' , the process is as follows: ； In the formula, α C' and β C' is the learning parameter for scaling and offsetting the normalized features, b C' (x, y) and u C' (x, y) are b C' and u C' The eigenvalue at the spatial position (x, y), μ(·) is the mean, Var(·) is the variance, and ε is the minimum value to avoid the denominator being zero; Step S33: Calculate the enhanced style network output. The output S of the enhanced style network is obtained by the sigmoid activation function. The formula used is as follows: S = σ(B); Where B is the total number of feature maps b C' A set of , σ is the sigmoid activation function.

3. The image processing method based on artificial intelligence according to claim 1, characterized in that: In step S4, the established image enhancement network includes K+1 enhanced style networks, two adaptive normalization networks and an independent convolutional layer; when applied to a given image I, the enhanced style network and the adaptive normalization network are combined to produce an enhanced image R, specifically including the following steps: Step S41: Enhanced style network processing. For a given feature X, the enhanced style network processing is as follows: S=f S (X); In the formula, f S (·) is the feature representation of the enhanced style network, S is the normalized feature with 128 channels; Step S42: Adaptive normalized network processing. For a given feature X, the adaptive normalized network processing is as follows: N=f N (X); Where N is the normalized feature, f N (·) is the function representation of the adaptive normalized network; Step S43: Extract features. For image I, K consecutive enhanced style networks are placed after the first adaptive normalization network to extract features and make the distribution of features consistent. The specific process is as follows: ； In the formula, ∘ is the combination operation, N E is the final feature of the restored enhanced image; Step S44: Obtain an enhanced image. In the final stage, an enhanced image R is obtained based on a convolutional layer and a sigmoid activation function. The convolution operation ensures that the restored image has three channels. The specific process is as follows: R=σ(Conv(N E ))。 4. The image processing method based on artificial intelligence according to claim 1, characterized in that: In step S1, the data acquisition is to acquire image data, and perform resampling and denoising processing on the acquired image data.

5. An artificial intelligence-based image processing system, used to implement the artificial intelligence-based image processing method according to any one of claims 1 to 4, characterized in that: It includes a data acquisition module, an adaptive standardized network design module, an enhanced style network design module and an image enhancement network establishment module.

6. The artificial intelligence-based image processing system according to claim 5, characterized in that: The data acquisition module acquires image data, performs resampling and denoising on the acquired image data, and sends the data to the adaptive standardized network design module; The adaptive normalization network design module uses trainable percentile maximum and percentile minimum blocks to normalize by stretching the feature distribution and sends the data to the enhanced style network design module; The enhanced style network design module calculates the enhanced style network output based on convolution operation and batch processing, and sends the data to the image enhancement network establishment module; The image enhancement network building module includes K+1 enhanced style networks, two adaptive normalization networks and an independent convolutional layer. When applied to a given image I, the enhanced style network and the adaptive normalization network are combined to produce an enhanced image R.

Images