Multi-visual feature collaborative scalp image generation method

Abstract

The invention provides a multi-visual feature collaborative scalp image generation method, which comprises the following steps of: S100, obtaining a region-level structure feature, a pixel-level detail feature, an overall context relationship feature and a frequency domain feature of a scalp input image through a multi-scale deep learning network, the features are connected to serve as a key vector and a value vector of a cross attention mechanism; and step S200, the features of the scalp input image are obtained through an image encoder, noise is added to complete a diffusion process to obtain a noise image, the noise image is denoised through a denoising neural network under the guidance of a cross attention mechanism to obtain a clear image, and the clear image is a scalp image.

Description

Technical Field

[0001] The present invention relates to an image recognition technology, and in particular to a scalp image generation method using multiple visual features in collaboration. Background Art

[0002] In modern society, due to the stress of work and life, people often suffer from scalp conditions such as seborrheic dermatitis and folliculitis, leading to hair loss. Scalp image recognition can identify scalp conditions and their causes, playing an important role in the treatment of scalp diseases. Patent No. CN119672785A discloses a deep learning-based skin disease classification and recognition system that extracts and identifies skin features through a multi-head self-attention mechanism and feature extraction module. When acquiring features from scalp images and other skin images, image noise may be present, such as scalp motion artifacts, skin texture noise, random noise, and system noise. This noise can affect image feature recognition and disease diagnosis. Summary of the Invention

[0003] The present invention aims to provide a scalp image generation method based on the collaboration of multiple visual features, comprising:

[0004] Step S100: obtaining regional structural features, pixel-level detail features, overall contextual features, and frequency domain features of the scalp input image through a multi-scale deep learning network. The above features are concatenated as the key vector and value vector of the cross-attention mechanism.

[0005] In step S200, the features of the scalp input image are obtained through an image encoder and noise is added to complete the diffusion process to obtain a noisy image. The noisy image is guided by a cross-attention mechanism and denoised by a denoising neural network to obtain a clear image. The clear image is the scalp image.

[0006] Furthermore, the method for obtaining each feature in step S100 is:

[0007] (1) Obtain regional structural features F through void space pyramid pooling RS ,

[0008] F RS =ASSP(I)

[0009] Where ASSP() represents the atrous spatial pyramid pooling process, and I is the scalp input image;

[0010] (2) Obtain pixel-level detail features F through small kernel convolution and residual connection PD ,

[0011] F PD =ResidualBlock(Conv3x3(I))

[0012] Among them, ResidualBlock() represents the residual connection process, and Conv3x3() represents the 3x3 convolution process;

[0013] (3) Obtain the overall contextual relationship feature F through average pooling and full connection CR ,

[0014] F CR =FC(GAP(I))

[0015] Among them, FC() represents the fully connected process, and GAP() represents the average pooling process;

[0016] (4) Obtain frequency domain features F through fast Fourier transform and learnable frequency domain filters FD ,

[0017] F FD =FFTLayer(I,learnable_mask=True)

[0018] Among them, FFTLayer represents the fast Fourier transform process, and learnable_mask represents the learnable frequency domain filtering process.

[0019] Furthermore, the features in step S100 are spatially aligned and channel-aligned, where

[0020] In spatial alignment, the branch feature maps of different resolutions are adjusted to the same target resolution through sampling;

[0021] In channel alignment, the number of channels of each branch feature map is adjusted to make them consistent through 1x1 convolution.

[0022] Furthermore, the diffusion process of step S200 is based on the prior distribution q(x t |x t-1 )Add noise to get a noisy image,

[0023]

[0024] Among them, α t is the noise scheduling parameter, ε t is Gaussian noise, I is the identity matrix.

[0025] Furthermore, for the features of the noisy image, position information and feature type are added as input V for denoising.

[0026] Furthermore, the denoising neural network includes a BEIT layer and an LGM layer, wherein the LGM layer is arranged after the BEIT layer. The features obtained after the BEIT layer learns the input V are used as queries for the cross-attention mechanism. The fusion features obtained by weighted summation of the cross-attention mechanism are denoised by the LGM layer and returned to the BEIT layer until the denoising process time step ends.

[0027] Furthermore, the denoising process specifically includes:

[0028] Step S2031: BEIT layer learns the input V to obtain a set of features {v1, v2, ..., v m};

[0029] In step S2032, the set of features is used as the query vector of the cross attention mechanism module, and the similarity calculation is performed with the key vector obtained in step S106 to generate attention weights. The attention weights are used to perform weighted summation on the values ​​to obtain the fused feature representation {v'1,v'2,...,v' m};

[0030] Step S2033, output features {v'1, v'2, ..., v' m Perform denoising through the LGM layer. If t≠0, go to step S2031, and use the denoised output as the input of the BEIT layer; if t=0, the denoising process ends.

[0031] Furthermore, KL divergence is used as the loss function to train the denoising neural network.

[0032] D KL (q(x t |x t-1 )||p θ (x t-1 |x t ,V))

[0033] Among them, p θ (x t-1 |x t ,V)=N(x t-1 ,μ θ (x t ,V),σ(t)I) is the conditional probability distribution of the denoising process, μ θ () and σ(t) represent the mean and variance of the denoising neural network output, θ is the parameter that needs to be trained for the denoising neural network, and I is the unit matrix.

[0034] Compared with the existing technology, the present invention has the following advantages: (1) It establishes regional-level structural features, pixel-level detail features, overall contextual relationships, and frequency domain features, solving the problem that a single image feature cannot identify multiple groups of features on the scalp; (2) It uses image diffusion and denoising to solve the influence of image noise on image generation; (3) It uses a cross-attention mechanism module to fuse various features, thereby enhancing the extraction of scalp lesion features.

[0035] The present invention will be further described below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 Schematic diagram of the method of the present invention.

[0037] Figure 2 Schematic diagram of the multi-scale deep learning network structure. DETAILED DESCRIPTION

[0038] Combine Figure 1 A scalp image generation method that uses multiple visual features in a coordinated manner is proposed. This method is based on a classifier-free image recognition network consisting of an image encoder, a multi-scale deep learning network, a cross-attention mechanism module, a diffusion module, and a denoising neural network. The method performs forward diffusion and reverse denoising processes, improving the image generation effect. The image encoder, diffusion module, and denoising neural network form an image diffusion and denoising branch, in which the image undergoes forward diffusion and reverse denoising, and generates the query required by the cross-attention mechanism module. The multi-scale deep learning network constitutes the guiding branch, and the multi-scale deep learning network uses the multi-scale features of the scalp image as the key and value in the cross-attention mechanism module to calculate the attention weight and output the actual value of each position. The method includes the following steps:

[0039] Step S100: In the guiding branch, the scalp input image is input into the multi-scale deep learning network to obtain multi-scale features of the scalp image. The multi-scale features are used as the key vector and value vector of the cross-attention mechanism module.

[0040] Step S200: In the image diffusion denoising branch, the clear scalp input image is encoded by an image encoder to obtain image features, noise is injected into the image features in stepwise time steps to obtain a noisy image, and the noisy scalp input image is denoised in stepwise time steps under the guidance of a cross-attention mechanism module to obtain a clear image, and a relationship between the clear image and the noisy image is established; wherein the query vector of the cross-attention mechanism module comes from the image features in the image diffusion denoising branch.

[0041] Combine Figure 2 Scalp image analysis (such as hair loss detection and hair follicle segmentation) needs to capture local details (such as hair texture) and global context (such as scalp area distribution) at the same time. Traditional single-scale features are difficult to take into account both points. In step S100, the multi-scale deep learning network can capture the feature information of the image at different scales. The specific multi-scale deep learning network includes a large-scale branch (Large-Scale Branch), a local branch (Local Branch), a global context branch (Global Context Branch), and a frequency domain branch (Frequency Domain Branch). Among them, the large-scale branch captures regional-level structural features (such as scalp partitioning and hair follicle group distribution), the local branch extracts pixel-level detail features (such as single hair follicle morphology and hair texture), the global context branch constructs the overall context relationship (such as scalp-hairline geometric constraints), and the frequency domain branch analyzes frequency domain features (such as texture periodicity and abnormal high-frequency noise).

[0042] Step S101: Obtain regional level structural features F through dilated spatial pyramid pooling RS ,

[0043] F RS =ASSP(I)

[0044] Where ASSP() represents the atrous spatial pyramid pooling process, and I is the scalp input image;

[0045] Step S102: Obtain pixel-level detail features F through small kernel convolution and residual connection PD ,

[0046] F PD =ResidualBlock(Conv3x3(I))

[0047] Among them, ResidualBlock() represents the residual connection process, and Conv3x3() represents the 3x3 convolution process;

[0048] Step S103: Obtain the overall context relationship F through average pooling and full connection CR ,

[0049] F CR =FC(GAP(I))

[0050] Among them, FC() represents the full connection process, GAP() represents the average pooling process;

[0051] Step S104: Obtain frequency domain features F by using fast Fourier transform and learnable frequency domain filter FD ,

[0052] FFD =FFTLayer(I,learnable_mask=True)

[0053] Among them, FFTLayer represents the fast Fourier transform process, and learnable_mask represents the learnable frequency domain filtering process;

[0054] Step S105: perform spatial alignment and channel alignment on the above four types of features, and perform feature fusion;

[0055] In step S106, the fused features are subjected to key transformation and value transformation to generate the key vector and value vector of the cross-self-attention mechanism module respectively.

[0056] In step S105, in spatial alignment, feature maps of different resolutions are adjusted to the same target resolution by upsampling or downsampling; in channel alignment, the number of channels of each branch feature map is adjusted to be consistent through 1x1 convolution.

[0057] In step S200, during the diffusion process of the image, from t=0 to t=T, starting from the initial state distribution x0 of the clear image x, noise is gradually injected in time steps, and finally the state distribution x of the noisy image y is reached. T ; In the denoising process, a neural network is learned to transform the noise image y from its initial state x to its initial state x in the time period t = T to t = 0. T Initially, denoising is performed gradually within the time step, ultimately reaching the state distribution x0 of the clear image x. KL divergence is used as the loss function to optimize the probability distribution during diffusion and denoising. The specific process is as follows.

[0058] Step S201: During the diffusion process, at each time step t, the noise is distributed according to the following prior distribution q(x t |x t-1 ) to get the noise image,

[0059]

[0060] Among them, α t is the noise scheduling parameter, ε t is Gaussian noise, obey

[0061] Step S202: For the features of the noisy image, add position information and feature type as input V for denoising; adding position information can help the model understand the positional relationship of each pixel or feature in the image, so as to better generate an image with spatial consistency; adding feature type can distinguish different object categories or types of image areas.

[0062] Step S203, construct a denoising neural network of the BEIT layer and the LGM layer, specifically after the LGM layer is set in the BEIT layer, the BEIT layer and the local generation module continuously iterate to perform denoising. The core of BEIT (BERT Pre-Training of Image Transformers) is to divide the image into multiple small patches (patches) and regard them as "visual tokens", and then pre-train to obtain abstract features by masking these visual tokens and predicting their original content. The local generation module (LGM) can effectively remove noise and retain image details by extracting and optimizing local features of the image. The denoising process in this embodiment includes:

[0063] Step S2031: BEIT layer learns the input V to obtain a set of features {v1, v2, ..., v m};

[0064] In step S2032, the set of features is used as the query vector of the cross attention mechanism module, and the similarity calculation is performed with the key vector obtained in step S106 to generate attention weights. The attention weights are used to perform weighted summation on the values ​​to obtain the fused feature representation {v'1,v'2,...,v' m}, where m is the number of features;

[0065] Step S2033, output features {v'1, v'2, ..., v' m Perform denoising through the LGM layer. If t≠0, go to step S2031, and use the denoised output as the input of the BEIT layer; if t=0, the denoising process ends.

[0066] In the denoising process, the conditional probability distribution p θ (x t-1 |x t ) is the key distribution that the model needs to learn. In this embodiment, due to the guidance of the cross-attention mechanism module, the conditional probability distribution is expressed as p θ (x t-1 |x t ,V)

[0067] p θ (x t-1 |x t ,V)=N(x t-1 ,μ θ (x t ,V),σ(t)I)

[0068] Among them, μ θ () and σ(t) represent the mean and variance of the denoising neural network output, θ is the parameter that needs to be trained for the denoising neural network, and I is the unit matrix.

[0069] In the process of training and learning the image diffusion denoising branch, the prior distribution q(x t |x t-1 ) and the conditional probability distribution is expressed as p θ (x t-1 |x t ,V), establish KL divergence as the loss function D KL (q(x t |x t-1 )||p θ (x t-1 |x t ,V)), optimize the loss function and obtain the trained neural network.

Claims

1. A scalp image generation method based on multi-visual feature collaboration, characterized in that: include: Step S100: obtaining regional structural features, pixel-level detail features, overall contextual features, and frequency domain features of the scalp input image through a multi-scale deep learning network. The above features are concatenated as the key vector and value vector of the cross-attention mechanism. In step S200, the features of the scalp input image are obtained through an image encoder and noise is added to complete the diffusion process to obtain a noisy image. The noisy image is guided by a cross-attention mechanism and denoised by a denoising neural network to obtain a clear image. The clear image is the scalp image.

2. The method according to claim 1, characterized in that The method for obtaining each feature in step S100 is: (1) Obtain regional structural features F through void space pyramid pooling RS , F RS =ASSP(I) Where ASSP() represents the atrous spatial pyramid pooling process, and I is the scalp input image; (2) Obtain pixel-level detail features F through small kernel convolution and residual connection PD , F PD =ResidualBlock(Conv3x3(I)) Among them, ResidualBlock() represents the residual connection process, and Conv3x3() represents the 3x3 convolution process; (3) Obtain the overall contextual relationship feature F through average pooling and full connection CR , F CR =FC(GAP(I)) Among them, FC() represents the full connection process, GAP() represents the average pooling process; (4) Obtain frequency domain features F through fast Fourier transform and learnable frequency domain filters FD , F FD =FFTLayer(I,learnable_mask=True) Among them, FFTLayer represents the fast Fourier transform process, and learnable_mask represents the learnable frequency domain filtering process.

3. The method according to claim 2, characterized in that The features in step S100 are spatially aligned and channel-aligned, where In spatial alignment, the branch feature maps of different resolutions are adjusted to the same target resolution through sampling; In channel alignment, the number of channels of each branch feature map is adjusted to make them consistent through 1x1 convolution.

4. The method according to claim 1, wherein The diffusion process of step S200 is based on the prior distribution q(x t |x t-1 )Add noise to get a noisy image, Among them, α t is the noise scheduling parameter, ε t is Gaussian noise, I is the identity matrix.

5. The method according to claim 4, characterized in that For the features of the noisy image, position information and feature type are added as input V for denoising.

6. The method according to claim 5, characterized in that The denoising neural network includes a BEIT layer and an LGM layer, where the LGM layer is set after the BEIT layer. The features obtained after the BEIT layer learns the input V are used as queries for the cross-attention mechanism. The fusion features obtained by weighted summation of the cross-attention mechanism are denoised by the LGM layer and returned to the BEIT layer until the denoising process time step ends.

7. The method according to claim 6, characterized in that The denoising process specifically includes: Step S2031: BEIT layer learns the input V to obtain a set of features {v1, v2, ..., v m }; In step S2032, the set of features is used as the query vector of the cross attention mechanism module, and the similarity calculation is performed with the key vector obtained in step S106 to generate attention weights. The attention weights are used to perform weighted summation on the values ​​to obtain the fused feature representation {v'1,v'2,...,v' m }; Step S2033, output features {v'1, v'2, ..., v' m Perform denoising through the LGM layer. If t≠0, go to step S2031, and use the denoised output as the input of the BEIT layer; if t=0, the denoising process ends.

8. The method according to claim 7, characterized in that Use KL divergence as the loss function to train the denoising neural network. D KL (q(x t |x t-1 )||p θ (x t-1 |x t ,V)) Among them, p θ (x t-1 |x t ,V)=N(x t-1 ,μ θ (x t ,V),σ(t)I) is the conditional probability distribution of the denoising process, μ θ () and σ(t) represent the mean and variance of the denoising neural network output, θ is the parameter that needs to be trained for the denoising neural network, and I is the unit matrix.

Images