Method for removing couperose skin based on implicit space feature editing

Abstract

The invention discloses a method for removing couperose skin based on implicit space feature editing. The method comprises the steps that an eye region image in an original image is extracted, a couperose skin region mask of the eye region is extracted through a segmentation network, and an eye alignment matrix is calculated according to the obtained preliminary eye region image, the couperose skin region mask and a set image; respectively extracting the aligned eye region image and the hidden space features of the mask; aiming at the hidden space features of the eye region image, editing and adjusting the hidden space features of the eye region image through the hidden space feature information of the mask; and generating a result image based on the hidden space features of the edited eye region image, and aligning the result image to the original image through an inverse matrix of the alignment matrix to obtain a target image. The device can automatically remove telangiectasis of eyes.

Description

Technical Field

[0001] The present invention relates to the technical field of portrait photography post-production, and in particular to a method for removing red blood streaks based on latent space feature editing. Background Art

[0002] In modern society, the widespread use of digital images and videos has made the display of personal image particularly important. Red bloodshot eyes affect people's appearance, especially in high-resolution photos and videos. Therefore, removing red bloodshot eyes is of great significance in the beauty, film and television, advertising and other industries.

[0003] At present, there are three main methods to remove red bloodshot eyes:

[0004] Manual photo editing: Use image editing software (such as Adobe Photoshop) to manually remove red bloodshot. This method relies on the skill level of the photo editor, is time-consuming and labor-intensive, and is not easy to process in batches.

[0005] Filters and automated tools: Some image processing software provides special filters and automated tools to remove red bloodshot. These tools are usually based on color replacement and blurring, but the effect is not precise enough and it is easy to lose image details.

[0006] Traditional algorithms: Traditional algorithms based on image processing, such as color space transformation, threshold segmentation, etc. These methods are usually not accurate enough in locating and removing red blood vessels, and have difficulty in handling complex backgrounds and lighting conditions. Summary of the invention

[0007] In view of this, the purpose of the present invention is to propose a method for removing red blood streaks based on latent space feature editing, which can automatically remove red blood streaks from eyes.

[0008] According to one aspect of the present invention, a method for removing red blood streaks based on latent space feature editing is provided, the method comprising:

[0009] Extract the eye region image from the original image and extract the red blood region mask of the eye region through the segmentation network, and calculate the eye alignment matrix based on the obtained preliminary eye region image and the red blood region mask and a set image;

[0010] Extracting the aligned eye region image and the latent space features of the mask respectively; with respect to the latent space features of the eye region image, editing and adjusting the latent space features of the eye region image through the latent space feature information of the mask;

[0011] A result image is generated based on the latent space features of the edited eye region image, and the result image is aligned to the original image through the inverse matrix of the alignment matrix to obtain a target image.

[0012] In the above technical solution, first, the eye region image and the red blood region mask of the eye region are extracted from the original image by image processing technology. This step is used to identify and extract the eye region and its red blood region. Next, the alignment matrix of the mask is calculated and aligned with the original image. This process involves image alignment technology. For example, the Homography matrix can be used to map points on one image to corresponding points on another image to achieve image alignment. Secondly, the latent space features of the eye region image and the mask are extracted respectively. That is, the space where the latent variables in the generative model are located. Here, the generative adversarial network (GANs) can be used to learn the latent space features of the eye region image and the red blood mask. For the latent space features of the eye region image, the latent space features of the eye region image are edited and adjusted through the latent space feature information of the mask. This process can use, for example, the InterFaceGAN framework, which performs semantic facial editing by interpreting the latent semantics learned by GAN, including the control of attributes such as gender, age, and expression. Here, we can apply this technology to the editing of red blood streaks, and reduce or eliminate them by adjusting the latent space features. Finally, the result image is generated based on the latent space features of the edited eye area image. This step can use, for example, the DreamOmni model, which can generate images of different resolutions based on the latent space features. The result image is aligned to the original image through the alignment matrix to obtain the target image, ensuring that the generated image is consistent with the original image in spatial position. The above method is based on the red blood streak removal method based on latent space feature editing. By extracting and editing the latent space features of the eye area and its red blood streaks, and combining image alignment technology, it can effectively remove or reduce red blood streaks and generate a natural target image that is aligned with the original image.

[0013] In some embodiments, the aligned eye region image and the mask latent space features are extracted respectively; with respect to the latent space features of the eye region image, the latent space features of the eye region image are edited and adjusted through the latent space feature information of the mask, including:

[0014] Performing multi-resolution feature extraction on the eye region image and the mask to obtain a plurality of eye region images and the mask at different resolutions, and extracting latent space features respectively according to the eye region image and the mask at each level of resolution;

[0015] For the latent space features of the eye region image at each level of resolution, the latent space features of the eye region image are adjusted by editing the latent space features of the corresponding mask.

[0016] In the above technical solution, the image to be removed from the red blood and the mask image are input into the network together to obtain several images of different resolutions and mask images of the red blood area. By introducing spatial attention and channel attention, the model can adaptively select the key features of the red blood area at each resolution level and dynamically adjust the weights of features of different scales. This can help the model focus on important areas when processing images of different resolutions, and remove red blood more accurately. At the low-resolution level, the main focus is on the overall red blood area; at the medium and high-resolution levels, the processing of local high-frequency areas is refined. Using image blocks of different resolutions for feature extraction can extract more details, especially subtle changes in the red blood area, to ensure that the effect of removing red blood is more natural. The editing process of latent space features enables us to accurately control the features of the red blood removal area and retain other features related to the texture of the original image, thereby ensuring that the final image is natural and realistic.

[0017] In some embodiments, the latent spatial features of the mask include red blood vessel density.

[0018] In the above technical solution, the red blood density refers to the distribution of red blood in the eye area image, which can be extracted as a feature of the image. The red blood density can be described by a probability model, in which the probability distribution of the red blood part and the normal part can be assumed to be a Gaussian distribution. The red blood density can be characterized by its expected value (mean) and variance in the probability density function. The latent space features of the mask include the red blood density for the detection, enhancement and editing of red blood. In this way, the red blood in the eye area image can be processed more accurately.

[0019] In some embodiments, editing and adjusting the latent space features of the eye region image through the latent space feature information of the mask further includes:

[0020] The editing intensity is adjusted according to the red blood density.

[0021] In the above technical solution, by introducing the feature editing strength adjustment module, the model can dynamically adjust the strength of feature editing according to the severity of red blood streaks. In areas with lower red blood streak density, the model can keep more original textures, while in areas with higher red blood streak density, the feature editing strength is increased. This method makes the red blood streak removal effect more personalized and adaptable, and can handle red blood streaks of different severity.

[0022] In some embodiments, generating a result image based on the latent space features of the edited eye region image, and aligning the result image to the original image through the inverse matrix of the alignment matrix to obtain a target image includes:

[0023] The latent space features of different resolutions are fused, and the resulting image is generated based on the fused features.

[0024] After the result image is aligned to the original image through the alignment matrix, the result image needs to be fused with the mask and the aligned image, and the fusion is performed using the following formula:

[0025] I=M⊙I 1 +(1-M)⊙I 2

[0026] Where I represents the target image, M represents the red blood region mask, and I 1 represents the aligned image, I 2 Represents the resulting image.

[0027] In the above technical solution, before generating the result image, it is necessary to fuse the latent space features of the eye area images of different resolutions. This step enhances the representation ability of the model by fusing features of different resolutions. In the processing of the eye area image, the fusion of multi-resolution features can help capture richer details and contextual information, thereby generating a higher quality result image. The alpha fusion process can smooth the image boundaries and avoid obvious seams.

[0028] According to another aspect of the present invention, a training method for a red blood streak removal model based on latent space feature editing is provided, wherein the red blood streak removal model comprises: a red blood streak region segmentation network, a latent space feature extraction network, and a generation network;

[0029] Constructing a data set, the data set includes a plurality of data pairs, the data pairs including sample images of eyes with red blood streaks and paired images of eyes without red blood streaks;

[0030] For each sample image in a number of data pairs, input it into the red blood region segmentation network to extract the eye region image of the sample image and extract the red blood region mask of the eye region through the segmentation network, and calculate the eye alignment matrix based on the obtained preliminary eye region image and the red blood mask and a set image;

[0031] For each eye region image and the corresponding mask, input them into the latent space feature extraction network to extract the aligned sample image and the latent space features of the mask respectively; for the latent space features of the eye region image, edit and adjust the latent space features of the eye region image through the latent space feature information of the mask;

[0032] Inputting the latent space features of the edited eye region image into a generative network, generating a result image based on the latent space features of the edited eye region image, and aligning the result image to the original image through the inverse matrix of the alignment matrix to obtain a target image;

[0033] Calculate the loss value for each paired image of the target image and the sample image corresponding to the eye without red bloodshot; and,

[0034] The model is trained based on the loss value.

[0035] In the above technical scheme, in order to better use the above method, the present application proposes a training method for a red blood streak removal model based on latent space feature editing. Each step corresponds to each step of the above method. The specific principles have been described above and will not be repeated here.

[0036] According to another aspect of the present invention, a device for removing red blood streaks based on latent space feature editing is provided, comprising:

[0037] A segmentation module is used to extract the eye region image in the original image and extract the red blood region mask of the eye region through the segmentation network, and calculate the eye alignment matrix based on the obtained preliminary eye region image and the red blood region mask and a set image;

[0038] A latent space feature module is used to extract the aligned eye region image and the latent space features of the mask respectively; with respect to the latent space features of the eye region image, the latent space features of the eye region image are edited and adjusted through the latent space feature information of the mask;

[0039] The generation module is used to generate a result image based on the latent space features of the edited eye area image, and align the result image to the original image through the inverse matrix of the alignment matrix to obtain a target image.

[0040] In the above technical scheme, in order to better use the above method, the present application proposes a device for removing red blood streaks based on latent space feature editing. Each module corresponds to each step of the above method. The specific principles have been described above and will not be repeated here.

[0041] According to another aspect of the present invention, a device for removing red blood streak model based on latent space feature editing is provided, comprising:

[0042] at least one processor and a memory communicatively coupled to the at least one processor;

[0043] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the above method.

[0044] In the above technical solution, in order to better run and process the method, the above method is stored in a memory, and the stored method is executed by a processor. It should be noted that the principle and effect of each step have been described above and will not be described in detail here.

[0045] According to another aspect of the present invention, a computer-readable storage medium is provided, storing a computer program, wherein the computer program implements the above method when executed by a processor.

[0046] In the above technical solution, in order to better run and use the method, the above method is stored in a computer-readable storage medium, and the above method is implemented by a processor. It should be noted that the principle and effect of each step have been described above and will not be described in detail here. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0048] Figure 1 It is a flow chart of an embodiment of a method for removing red blood streaks based on latent space feature editing of the present invention;

[0049] Figure 2 It is a flow chart of an embodiment of a training method for a red blood streak removal model based on latent space feature editing of the present invention;

[0050] Figure 3 It is a structural schematic diagram of an embodiment of a device for removing red blood streaks based on latent space feature editing of the present invention. DETAILED DESCRIPTION

[0051] The present invention will be further described in detail below in conjunction with the accompanying drawings and examples. It is particularly noted that the following examples are only used to illustrate the present invention, but are not intended to limit the scope of the present invention. Similarly, the following examples are only partial embodiments of the present invention rather than all embodiments, and all other embodiments obtained by those of ordinary skill in the art without creative work are within the scope of protection of the present invention.

[0052] The present invention provides a method for removing red blood streaks based on latent space feature editing, which can automatically remove red blood streaks in eyes.

[0053] Embodiment 1

[0054] See also Figure 1 , a method for removing red blood streaks based on latent space feature editing, the method comprising:

[0055] A1. Extract the eye region image in the original image and extract the red blood region mask of the eye region through the segmentation network, and calculate the eye alignment matrix based on the obtained preliminary eye region image and the red blood region mask and a set image;

[0056] A2, respectively extracting the aligned eye region image and the latent space features of the mask; with respect to the latent space features of the eye region image, editing and adjusting the latent space features of the eye region image through the latent space feature information of the mask;

[0057] A3. Generate a result image based on the latent space features of the edited eye region image, and align the result image to the original image through the inverse matrix of the alignment matrix to obtain a target image.

[0058] In this embodiment, first, the eye region image and the red blood region mask of the eye region are extracted from the original image by image processing technology. This step is used to identify and extract the eye region and its red blood region. Next, the alignment matrix of the mask is calculated and aligned with the original image. This process involves image alignment technology. For example, the Homography matrix can be used to map points on one image to corresponding points on another image to achieve image alignment. Secondly, the latent space features of the eye region image and the mask are extracted respectively. That is, the space where the latent variables in the generative model are located. Here, the generative adversarial network (GANs) can be used to learn the latent space features of the eye region image and the red blood mask. For the latent space features of the eye region image, the latent space features of the eye region image are edited and adjusted by the latent space feature information of the mask. This process can use, for example, the InterFaceGAN framework, which performs semantic facial editing by interpreting the latent semantics learned by GAN, including the control of attributes such as gender, age, and expression. Here, we can apply this technology to the editing of red blood, and reduce or eliminate red blood by adjusting the latent space features. Finally, the result image is generated based on the latent space features of the edited eye area image. This step can use, for example, the DreamOmni model, which can generate images of different resolutions based on the latent space features. The result image is aligned to the original image through the alignment matrix to obtain the target image, ensuring that the generated image is consistent with the original image in spatial position. The above method is based on the red blood streak removal method based on latent space feature editing. By extracting and editing the latent space features of the eye area and its red blood streak, and combining image alignment technology, it can effectively remove or reduce the red blood streak and generate a natural target image that is aligned with the original image.

[0059] In this embodiment, the aligned eye region image and the latent space features of the mask are extracted respectively; with respect to the latent space features of the eye region image, the latent space features of the eye region image are edited and adjusted through the latent space feature information of the mask, including:

[0060] Performing multi-resolution feature extraction on the eye region image and the mask to obtain a plurality of eye region images and the mask at different resolutions, and extracting latent space features respectively according to the eye region image and the mask at each level of resolution;

[0061] For the latent space features of the eye region image at each level of resolution, the latent space features of the eye region image are adjusted by editing the latent space features of the corresponding mask.

[0062] In this embodiment, the image to be removed of red blood and the mask image are input into the network together and several images of different resolutions and mask images of the red blood area are obtained. By introducing spatial attention and channel attention, the model can adaptively select the key features of the red blood area at each resolution level and dynamically adjust the weights of features of different scales. This can help the model focus on important areas when processing images of different resolutions, and remove red blood more accurately. At the low-resolution level, the overall red blood area is mainly concerned; at the medium and high-resolution levels, the processing of local high-frequency areas is refined. Using image blocks of different resolutions for feature extraction can extract more details, especially subtle changes in the red blood area, to ensure that the effect of removing red blood is more natural. The editing process of latent space features enables us to accurately control the features of the red blood removal area and retain other features related to the texture of the original image, thereby ensuring that the final image is natural and realistic.

[0063] In this embodiment, the latent space features of the mask include red blood density. Red blood density refers to the distribution of red blood in the eye area image, which can be extracted as a feature of the image. Red blood density can be described by a probability model, in which the probability distribution of the red blood part and the normal part can be assumed to be a Gaussian distribution. Red blood density can be characterized by its expected value (mean) and variance in the probability density function. The latent space features of the mask include red blood density for red blood detection, enhancement and editing. In this way, the red blood in the eye area image can be processed more accurately.

[0064] In this embodiment, the latent space features of the eye area image are adjusted by editing the latent space feature information of the mask, and further comprising: adjusting the editing intensity according to the red blood density. By introducing a feature editing intensity adjustment module, the model can dynamically adjust the intensity of feature editing according to the severity of the red blood. In areas with lower red blood density, the model can retain more original textures, while in areas with higher red blood density, the feature editing intensity is increased. This approach makes the red blood removal effect more personalized and adaptable, and can handle red blood of different severities.

[0065] In this embodiment, a result image is generated based on the latent space features of the edited eye region image, and the result image is aligned to the original image through the inverse matrix of the alignment matrix to obtain a target image, including: fusing latent space features of different resolutions, and generating a result image based on the fused features,

[0066] After the result image is aligned to the original image through the alignment matrix, the result image needs to be fused with the mask and the aligned image, and the fusion is performed using the following formula:

[0067] I=M⊙I 1 +(1-M)⊙I 2

[0068] Where I represents the target image, M represents the red blood region mask, and I 1 represents the aligned image, I 2 Represents the resulting image.

[0069] Before generating the result image, the latent space features of the eye area images of different resolutions need to be fused. This step enhances the representation ability of the model by fusing features of different resolutions. In the processing of eye area images, the fusion of multi-resolution features can help capture richer details and contextual information, thereby generating a higher quality result image. The alpha fusion process can smooth the image boundaries and avoid obvious seams.

[0070] In order to better explain this example, the following will be further explained. The detailed steps are as follows:

[0071] 1. Input the image to the red blood region segmentation network: Input the image to be de-reduce red blood into the red blood region segmentation network to obtain the mask of the region containing red blood in the image. The segmentation network is used to identify and extract the red blood region. This helps to accurately locate the red blood region and avoid processing other irrelevant regions.

[0072] 2. Calculate the alignment matrix and perform image alignment: Calculate the alignment matrix of the eye image based on the red blood streak area mask obtained by segmentation, and align the image to be de-reduce and the mask image.

[0073] 3. Multi-level latent space feature extraction: The image to be removed of red blood streaks and the mask image are input into the network together and 5 images of different resolutions and the mask image of the red blood streak area are obtained. By introducing spatial attention and channel attention, the model can adaptively select the key features of the red blood streak area at each resolution level and dynamically adjust the weights of features of different scales. This can help the model focus on important areas when processing images of different resolutions, and remove red blood streaks more accurately. At the low-resolution level, the main focus is on the overall red blood streak area; at the medium and high-resolution levels, the processing of local high-frequency areas is refined. Multi-resolution feature extraction: Using image blocks of different resolutions for feature extraction can extract more details, especially subtle changes in the red blood streak area, to ensure a more natural red blood streak removal effect.

[0074] 4. Latent Space Feature Editing and Reconstruction

[0075] (1) Each input image provides mask information, such as the size of the red blood region, so as to introduce contextual information when editing features. The model not only recognizes red blood, but also performs more detailed editing based on different skin colors, red blood density and other information. This process is carried out through a multi-scale generator to control the editing effect of features at different scales, making the removal of red blood more natural.

[0076] (2) By introducing a feature editing strength adjustment module, the model can dynamically adjust the strength of feature editing according to the severity of red blood streaks. In areas with mild red blood streaks, the model can maintain more original textures, while in areas with more severe red blood streaks, the feature editing strength is increased. This innovation makes the red blood streak removal effect more personalized and adaptable, and can handle red blood streaks of different severity.

[0077] The editing process of latent space features enables us to precisely control the features of the red bloodshot area and retain other features related to the texture of the original image, thus ensuring that the final image is natural and realistic.

[0078] 5. Image reconstruction: The latent space features obtained at 5 different scales are fused and passed through the decoder to obtain the preliminary result image of red blood streak removal. The anti-alignment matrix is ​​calculated based on the alignment matrix, and the result image is aligned to the position of the eye of the original image. Alpha fusion is performed in combination with the degree coefficient to obtain the final red blood streak removal effect image. The anti-alignment step ensures that the geometric structure of the image is restored, while the alpha fusion process can smooth the image boundaries and avoid obvious seams.

[0079] Embodiment 2

[0080] See also Figure 2, a training method for a red blood streak removal model based on latent space feature editing, the red blood streak removal model comprising: a red blood streak region segmentation network, a latent space feature extraction network, and a generation network;

[0081] B1. Construct a data set, which includes a number of data pairs, each of which includes a sample image of an eye with red blood streaks and a paired image of an eye without red blood streaks;

[0082] B2. For each sample image in the plurality of data pairs, input the sample image into the red blood region segmentation network to extract the eye region image in the sample image and extract the red blood region mask of the eye region through the segmentation network, and calculate the eye alignment matrix according to the obtained preliminary eye region image and the red blood region mask and a set image;

[0083] B3. For each eye region image and the corresponding mask, input them into the latent space feature extraction network to extract the aligned sample image and the latent space features of the mask respectively; for the latent space features of the eye region image, edit and adjust the latent space features of the eye region image through the latent space feature information of the mask;

[0084] B4. Inputting the latent space features of the edited eye region image into the generation network, generating a result image based on the latent space features of the edited eye region image, and aligning the result image to the original image through the inverse matrix of the alignment matrix to obtain a target image;

[0085] B5, calculating the loss value for each paired image of the target image and the sample image corresponding to the eye without red bloodshot; and,

[0086] B6. Train the model based on the loss value.

[0087] In the above technical scheme, in order to better use the above method, the present application proposes a training method for a red blood streak removal model based on latent space feature editing. Each step corresponds to each step of the above method. The specific principles have been described above and will not be repeated here.

[0088] The loss value is calculated as follows:

[0089] 1. GAN Loss

[0090] (1) Generator loss function

[0091]

[0092] in, represents the loss of the generator; represents the expected value; z is from the prior distribution; p z(z); G(z) is the generator network, which receives the noise z and generates fake samples; D is the discriminator network, which tries to distinguish between real samples and fake samples generated by the generator; log D(G(z)) is the logarithmic probability of the discriminator's output for samples generated by the generator.

[0093] (2) Discriminator loss function

[0094]

[0095] in, represents the loss of the discriminator; represents the expected value; x is the value from the real data distribution p data (x); D(x) is the output of the discriminator network for the real data; z is the data sampled from the prior distribution p z (z); G(z) is the generator network, which receives the noise z and generates fake samples; D(G(z)) is the output of the discriminator network for the fake samples generated by the generator.

[0096] This loss function consists of two parts:

[0097] 1. Part I Encourage the discriminator to correctly identify real data.

[0098] 2. Part II The discriminator is encouraged to correctly identify the generated fake data.

[0099] (3) The overall optimization goal of GAN:

[0100]

[0101] 2. VGG Loss

[0102] VGG Loss is a perceptual loss based on the VGG network, which is used to measure the difference between the generated image and the target image in high-level features. It is often used in tasks such as image generation and image super-resolution. The following is the calculation formula of VGG loss:

[0103] Define input image I and target image I gt ; Use the pre-trained VGG network to extract features. Assuming that the feature extraction layer of the VGG network is φ, the extracted features are represented as φ(I) and φ(I gt )

[0104] The VGG loss is defined as the Euclidean distance between the input image and the target image in the feature space:

[0105]

[0106] in, represents VGG loss; I is the input image or generated image; I gt is the target image or the real image; N is the total number of elements in the feature map; φ i Represents the i-th element of the feature map; Represents the square of the Euclidean distance (L2 norm), which is used to calculate the difference between two feature vectors.

[0107] 3. Cross Entropy Loss, a loss function suitable for multi-category segmentation tasks, is mainly used to supervise the accuracy of segmentation.

[0108] Embodiment 3

[0109] See also Figure 3 , a device for removing red blood streaks based on latent space feature editing, comprising

[0110] A segmentation module is used to extract the eye region image in the original image and extract the red blood region mask of the eye region through the segmentation network, and calculate the eye alignment matrix based on the obtained preliminary eye region image and the red blood region mask and a set image;

[0111] A latent space feature module is used to extract the aligned eye region image and the latent space features of the mask respectively; with respect to the latent space features of the eye region image, the latent space features of the eye region image are edited and adjusted through the latent space feature information of the mask;

[0112] The generation module is used to generate a result image based on the latent space features of the edited eye area image, and align the result image to the original image through the inverse matrix of the alignment matrix to obtain a target image.

[0113] In the above technical scheme, in order to better use the method described in Example 1 or Example 2, the present application proposes a device for removing red blood streaks based on latent space feature editing. Each module corresponds to each step of the above method. The specific principles have been described above and will not be repeated here.

[0114] Embodiment 4

[0115] A device for removing red blood streak model based on latent space feature editing, comprising:

[0116] at least one processor and a memory communicatively coupled to the at least one processor;

[0117] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the above method.

[0118] In the above technical solution, in order to better run and process the method described in Embodiment 1 or Embodiment 2, the method described in Embodiment 1 or Embodiment 2 is stored in a memory, and a processor is used to execute the stored method. It should be noted that the principle and effect of each step have been described above and will not be described in detail here.

[0119] Embodiment 5

[0120] A computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the method described in one or two embodiments is implemented.

[0121] In the above technical solution, in order to better run and use the method described in Embodiment 1 or Embodiment 2, the method described in Embodiment 1 or Embodiment 2 is stored in a computer-readable storage medium, and the above method is implemented using a processor. It should be noted that the principle and effect of each step have been described above and will not be described in detail here.

[0122] The above descriptions are only some embodiments of the present invention, and are not intended to limit the protection scope of the present invention. Any equivalent device or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for removing red blood streaks based on latent space feature editing, characterized in that: The method comprises: Extract the eye region image from the original image and extract the red bloodshot region mask of the eye region through the segmentation network, and calculate the eye alignment matrix based on the obtained preliminary eye region image and the red bloodshot mask and a set image; Extracting the aligned eye region image and the latent space features of the mask respectively; with respect to the latent space features of the eye region image, editing and adjusting the latent space features of the eye region image through the latent space feature information of the mask; A result image is generated based on the latent space features of the edited eye region image, and the result image is aligned to the original image through the inverse matrix of the alignment matrix to obtain a target image.

2. The method for removing red blood streaks based on latent space feature editing as claimed in claim 1, characterized in that: Extract the aligned eye region image and the mask latent space features respectively; With respect to the latent space features of the eye region image, the latent space features of the eye region image are edited and adjusted through the latent space feature information of the mask, including: Performing multi-resolution feature extraction on the eye region image and the mask to obtain a plurality of eye region images and the mask at different resolutions, and extracting latent space features respectively according to the eye region image and the mask at each level of resolution; For the latent space features of the eye region image at each level of resolution, the latent space features of the eye region image are adjusted by editing the latent space features of the corresponding mask.

3. The method for removing red blood streaks based on latent space feature editing as claimed in claim 2, characterized in that: The latent spatial features of the mask include red blood vessel density.

4. The method for removing red blood streaks based on latent space feature editing as claimed in claim 3, characterized in that: Editing and adjusting the latent space features of the eye region image through the latent space feature information of the mask also includes: The editing intensity is adjusted according to the red blood density.

5. The method for removing red blood streaks based on latent space feature editing as claimed in claim 2, characterized in that: A result image is generated based on the latent space features of the edited eye region image, and the result image is aligned to the original image through the inverse matrix of the alignment matrix to obtain a target image, including: Fusion of latent space features of different resolutions, and generation of result images based on the fused features; After the result image is aligned to the original image through the alignment matrix, the result image needs to be fused with the mask and the aligned image, and the fusion is performed using the following formula: I=M⊙I1+(1-M)⊙I2 Where I represents the target image, M represents the red blood region mask, I1 represents the aligned image, and I2 represents the result image.

6. A training method for removing red blood streak model based on latent space feature editing, characterized in that: The red blood streak removal model includes: a red blood streak region segmentation network, a latent space feature extraction network, and a generation network; Constructing a data set, the data set includes a plurality of data pairs, the data pairs including sample images of eyes with red blood streaks and paired images of eyes without red blood streaks; For each sample image in a number of data pairs, input it into the red blood region segmentation network to extract the eye region image in the sample image and extract the red blood region mask of the eye region through the segmentation network, and calculate the eye alignment matrix according to the obtained preliminary eye region image and the red blood mask and a set image; For each eye region image and the corresponding mask, input them into the latent space feature extraction network to extract the aligned sample image and the latent space features of the mask respectively; for the latent space features of the eye region image, edit and adjust the latent space features of the eye region image through the latent space feature information of the mask; Inputting the latent space features of the edited eye region image into a generative network, generating a result image based on the latent space features of the edited eye region image, and aligning the result image to the sample image through the inverse matrix of the alignment matrix to obtain a target image; Calculate the loss value for each paired image of the target image and the sample image corresponding to the eye without red bloodshot; and, The model is trained based on the loss value.

7. A device for removing red blood streaks based on latent space feature editing, characterized in that: include A segmentation module is used to extract the eye region image in the original image and extract the red blood region mask of the eye region through the segmentation network, and calculate the eye alignment matrix based on the obtained preliminary eye region image and the red blood region mask and a set image; A latent space feature module, used to extract the aligned eye region image and the latent space features of the mask respectively; For the latent space features of the eye region image, the latent space features of the eye region image are edited and adjusted through the latent space feature information of the mask; The generation module is used to generate a result image based on the latent space features of the edited eye area image, and align the result image to the original image through the inverse matrix of the alignment matrix to obtain a target image.

8. A device for removing red blood streak model based on latent space feature editing, characterized in that: include: at least one processor and a memory communicatively coupled to the at least one processor; The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method as claimed in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images