A chromosome image processing method, device and storage medium based on autoencoder

Through the autoencoder-based method, using vector quantization and random masking operation, the generation problems of adversarial networks and diffusion models in chromosome image processing are solved, the image quality and model robustness are improved, and better characterization learning is achieved and generalization ability.

CN118212318BActive Publication Date: 2024-11-19笑纳科技(苏州)有限公司
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410441497.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-04-12
Publication Date
2024-11-19
Estimated Expiration
2044-04-12

AI Technical Summary

Technical Problem

The existing adversarial networks and diffusion models have problems in chromosomal image processing, such as the generated samples are too similar, lack of diversity, unstable training process, insufficient real details, and single data types. The self-supervised learning reconstruction problems occur with blur and distortion.

Method used

Using an autoencoder-based method, the chromosomal image is reconstructed using pre-trained autoencoder (including encoder and decoder) through vector quantization processing and random mask operation, and the shared class marker is replaced by specific class markers, which enhances the Image generation and characterization learning performance.

Benefits of technology

It improves the quality of generated images and the robustness of the model, enhances the fine-grained analysis of image content, and improves the generalization ability of the model.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118212318B_ABST
    Figure CN118212318B_ABST
Patent Text Reader

Abstract

The present invention proposes a chromosome image processing method, device and storage medium based on an autoencoder, comprising the following steps: obtaining an original image, performing vector quantization processing on the original image to obtain preprocessing tokens; performing random mask processing on the preprocessing tokens to obtain masked tokens; using a pre-trained autoencoder to reconstruct the masked tokens to obtain a generated image; the present invention utilizes the encoder-decoder structure of the autoencoder to better learn the high-level feature representation of the input image and a more fine-grained analysis of the image content, which not only improves the quality of the generated image, but also increases the robustness of the model to input changes, so that the model can better adapt to different image changes and noises, and improves its generalization ability.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of medical image processing, and in particular to a chromosome image processing method, device and storage medium based on an autoencoder. Background Art

[0002] Chromosomes are the carriers of genes. Chromosome abnormalities will inevitably cause imbalanced gene expression, thereby causing a series of clinical symptoms. Chromosome structural abnormalities: deletion, duplication, inversion, insertion, translocation. Doctors can determine whether the chromosome is abnormal by analyzing the structure of the chromosome in the image. At present, the use of deep learning algorithms is widely used in the fields of recognition and detection. It only requires collecting enough samples and selecting a suitable deep learning network. After training, a better recognition model or detection model can be obtained.

[0003] At present, there are several classification algorithms: Adversarial networks are a powerful image generation technology that generates realistic images by adversarially training generators and discriminators. In image classification, adversarial networks can be used to synthesize images and expand training data sets, thereby improving the generalization ability of classifiers. Diffusion model is an emerging image generation paradigm that generates images by iteratively diffusing noise. Its core idea is to generate images through multiple steps of iteration, and each step gradually improves the image quality. Self-supervised learning is to mask parts of the image and reconstruct these areas to learn image features.

[0004] However, adversarial networks may experience mode collapse when generating images, that is, the generated samples are too similar and lack diversity. This will result in the quality of the generated images being lower than expected, and the training process is usually unstable and prone to gradient vanishing or gradient exploding, which leads to the unreliability of the training process. At the same time, the images generated by the adversarial network may not be realistic enough in details, such as lacking clear edges or textures; in addition, adversarial networks usually focus on generation tasks and ignore representation learning, which may result in the generated images being not rich enough in representation. The diffusion model requires a large number of sampling steps and a long sampling time, and also faces the problem of processing a single data type. Although the images reconstructed by self-supervised learning have similar semantic information to the original images, they will have serious blurring and distortion problems. Summary of the invention

[0005] To solve the above problems, the present invention proposes a chromosome image processing method, device and storage medium based on an autoencoder.

[0006] The main contents of the present invention include:

[0007] A chromosome image processing method based on an autoencoder comprises the following steps:

[0008] Get the original image, perform vector quantization on the original image, and obtain preprocessed tokens;

[0009] Performing random masking on the preprocessed tokens to obtain masked tokens;

[0010] Use the pre-trained autoencoder to reconstruct the masked tokens to obtain the generated image;

[0011] The autoencoder includes an encoder and a decoder; the latent feature space of the pre-trained encoder is recorded as a class label [C0]; the latent feature space currently learned by the encoder is recorded as a specific class label [C]; the specific class label [C] is used to replace the class label feature [C0] in the current encoder output to obtain a first output; the decoder uses the first output as input for decoding processing to generate an image.

[0012] Preferably, vector quantization is performed on the original image to obtain preprocessed tokens, including:

[0013] Use the convolutional network encoder to encode the original image and obtain the intermediate feature variable ;

[0014] According to the pre-generated codebook: , generating variables with the same dimensions ;

[0015] in,

[0016] ;

[0017] The variable Constitute the preprocessing tokens.

[0018] Preferably, the training step of the pre-trained autoencoder includes training the autoencoder using different masking rates and corresponding pre-processed tokens.

[0019] Preferably, the autoencoder is a transformer model.

[0020] Preferably, the corresponding pre-processed tokens are chromosome images that have been vector quantized; and the chromosome images have been stained.

[0021] Preferably, the chromosome image is dyed, including:

[0022] After G-banding treatment of chromosomes, chromosomes showing banding textures were obtained;

[0023] A chromosome picture showing a banding texture is obtained as the chromosome image to be processed by vector quantization.

[0024] The present invention also proposes an image processing device based on an autoencoder, comprising:

[0025] An acquisition module, used for acquiring the original image;

[0026] The vector quantization module is used to perform vector quantization on the original image and generate tokens to be processed;

[0027] A mask generation module, used to randomly generate masks;

[0028] A pre-training module, used for pre-training the autoencoder with different masking rates to obtain a pre-trained autoencoder;

[0029] The generation module uses the specific class label [C] currently learned by the encoder of the autoencoder and the mask tokens as the input of the decoder of the autoencoder according to the mask provided by the mask generation module to obtain a generated image.

[0030] Preferably, according to the mask provided by the mask generation module, the specific class label [C] currently learned by the encoder of the autoencoder and the mask tokens are used as the input of the decoder of the autoencoder, including:

[0031] The class label [C0] of the autoencoder is replaced by the specific class label [C] learned by the encoder of the autoencoder, and is combined with the unmasked tokens after the current mask processing as the input of the decoder of the autoencoder.

[0032] The present invention also proposes a storage medium on which a computer program is stored, characterized in that the computer program is executed by a processor to implement the above-mentioned chromosome image processing method based on an autoencoder.

[0033] The beneficial effects of the present invention are as follows: the present invention proposes a chromosome image processing method, device and storage medium based on an autoencoder, which utilizes the encoder-decoder structure of the autoencoder to better learn the high-level feature representation of the input image and a more fine-grained analysis of the image content, and reconstruct the image more accurately through the decoder, thereby improving the quality of the generated image; the random mask operation and the corresponding decoding process can increase the robustness of the model to input changes, making the model more adaptable to different image changes and noises, and improving its generalization ability. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0035] The technical solution protected by the present invention will be described in detail below with reference to the accompanying drawings.

[0036] The present invention also proposes a chromosome image processing method, device and storage medium based on an autoencoder, which uses a pre-trained autoencoder to process the original image, and controls the input of the decoder so that the decoder can reconstruct a clear generated image, which not only enhances the image generation performance, but also enhances the representation learning performance of the autoencoder. Figure 1 The image processing method of the present invention comprises the following steps:

[0037] A chromosome image processing method based on an autoencoder comprises the following steps:

[0038] First, the original image is obtained and vector quantized to obtain preprocessed tokens, wherein the preprocessed tokens contain the semantic information of the original image. The specific steps include:

[0039] Use the convolutional network encoder to encode the original image and obtain the intermediate feature variable ;

[0040] According to the pre-generated codebook: , generating variables with the same dimensions ;

[0041] in,

[0042] ;

[0043] The variable The tokens to be processed are formed; that is, the original image is converted into an input suitable for inputting into the autoencoder model; then the tokens to be processed are randomly masked and input into the pre-trained autoencoder. The present invention improves the quality of the generated image by controlling the input of the decoder of the autoencoder, and specifically includes the following steps:

[0044] The tokens to be processed are randomly masked to obtain masked tokens; the masked tokens are used as the input of a pre-trained autoencoder, wherein the autoencoder includes an encoder and a decoder, that is, the masked tokens are used as the input of the encoder; because the pre-trained autoencoder can reconstruct images of a certain quality, the potential feature space of the encoder of the model is recorded as a class label [C0]; and under the guidance of the learned class label [C0], the encoder records the features learned through training as a specific class label [C] for the masked tokens obtained by this masking process.

[0045] The output of the encoder includes both the trained and learned class label features [C0] and the target sequence token, as well as the attention information and position coding from the encoder. Before the output of the encoder is input to the decoder, the present invention replaces the class label [C0] with a specific class label [C], and together with the unmasked tokens and the encoder's attention information, position coding and other information, forms a first output, and uses the first output as the input of the decoder, so that the decoder reconstructs the image to obtain the generated image. That is, in the decoding process, instead of using a learnable mask label shared between different images, a class label that is specific to each image is used to fill the output of the encoder, thereby simultaneously enhancing image generation and representation learning performance.

[0046] Among them, the autoencoder is a model obtained after training with different mask rates, and its class label feature [C0] includes the global features of the input tokens. Since the specific class label feature [C] is inevitably affected by the class label feature [C0], and the specific class label feature [C] also includes the corresponding specific features learned this time, the specific class label [C] is used to replace the class label [C0] as the input of the decoder, so that the decoder can reconstruct the image with higher quality, which not only enhances the image generation performance, but also improves the model representation learning performance.

[0047] Of course, after this processing, the class label feature [C0] may remain unchanged or may change. In this embodiment, the autoencoder adopts a transformer model.

[0048] Furthermore, in order to enable the autoencoder to obtain more semantic information of the tokens to be processed, the dyed chromosome image is used for vector quantization when training the autoencoder. Specifically, before obtaining the dyed image, the chromosome is first dyed to obtain a chromosome showing a striped texture; then a chromosome picture showing a striped texture is obtained as the chromosome image to be vector quantized, and the chromosome image showing the chromosome striped texture and different mask rates are selected according to different usage scenarios to train the autoencoder, so that the autoencoder can learn more semantic information.

[0049] In this embodiment, the chromosomes are processed using the G-banding technique, and the specific steps include:

[0050] (1) Sample preparation: Chromosome Giemsa staining is usually performed using white blood cells, which can be peripheral blood cells, bone marrow cells, etc. The sample should be prepared on a glass slide, usually by evenly spreading the sample on the slide using the smear method.

[0051] (2) Fix the sample: Use a fixative to fix the sample on a glass slide. Commonly used fixatives include ethanol, methanol or ethyl acetate. During the fixation process, an appropriate amount of deionized water can be added to help dissolve and fix the cells.

[0052] (3) Staining: The fixed sample is immersed in Giemsa staining solution, usually a diluted solution of Giemsa staining solution. The staining time is usually 5-20 minutes, depending on the desired degree of staining.

[0053] (4) Cleaning: After staining is completed, remove the sample from the staining solution and gently wash it with deionized water to remove excess staining solution and fixative.

[0054] (5) Drying: Place the slide in a ventilated place or blow dry with warm air to make the sample completely dry.

[0055] (6) Observation: Place the dried sample under a microscope and observe it with a 10x to 100x objective lens to obtain the desired stained chromosomes. The bands and structure of the desired chromosomes should be clearly visible.

[0056] The present invention also proposes an image processing device based on an autoencoder, which can execute the above-mentioned image processing method. Specifically, the device includes:

[0057] An acquisition module, used for acquiring the original image;

[0058] The vector quantization module is used to perform vector quantization on the original image and generate tokens to be processed;

[0059] A mask generation module, used to randomly generate masks;

[0060] A pre-training module, used for pre-training the autoencoder with different masking rates to obtain a pre-trained autoencoder;

[0061] The generation module uses the specific class label [C] currently learned by the encoder of the autoencoder and the mask tokens as the input of the decoder of the autoencoder according to the mask provided by the mask generation module to obtain a generated image.

[0062] Preferably, according to the mask provided by the mask generation module, the specific class label [C] currently learned by the encoder of the autoencoder and the mask tokens are used as the input of the decoder of the autoencoder, including:

[0063] The class label [C0] of the autoencoder is replaced by the specific class label [C] learned by the encoder of the autoencoder, and is combined with the unmasked tokens after the current mask processing as the input of the decoder of the autoencoder.

[0064] The present invention also proposes a storage medium on which a computer program is stored, characterized in that the computer program is executed by a processor to implement the above-mentioned chromosome image processing method based on an autoencoder.

[0065] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A chromosome image processing method based on an autoencoder, characterized in that: The steps include: Get the original image, perform vector quantization on the original image, and obtain preprocessed tokens; Performing random masking on the preprocessed tokens to obtain masked tokens; Use the pre-trained autoencoder to reconstruct the masked tokens to obtain the generated image; The autoencoder includes an encoder and a decoder; the latent feature space of the pre-trained encoder is recorded as a class label [C0]; the latent feature space currently learned by the encoder is recorded as a specific class label [C]; the class label feature [C0] in the current encoder output is replaced by the specific class label [C] to obtain a first output; the decoder uses the first output as an input for decoding processing to generate an image; Perform vector quantization on the original image to obtain preprocessed tokens, including: Use the convolutional network encoder to encode the original image and obtain the intermediate feature variable ; According to the pre-generated codebook: , generating variables with the same dimensions ; in, ; The variable Constitute the preprocessing tokens.

2. According to the chromosome image processing method based on autoencoder according to claim 1, it is characterized in that: The training step of the pre-trained autoencoder includes training the autoencoder using different masking rates and corresponding pre-processed tokens.

3. The chromosome image processing method based on autoencoder according to claim 2 is characterized in that: The autoencoder is a transformer model.

4. The chromosome image processing method based on autoencoder according to claim 2, characterized in that: The corresponding pre-processed tokens are chromosome images that have been vector quantized; the chromosome images have been dyed.

5. The chromosome image processing method based on autoencoder according to claim 4 is characterized in that: The chromosome image is subjected to dyeing processing, including: After G-banding treatment of chromosomes, chromosomes showing banding textures were obtained; A chromosome picture showing a banding texture is obtained as the chromosome image to be processed by vector quantization.

6. An image processing device based on an autoencoder, characterized in that: include: An acquisition module, used for acquiring the original image; The vector quantization module is used to perform vector quantization on the original image and generate tokens to be processed; A mask generation module, used to randomly generate masks; A pre-training module, used for pre-training the autoencoder with different masking rates to obtain a pre-trained autoencoder; A generation module, according to the mask provided by the mask generation module, uses the specific class label [C] currently learned by the encoder of the autoencoder and the mask tokens as the input of the decoder of the autoencoder to obtain a generated image; Perform vector quantization on the original image to obtain preprocessed tokens, including: Use the convolutional network encoder to encode the original image and obtain the intermediate feature variable ; According to the pre-generated codebook: , generating variables with the same dimensions ; in, ; The variable Constitute the preprocessing tokens.

7. The image processing device based on the autoencoder according to claim 6, characterized in that: According to the mask provided by the mask generation module, the specific class label [C] currently learned by the encoder of the autoencoder and the mask tokens are used as the input of the decoder of the autoencoder, including: The class label [C0] of the autoencoder is replaced by the specific class label [C] learned by the encoder of the autoencoder, and is combined with the unmasked tokens after the current mask processing as the input of the decoder of the autoencoder.

8. A storage medium having a computer program stored thereon, characterized in that: The computer program is executed by a processor to implement a chromosome image processing method based on an autoencoder as described in any one of claims 1 to 5.