Self-supervised anomaly recognition method and device for cell scatter diagram

Abstract

The invention provides a self-supervised anomaly recognition method and device for a cell scatter diagram. The method comprises the following steps: acquiring the cell scatter diagram of a blood sample to be recognized; inputting the cell scatter diagram into a pre-constructed anomaly recognition model to obtain a recognition result; wherein the anomaly recognition model is finally obtained by training an upstream proxy task based on a deep neural network by using massive cell scatter diagram samples in a self-supervision mode and training a downstream classification task in a supervised mode. According to the method, the anomaly recognition model is trained in a mode of combining self-supervision and supervision, the problem of insufficient data is solved by adopting the generative self-supervision upstream proxy task, and meanwhile, the classification task is trained in a supervised manner on the basis of the upstream proxy task; and the semantic information of the data is fully utilized while excessive marking cost is not needed, so that the trained model can realize cell scatter diagram anomaly recognition with high data utilization rate and high recognition accuracy.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a self-supervised anomaly recognition method and device for cell scattergrams. Background Art

[0002] As an important medical resource for saving lives, blood safety is paramount, and blood screening is the most critical part of ensuring blood safety. Currently, blood screening is mainly performed using blood cell analyzers.

[0003] A hematology analyzer, also known as a hemocytometer, is a conventional instrument that automatically analyzes the heterogeneity of blood cells within a volume of whole blood. It typically consists of a hematology detection module, a hemoglobin measurement module, a mechanical module, an electronic module, and a computer system. The principles typically employed include electrical impedance, colorimetry, and flow cytometry. Current hematology analyzers utilize the most advanced screw drive technology, avoiding the temperature-dependent drawbacks of traditional drive methods (such as belt drives), resulting in more accurate and durable quantification. The general principle is to disrupt red blood cells with a hemolytic agent, while also slightly damaging the white blood cell membrane. Side scattered light (SSC) is then used to characterize the internal structural complexity of different white blood cells. Fluorescent staining, coupled with side fluorescence (SFL), allows for the identification of differences in the types and abundance of nucleic acids and organelles in different white blood cells. This allows for the differential counting of each type of white blood cell and statistical analysis of the measured cells. A WDF scatter plot is generated with SSC as the horizontal axis and SFL as the vertical axis. Among them, the statistical types mainly include the following cells: ghost cells / cell fragments (Debris), nucleated red blood cells (NRBC), lymphocytes (LYMPH), abnormal lymphocytes / prolymphocytes (Abnormal lymph Blast), atypical lymphocytes / atypical lymphocytes (Atypically lymph), monocytes (MONO), primitive cells (Blast), neutrophils and basophils (NEUT+BASO), left shift (Left shift), immature granulocytes (IG) and eosinophils (EO), such as Figure 2 shown.

[0004] Scatter plot-based anomaly detection is an effective means of improving the performance of blood cell analyzers. Currently, scatter plot-based anomaly detection is mainly implemented through neural networks. Existing neural network training methods are mainly divided into three categories: supervised tasks, semi-supervised tasks, and unsupervised tasks.

[0005] In supervised tasks, the data has clear labels of abnormal or normal. This is typically handled as a binary classification task, but this faces the challenges of expensive data labeling and the scarcity of outliers in practice. Consequently, some studies have adopted semi-supervised tasks for training. In these tasks, the data has clear labels for normal samples, but no labels for abnormal classes. However, due to the nature of the problem, obtaining large amounts of abnormal data (either labeled or unlabeled) is difficult during training. Due to limited access to abnormal data, anomaly detectors typically use only normal data in a semi-supervised or one-class classification setting. In this case, a proxy task is often introduced to achieve self-supervision of the anomaly detection task. This can lead to the generative model declaring anomalies when the probability density falls below a certain threshold. Furthermore, because the anomaly score in these tasks is defined as the set of pixel-level reconstruction errors or probability densities, this approach fails to capture high-level semantic information. Consequently, some studies have adopted unsupervised tasks for training. In these cases, all data is unlabeled. In this case, feature extraction is performed on the data, followed by classification using methods similar to odometry. Feature extraction can be considered a generative task. However, state-of-the-art methods rely on deep autoencoders (AEs) or convolutional autoencoders (CAEs). This results in ineffective representation learning in AEs / CAEs for easy-to-implement unsupervised anomaly detection models based on DNNs, leading to insufficient data utilization and compromising model accuracy.

[0006] In summary, the existing technology has the problems of insufficient data utilization and low recognition accuracy. Summary of the Invention

[0007] The present invention provides a self-supervised anomaly recognition method and device for cell scatter plots, which are used to solve the defects of insufficient data utilization and low recognition accuracy in the prior art, and realize cell scatter plot anomaly recognition with high data utilization and high recognition accuracy.

[0008] The present invention provides a self-supervised anomaly recognition method for cell scatter plots, comprising:

[0009] obtaining a cell scattergram of a blood sample to be identified;

[0010] Inputting the cell scattergram into a pre-built abnormality recognition model to obtain a recognition result;

[0011] Among them, the anomaly recognition model is finally obtained by using a deep neural network to use massive cell scatter plot samples to train the upstream agent task in a self-supervised manner, and to train the downstream classification task in a supervised manner.

[0012] According to the present invention, a self-supervised anomaly recognition method for cell scatter plots is provided. Based on a deep neural network, a large number of cell scatter plot samples are used to train upstream agent tasks in a self-supervised manner, and downstream classification tasks are trained in a supervised manner to finally obtain the anomaly recognition model. Specifically, the method includes:

[0013] Obtain massive cell scatter plot samples;

[0014] Dividing the cell scattergram sample into a preset number of image blocks according to a preset splitting rule, and obtaining a splitting result corresponding to the cell scattergram sample;

[0015] Visually marking each of the image blocks in the splitting result;

[0016] Randomly masking a preset proportion of image blocks of the splitting result to obtain mask samples;

[0017] Inputting the mask sample into a mask vision model built based on a deep neural network, with the goal of restoring the visual mark of the masked image block in the mask sample, and training the mask vision model using a preset training target to obtain an upstream model;

[0018] Building a basic classification model based on the deep neural network and the upstream model;

[0019] Classifying and labeling a small number of the cell scatter plot samples to obtain a training data set;

[0020] The basic classification model is subjected to supervised classification training using the training data set until a first end condition is met, thereby obtaining an anomaly recognition model.

[0021] According to a self-supervised anomaly recognition method for cell scattergrams provided by the present invention, the mask sample is input into a mask vision model constructed based on a deep neural network, with the goal of restoring the visual mark of the masked image block in the mask sample. The mask vision model is trained using a preset training target to obtain an upstream model, which specifically includes:

[0022] Constructing a masked vision model based on a deep neural network with a self-attention mechanism; wherein the masked vision model includes an encoder and a decoder;

[0023] replacing the masked image block in the mask sample with an embedding vector, and inputting the replaced mask sample into the mask vision model to obtain an encoded representation of the image block;

[0024] Predicting a visual label of the masked image block using a softmax classifier according to the encoded representation of the masked image block to obtain a visual label prediction result;

[0025] According to the visual marker prediction result, the mask vision model is trained using the preset training target until a second end condition is met to obtain an upstream model.

[0026] According to a self-supervised anomaly recognition method for cell scatter plots provided by the present invention, the preset training objectives include:

[0027]

[0028] Where D is the training corpus composed of all cell scatter plot samples, M represents the position of the random mask, and x M Represents the mask sample obtained according to M, z i represents the visual label of the i-th image block, p MIM (z i |x M ) represents the visual label prediction result.

[0029] According to a self-supervised anomaly recognition method for cell scatter plots provided by the present invention, the basic classification model is constructed based on the deep neural network and the upstream model, specifically comprising:

[0030] A fully connected layer is added to the upstream model based on a deep neural network to obtain a basic classification model.

[0031] According to a self-supervised anomaly recognition method for cell scatter plots provided by the present invention, the mask vision model is trained using the preset training target according to the visual marker prediction result until the second end condition is met to obtain an upstream model, specifically comprising:

[0032] comparing the visual marker prediction result and the corresponding visual marker of the masked image block to obtain a prediction difference;

[0033] Based on the preset training objective, the mask vision model is trained with the goal of minimizing the prediction difference until a second end condition is met, thereby obtaining an upstream model.

[0034] The present invention also provides a self-supervised anomaly recognition device for cell scattergrams, comprising:

[0035] an acquisition unit, configured to acquire a cell scattergram of a blood sample to be identified;

[0036] an identification unit, configured to input the cell scattergram into a pre-built abnormality identification model to obtain an identification result;

[0037] The training unit is used to train the upstream agent task in a self-supervised manner based on a deep neural network using massive cell scatter plot samples, and to train the downstream classification task in a supervised manner to finally obtain an anomaly recognition model.

[0038] The present invention also provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the self-supervised anomaly identification method for cell scatter plots as described above is implemented.

[0039] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described self-supervised anomaly identification methods for cell scatter plots.

[0040] The present invention also provides a computer program product, comprising a computer program, which, when executed by a processor, implements any of the above-described self-supervised anomaly identification methods for cell scatter plots.

[0041] The present invention provides a self-supervised anomaly recognition method and device for cell scatter plots. The method and device obtain a cell scatter plot of a blood sample to be identified, input the cell scatter plot into a pre-built anomaly recognition model, and obtain an identification result. The anomaly recognition model is based on a deep neural network and uses a large number of cell scatter plot samples to train upstream proxy tasks in a self-supervised manner, and then trains downstream classification tasks in a supervised manner. The present invention trains the anomaly recognition model through a combination of self-supervision and supervision. The problem of insufficient data is solved by adopting a generative self-supervised upstream proxy task. At the same time, the classification task is supervised based on the upstream proxy task. This method does not require excessive labeling costs while fully utilizing the semantic information of the data, so that the trained model can achieve high data utilization and high recognition accuracy in cell scatter plot anomaly recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0043] Figure 1 This is one of the flow charts of the self-supervised anomaly recognition method for cell scatter plots provided by the present invention;

[0044] Figure 2It is a scatter diagram schematic diagram of the prior art provided by the present invention;

[0045] Figure 3 This is the second flow chart of the self-supervised anomaly recognition method for cell scatter plots provided by the present invention;

[0046] Figure 4 This is the third flow chart of the self-supervised anomaly recognition method for cell scatter plots provided by the present invention;

[0047] Figure 5 is an example of abnormal scatter plot distribution and normal scatter plot distribution of the self-supervised abnormality recognition method for cell scatter plots provided by the present invention; wherein, Figure 5 (a) and Figure 5 (b) is the abnormal scatter plot distribution, Figure 5 (c) and Figure 5 (d) is the normal scatter plot distribution;

[0048] Figure 6 Schematic diagram of the structure of the self-supervised anomaly recognition device for cell scattergrams provided by the present invention;

[0049] Figure 7 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0050] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0051] The following combination Figure 1-Figure 5 Describe the self-supervised anomaly recognition method for cell scatter plots of the present invention, Figure 1 This is one of the flow charts of the self-supervised anomaly recognition method for cell scatter plots provided by the present invention, such as Figure 1 As shown, the method includes the following steps:

[0052] Step 110: Obtain a cell scattergram of the blood sample to be identified.

[0053] It should be noted that the cell scatter plot of the blood sample to be identified is obtained, and the cell scatter plot includes a WDF scatter plot. It should be noted that the cell scatter plot of the blood sample to be identified can be obtained using a blood cell analyzer or other detection instruments, and the present invention is not limited thereto. The present invention can achieve anomaly identification for any image input that meets the requirements of the anomaly identification model, and has high applicability.

[0054] In some embodiments, the cell scattergrams obtained in the embodiments of the present invention are classified into the classification methods in the prior art (such as Figure 2 ) has been streamlined to include five categories: Debris (ghost cells / cell debris); NRBC (nucleated red blood cells); LYMPH (lymphocytes); Abnormal lymphocyte blast (abnormal lymphocyte / prolymphocyte); and Atypical lymphocyte / atypical lymphocyte. Each color represents a specific cellular component. Furthermore, each of the five statistical cell types is analyzed independently. For each type, side scattered light (SSC) is used to measure the complexity of the internal structure of different white blood cells, while side fluorescence (SFL) is used to measure the type and number of nucleic acids and organelles in different white blood cells. Cell counts are differentiated using SSC as the horizontal axis and SFL as the vertical axis to construct a cell scatter plot.

[0055] The purpose of the present invention is to evaluate abnormal cells from the perspective of scatter plot data distribution. It is understandable that in blood cell instruments, scatter plots are usually used as a risk assessment factor to alleviate the missed detection of abnormal blood routine conditions. Generally speaking, the accuracy of current instruments is 70%, so 30% requires manual review. However, since the abnormal cell environment in the scatter plot will be significantly different from that of normal cells, such as Figure 5 As shown, Figure 5 (a) and (b) are abnormal scatter plot distributions. Figure 5 (c) and (d) are scatter plots with normal distribution. Therefore, if problems can be detected immediately from the scatter plot of blood routine results, it will greatly save the analysis and review time of the results and improve work efficiency. The present invention aims to distinguish abnormal samples from the perspective of image classification.

[0056] Step 120: inputting the cell scattergram into a pre-built abnormality recognition model to obtain a recognition result;

[0057] Understandably, block-wise masks are also widely used in BERT-like models for multi-task learning. However, directly using pixel-level autoencoders for visual pre-training forces the model to focus on local correlations and high-frequency details. Furthermore, scatter plot data naturally lacks semantic information, making it difficult to extract meaningful features using the aforementioned deep learning methods. Furthermore, due to the high computational performance required for deep learning, the blood analyzer under development would need to include a graphics card, which increases the portability and cost of the machine.

[0058] On this basis, the anomaly recognition model of the present invention is obtained by using a deep neural network to train upstream agent tasks in a self-supervised manner using massive cell scatter plot samples, and then training downstream classification tasks in a supervised manner.

[0059] It should be explained that the self-supervised task of the present invention refers to the mask language modeling task. First, a certain proportion of the labels in the random mask sample are restored, and then the mask labels are restored based on the encoding results of the damaged text. However, it is challenging to directly pre-train the image data in the same way as the mask text. It is understandable that for cell scatter plots, there is no pre-existing visual transformer vocabulary input unit, that is, the image block. Therefore, a softmax classifier cannot be simply used to predict all possible candidate mask blocks. A simple solution is to regard the task as a regression problem, which aims to predict the original pixels of the mask patch. However, this pixel-level restoration task often wastes modeling capabilities on pre-training short-distance dependencies and high-frequency details, which is not conducive to reducing computational performance and improving accuracy. On this basis, the present invention visually marks the cell scatter plot samples while masking, and uses visual labels to assist in the recovery of mask labels, thereby smoothly realizing the pre-training of the visual transformer, that is, the training of the self-supervised upstream agent task. In downstream tasks, the parameters of the pre-trained transformer are imported, and the scatter plot is classified into two categories based on the labeled normal and abnormal data to finally complete the training of the anomaly recognition model.

[0060] The following further describes the training of the anomaly recognition model. In some embodiments, the anomaly recognition model is ultimately obtained by using a deep neural network to train upstream proxy tasks in a self-supervised manner using massive cell scatter plot samples, and then using a supervised method to train downstream classification tasks, specifically including:

[0061] Step 210: Obtaining a large number of cell scatter plot samples;

[0062] Step 220: Divide the cell scattergram sample into a preset number of image blocks according to a preset splitting rule, and obtain a splitting result corresponding to the cell scattergram sample;

[0063] Step 230: visually marking each of the image blocks in the segmentation result;

[0064] Step 240: randomly masking a preset proportion of image blocks of the split result to obtain mask samples;

[0065] Step 250: Inputting the mask sample into a mask vision model built based on a deep neural network, with the goal of restoring the visual labels of the masked image blocks in the mask sample, and training the mask vision model using a preset training target to obtain an upstream model;

[0066] Step 260: Constructing a basic classification model based on the deep neural network and the upstream model;

[0067] Step 270: Classify and label a small number of the cell scattergram samples to obtain a training data set;

[0068] Step 280: Perform supervised classification training on the basic classification model using the training data set until a first end condition is met, thereby obtaining an anomaly recognition model.

[0069] Specifically, for the massive cell scatter plot samples obtained, the embodiment of the present invention first proposes a pre-training task, namely, a mask visual model. Figure 3 As shown, the masked vision model uses two views for each cell scatter plot sample image: image segmentation and visual labeling. In step 220, each cell scatter plot sample image is divided into a predetermined number of small block grids, i.e., image blocks. These serve as the segmentation results corresponding to the cell scatter plot sample and are the input representations of the backbone model.

[0070] It should be noted that the two-dimensional image cell scattergram sample is divided into a series of small blocks, and the block format is image data that can be directly accepted by a standard transformer, and the present invention does not impose any limitation on this.

[0071] Furthermore, in form, the embodiment of the present invention converts the cell scattergram sample Substitute N = HW / P 2 patch That is, the cell scatter plot samples Split into N patches to get image blocks Where C is the number of channels, (H, W) is the input image resolution, and (P, P) is the resolution of each block. are mapped into vectors and linearly projected, which is similar to word embeddings in BERT. Image patches retain their original pixels and are used as input features in BEIT.

[0072] In a specific embodiment, each 224×224 cell scattergram sample image is segmented into a 14×14 image block grid, where each block is 16×16.

[0073] In addition, in step 230, the embodiment of the present invention "segmentation marks" the image into discrete visual markers (tokens). In actual operation, it can be equivalent to visually marking each image block in the split result corresponding to each cell scatter plot sample. Based on the above embodiment, in a specific embodiment, each cell scatter plot sample image is marked with a 14×14 grid of visual markers. It should be noted that the number of visual markers and the number of image blocks of an image are the same. In this embodiment, the vocabulary size is set to |V|=8192.

[0074] Further, in some embodiments, using [RPG + The publicly available image labeler in

[21] visually labels each image block in the split result corresponding to the cell scatter plot sample.

[0075] After that, the upstream pre-training task can be trained. In step 240, before training, for each cell scatter plot sample, a certain proportion of the image blocks corresponding to the split result are randomly masked to obtain masked image blocks (i.e., masked samples). In one embodiment, 40% of the image blocks are randomly masked to obtain masked samples.

[0076] In step 250, the masked image block (i.e., mask sample) is input into a transformer built based on a deep neural network. The model learns to restore the visual markers of the original image, rather than the pixels of the mask patch of the original image, and finally obtains the upstream model. It should be explained that during the training process of step 250, the present invention imitates the denoising codec network, encodes and compresses the lossy image, and then reconstructs the image by decoding the intermediate features, so that the reconstructed image is as similar as possible to the lossless image.

[0077] The present invention solves the problem of insufficient data in the prior art by adopting a generative self-supervised upstream proxy task.

[0078] Furthermore, in step 260, a basic classification model is constructed based on the deep neural network and the upstream model. In the specific implementation process, the parameters of the upstream task are frozen and unchanged, and the classification model is constructed based on the upstream model.

[0079] In some embodiments, constructing a basic classification model based on a deep neural network and the upstream model specifically includes:

[0080] A fully connected layer is added to the upstream model based on a deep neural network to obtain a basic classification model.

[0081] Specifically, supervised training is performed by adding a classification layer, i.e., a fully connected layer, to the upstream task. In other words, the model of the downstream task (the base classification model) is obtained by adding a fully connected layer to the model of the upstream task.

[0082] Next, in step 270, a small number of cell scatter plot samples are classified and labeled, and a training dataset is constructed using this small amount of labeled data. In step 280, the basic classification model is trained in a supervised classification manner using the training dataset until the first termination condition is met, thereby obtaining an anomaly recognition model. It is important to note that during the training of the downstream task, the parameters of the upstream task model are frozen and unchanged, and only the newly added classification layer is trained to obtain the anomaly recognition model.

[0083] Based on the above embodiment, step 250 is further described below. In some embodiments, the mask sample is input into a mask vision model constructed based on a deep neural network, with the goal of restoring the visual mark of the masked image block in the mask sample. The mask vision model is trained using a preset training target to obtain an upstream model, specifically including:

[0084] Step 310: Constructing a masked vision model based on a deep neural network with a self-attention mechanism; wherein the masked vision model includes an encoder and a decoder;

[0085] Step 320: replacing the masked image block in the mask sample with the embedding vector, and inputting the replaced mask sample into the mask vision model to obtain an encoded representation of the image block;

[0086] Step 330: predicting a visual label of the masked image block using a softmax classifier based on the encoded representation of the masked image block to obtain a visual label prediction result;

[0087] Step 340: Based on the visual marker prediction result, the mask vision model is trained using the preset training target until a second end condition is met to obtain an upstream model.

[0088] Specifically, in step 310, a deep neural network (Transformer) based on a self-attention mechanism is first used to construct a mask vision model. It should be noted that the mask vision model adopts an encoder-decoder architecture. It is understandable that the Transformer model usually adopts an encoder-decoder architecture. The encoder consists of multiple encoding layers for processing the input sequence and generating an internal representation; the decoder also consists of multiple decoding layers for generating an output sequence, and can pay attention to the output of the encoder during the generation process.

[0089] In the upstream proxy task of the mask vision model, it is necessary to randomly mask a certain proportion of image blocks and then predict the visual labels corresponding to the masked blocks. More specifically, in the implementation process, for a given input cell scatter map sample x, it is split into N image blocks and segment and mark it as visual markers Randomly mask a certain proportion of the image block, where the position of the mask is represented by M∈{1,…,N} kN (k is the masking ratio). Next, in step 320, a learnable embedding Replace the masked block. Then the damaged block (ie mask sample) It is fed into the L-layer mask vision model based on Transformer, such as Figure 4 As shown. Finally, the hidden vector (potential representation) is obtained It can be viewed as an encoded representation of the input image block.

[0090] Further, in step 330, for each mask position Use the softmax classifier to predict the corresponding alternative visual tags and obtain the visual tag prediction results where x M is the corrupted image (mask sample), z' represents the target label, W c represents the weight matrix of the linear transformation, b c Represents the bias of the linear transformation. The weight matrix and bias of the linear transformation are used to map the hidden state to the logits space.

[0091] It should be emphasized that during the training of the upstream task, the pre-training goal is to maximize the log-likelihood of the correct visual token in the corrupted image.

[0092] In some embodiments, during this process, the preset training goals include:

[0093]

[0094] Where D is the training corpus composed of all cell scatter plot samples, M represents the position of the random mask, and x M Represents the mask sample obtained according to M, z i represents the visual label of the i-th image block, p MIN (z i |x M ) represents the visual label prediction result.

[0095] Further, step 340 is further described below. In some embodiments, the mask vision model is trained using the preset training target according to the visual marker prediction result until a second end condition is met to obtain an upstream model, specifically including:

[0096] Step 341: Compare the visual marker prediction result and the corresponding visual marker of the masked image block to obtain a prediction difference;

[0097] Step 342: Based on the preset training objective, the mask vision model is trained with the goal of minimizing the prediction difference until a second end condition is met to obtain an upstream model.

[0098] The goal of steps 341 and 342 is to make the visual mark prediction result (z') predicted by the mask visual model as close as possible to the original visual mark (z i ), that is, the prediction result of maximizing the output of the mask vision model is the visual label (z i ) probability.

[0099] In a specific embodiment, the second end condition is that the pre-training runs for 500k steps (i.e., 800 epochs) with a batch size of 2k. In other words, this means that the upstream model performs a total of 500,000 iterations during the pre-training process (i.e., the upstream model is fully trained 800 times on the entire dataset) and each iteration (step) uses 2000 samples to form a batch for training. During the training process, the Adam optimization algorithm is used to adjust the network weights to minimize the loss function, where the first hyperparameter β1 in the Adam algorithm is 0.9, and the second hyperparameter β2 in the Adam algorithm is 0.999. The algorithm sets the learning rate arat to 1.5e-3, the number of training rounds awarm uB to 10 epochs, and the cosine learning rate decay weight decay to 0.05. This embodiment uses random depth [HSL+16] and a rate of 0.1, and disables it.

[0100] It should be noted that this embodiment uses 16 Nvidia Telsa V100 32GB GPU cards, and 500k training steps take about 5 days.

[0101] Furthermore, based on the above embodiment, to increase the stability of the Transformer, before pre-training the upstream task, the Transformer parameters are first randomly initialized within a very small range. In a specific embodiment, the very small range mentioned in this embodiment can be set to [-0.02, 0.02].

[0102] The present invention provides a self-supervised anomaly recognition method for cell scatter plots, which obtains a cell scatter plot of a blood sample to be identified; inputs the cell scatter plot into a pre-built anomaly recognition model to obtain an identification result; wherein the anomaly recognition model is based on a deep neural network and uses a large number of cell scatter plot samples to train an upstream agent task in a self-supervised manner, and trains a downstream classification task in a supervised manner. The present invention trains the anomaly recognition model through a combination of self-supervision and supervision, solves the problem of insufficient data by adopting a generative self-supervised upstream agent task, and simultaneously performs supervised training on the classification task based on the upstream agent task, without requiring excessive labeling costs while making full use of the semantic information of the data, so that the trained model can achieve cell scatter plot anomaly recognition with high data utilization and high recognition accuracy.

[0103] The following describes the self-supervised anomaly identification device for cell scatter plots provided by the present invention. The self-supervised anomaly identification device for cell scatter plots described below and the self-supervised anomaly identification method for cell scatter plots described above can be used for reference. Figure 6 As shown, the device includes the following modules:

[0104] An acquisition unit 610 is configured to acquire a cell scattergram of a blood sample to be identified;

[0105] The recognition unit 620 is used to input the cell scattergram into a pre-built abnormality recognition model to obtain a recognition result;

[0106] The training unit 630 is used to train the upstream agent task in a self-supervised manner based on a deep neural network using massive cell scatter plot samples, and to train the downstream classification task in a supervised manner to finally obtain an anomaly recognition model.

[0107] According to the present invention, a self-supervised anomaly recognition device for cell scatter plots is provided. Based on a deep neural network, a large number of cell scatter plot samples are used to train upstream agent tasks in a self-supervised manner, and downstream classification tasks are trained in a supervised manner to finally obtain the anomaly recognition model. Specifically, the device includes:

[0108] Obtain massive cell scatter plot samples;

[0109] Dividing the cell scattergram sample into a preset number of image blocks according to a preset splitting rule, and obtaining a splitting result corresponding to the cell scattergram sample;

[0110] Visually marking each of the image blocks in the splitting result;

[0111] Randomly masking a preset proportion of image blocks of the splitting result to obtain mask samples;

[0112] Inputting the mask sample into a mask vision model built based on a deep neural network, with the goal of restoring the visual mark of the masked image block in the mask sample, and training the mask vision model using a preset training target to obtain an upstream model;

[0113] Building a basic classification model based on the deep neural network and the upstream model;

[0114] Classifying and labeling a small number of the cell scatter plot samples to obtain a training data set;

[0115] The basic classification model is subjected to supervised classification training using the training data set until a first end condition is met, thereby obtaining an anomaly recognition model.

[0116] According to a self-supervised anomaly recognition device for cell scattergrams provided by the present invention, the mask sample is input into a mask vision model constructed based on a deep neural network, with the goal of restoring the visual mark of the masked image block in the mask sample. The mask vision model is trained using a preset training target to obtain an upstream model, which specifically includes:

[0117] Constructing a masked vision model based on a deep neural network with a self-attention mechanism; wherein the masked vision model includes an encoder and a decoder;

[0118] replacing the masked image block in the mask sample with an embedding vector, and inputting the replaced mask sample into the mask vision model to obtain an encoded representation of the image block;

[0119] Predicting a visual label of the masked image block using a softmax classifier according to the encoded representation of the masked image block to obtain a visual label prediction result;

[0120] According to the visual marker prediction result, the mask vision model is trained using the preset training target until a second end condition is met to obtain an upstream model.

[0121] According to a self-supervised anomaly recognition device for cell scatter plots provided by the present invention, the preset training objectives include:

[0122]

[0123] Where D is the training corpus composed of all cell scatter plot samples, M represents the position of the random mask, and x M Represents the mask sample obtained according to M, z i represents the visual label of the i-th image block, p MIM (zi |x M ) represents the visual label prediction result.

[0124] According to a self-supervised anomaly recognition device for cell scatter plots provided by the present invention, the basic classification model is constructed based on the deep neural network and the upstream model, specifically comprising:

[0125] A fully connected layer is added to the upstream model based on a deep neural network to obtain a basic classification model.

[0126] According to a self-supervised anomaly recognition device for cell scattergrams provided by the present invention, the mask vision model is trained using the preset training target according to the visual marker prediction result until the second end condition is met to obtain an upstream model, specifically comprising:

[0127] comparing the visual marker prediction result and the corresponding visual marker of the masked image block to obtain a prediction difference;

[0128] Based on the preset training objective, the mask vision model is trained with the goal of minimizing the prediction difference until a second end condition is met, thereby obtaining an upstream model.

[0129] The present invention provides a self-supervised anomaly recognition device for cell scatter plots. The device obtains a cell scatter plot of a blood sample to be identified, inputs the cell scatter plot into a pre-built anomaly recognition model, and obtains an identification result. The anomaly recognition model is based on a deep neural network and uses a large number of cell scatter plot samples to train upstream proxy tasks in a self-supervised manner, and then trains downstream classification tasks in a supervised manner. The present invention trains the anomaly recognition model through a combination of self-supervision and supervision, solves the problem of insufficient data by adopting a generative self-supervised upstream proxy task, and simultaneously performs supervised training on the classification task based on the upstream proxy task. This eliminates the need for excessive labeling costs while fully utilizing the semantic information of the data, so that the trained model can achieve high data utilization and high recognition accuracy in cell scatter plot anomaly recognition.

[0130] Figure 7 An example of a physical structure diagram of an electronic device is shown below. Figure 7As shown, the electronic device may include: a processor 710, a communication interface 720, a memory 730, and a communication bus 740, wherein the processor 710, the communication interface 720, and the memory 730 communicate with each other via the communication bus 740. The processor 710 may call the logic instructions in the memory 730 to execute a self-supervised abnormality recognition method for cell scattergrams, the method comprising: obtaining a cell scattergram of a blood sample to be identified; inputting the cell scattergram into a pre-built abnormality recognition model to obtain a recognition result; wherein the abnormality recognition model is based on a deep neural network, using a large number of cell scattergram samples to train the upstream agent task in a self-supervised manner, and using a supervised manner to train the downstream classification task.

[0131] In addition, the logic instructions in the above-mentioned memory 730 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0132] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the self-supervised abnormality recognition method for cell scatter plots provided by the above methods, the method including: obtaining a cell scatter plot of a blood sample to be identified; inputting the cell scatter plot into a pre-built abnormality recognition model to obtain an identification result; wherein, the abnormality recognition model is based on a deep neural network and uses massive cell scatter plot samples to train the upstream agent task in a self-supervised manner, and uses a supervised manner to train the downstream classification task.

[0133] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the self-supervised abnormality recognition method for cell scatter plots provided by the above-mentioned methods, the method comprising: obtaining a cell scatter plot of a blood sample to be identified; inputting the cell scatter plot into a pre-built abnormality recognition model to obtain an identification result; wherein the abnormality recognition model is based on a deep neural network and uses massive cell scatter plot samples to train upstream agent tasks in a self-supervised manner, and uses a supervised manner to train downstream classification tasks.

[0134] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0135] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments.

[0136] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A self-supervised anomaly recognition method for cell scatter plots, characterized in that: include: Obtaining a cell scatter plot of a blood sample to be identified; Inputting the cell scattergram into a pre-built abnormality recognition model to obtain a recognition result; Among them, the anomaly recognition model is finally obtained by using a deep neural network to use massive cell scatter plot samples to train the upstream agent task in a self-supervised manner, and to train the downstream classification task in a supervised manner.

2. The self-supervised anomaly recognition method for cell scatter plots according to claim 1, characterized in that: The anomaly recognition model is finally obtained by using a deep neural network to train the upstream agent task in a self-supervised manner using massive cell scatter plot samples, and then using a supervised method to train the downstream classification task. Specifically, it includes: Obtain massive cell scatter plot samples; Dividing the cell scattergram sample into a preset number of image blocks according to a preset splitting rule, and obtaining a splitting result corresponding to the cell scattergram sample; Visually marking each of the image blocks in the splitting result; Randomly masking a preset proportion of image blocks of the splitting result to obtain mask samples; Inputting the mask sample into a mask vision model built based on a deep neural network, with the goal of restoring the visual mark of the masked image block in the mask sample, and training the mask vision model using a preset training target to obtain an upstream model; Building a basic classification model based on the deep neural network and the upstream model; Classifying and labeling a small number of the cell scatter plot samples to obtain a training data set; The basic classification model is subjected to supervised classification training using the training data set until a first end condition is met, thereby obtaining an anomaly recognition model.

3. The self-supervised anomaly recognition method for cell scatter plots according to claim 2, characterized in that: The mask sample is input into a mask vision model built based on a deep neural network, with the goal of restoring the visual mark of the masked image block in the mask sample. The mask vision model is trained using a preset training target to obtain an upstream model, specifically including: Constructing a masked vision model based on a deep neural network with a self-attention mechanism; wherein the masked vision model includes an encoder and a decoder; replacing the masked image block in the mask sample with an embedding vector, and inputting the replaced mask sample into the mask vision model to obtain an encoded representation of the image block; Predicting a visual label of the masked image block using a softmax classifier according to the encoded representation of the masked image block to obtain a visual label prediction result; According to the visual marker prediction result, the mask vision model is trained using the preset training target until a second end condition is met to obtain an upstream model.

4. The self-supervised anomaly recognition method for cell scatter plots according to claim 3, characterized in that: The preset training objectives include: Where D is the training corpus composed of all cell scatter plot samples, M represents the position of the random mask, and x M Represents the mask sample obtained according to M, z i represents the visual label of the i-th image block, p MIM (z i |x M ) represents the visual label prediction result.

5. The self-supervised anomaly recognition method for cell scatter plots according to claim 2, characterized in that: The construction of a basic classification model based on the deep neural network and the upstream model specifically includes: A fully connected layer is added to the upstream model based on a deep neural network to obtain a basic classification model.

6. The self-supervised anomaly recognition method for cell scatter plots according to claim 2, characterized in that: According to the visual marker prediction result, the mask vision model is trained using the preset training target until a second end condition is met to obtain an upstream model, specifically including: comparing the visual marker prediction result and the corresponding visual marker of the masked image block to obtain a prediction difference; Based on the preset training objective, the mask vision model is trained with the goal of minimizing the prediction difference until a second end condition is met, thereby obtaining an upstream model.

7. A self-supervised anomaly recognition device for cell scatter plots, characterized in that: include: an acquisition unit, configured to acquire a cell scattergram of a blood sample to be identified; an identification unit, configured to input the cell scattergram into a pre-built abnormality identification model to obtain an identification result; The training unit is used to train the upstream agent task in a self-supervised manner based on a deep neural network using massive cell scatter plot samples, and to train the downstream classification task in a supervised manner to finally obtain an anomaly recognition model.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the self-supervised anomaly recognition method for cell scatter plots as described in any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for self-supervised anomaly recognition for cell scattergrams according to any one of claims 1 to 6 is implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method for self-supervised anomaly recognition for cell scattergrams according to any one of claims 1 to 6 is implemented.

Images