An iterative method and device for cell morphology inspection system based on incremental learning

Abstract

The present application relates to the field of image analysis technology, and to an iterative method and device for a cell morphology inspection system based on incremental learning. The method comprises: acquiring image data of target cells, and constructing a first data set according to the image data; inputting the first data set into the out-of-distribution sample detection module to obtain a new category cell data set within the first data set; performing iterative rounds of incremental learning on the cell morphology inspection system according to each of the new cell data to update the attribute learning memory library and the out-of-distribution sample detection module, until each of the new cell data is marked as cell data that has been learned and memorized, and outputting the final round of the target cell morphology inspection system. On the one hand, the present application can improve the accuracy of identifying new categories of cells, and on the other hand, it can maintain the accuracy of identifying old categories of cells, and achieve incremental learning of new categories of cells without forgetting old categories of cells.

Description

Technical Field

[0001] The present application relates to the field of image analysis technology, and in particular, to an iterative method and device for a cell morphology inspection system based on incremental learning. Background Art

[0002] In the field of cytological morphological analysis, due to the large number of cell categories and the emergence of various different morphologies, the existing intelligent blood cell morphology testing system faces the problems of constantly adding new cell categories and weak ability to analyze and learn cell attribute characteristics during application. Therefore, the blood cell morphology testing system needs to continuously and repeatedly learn a large amount of original cell data during training and learning, which leads to the high time cost and low training efficiency required for training the blood cell morphology testing system. Summary of the Invention

[0003] The present application provides an iterative method and device for a cell morphology inspection system based on incremental learning to solve one or more technical problems existing in the prior art and at least provide a beneficial option or create conditions.

[0004] Other features and advantages of the present application will become apparent from the following detailed description, or may be learned in part by practice of the present application.

[0005] According to one aspect of an embodiment of the present application, an iterative method for a cell morphology inspection system based on incremental learning is provided, which is applied to a cell morphology inspection system. The cell morphology inspection system includes a preset out-of-distribution sample detection module and an attribute learning memory library. The attribute learning memory library stores a plurality of learned and memorized cell data. The method includes:

[0006] S1, acquiring image data of target cells and constructing a first data set based on the image data;

[0007] S2, inputting the first data set into the out-of-distribution sample detection module to obtain a new category cell data set within the first data set, wherein the new category cell data set includes a plurality of new cell data, and the new cell data is used to represent cell data that has not been learned and memorized;

[0008] S3, performing iterative rounds of incremental learning on the cell morphology inspection system according to each of the new cell data to update the attribute learning memory library and the out-of-distribution sample detection module, until each of the new cell data is marked as learned and memorized cell data, and outputting a final round of target cytological morphology inspection system;

[0009] The incremental learning of the current round of the cell morphology inspection system includes the following steps:

[0010] Obtaining the target new cell data of the current round, and performing incremental learning on the cell morphology inspection system outputted in the previous round according to each cell attribute of the target new cell data, until each cell attribute of the target new cell data is incrementally learned, thereby obtaining a feature fusion vector of the target new cell;

[0011] The attribute learning memory library and the out-of-distribution sample detection module are updated according to the feature fusion vector to output the cell morphology inspection system of the current round.

[0012] In one embodiment of the present application, based on the aforementioned scheme, the incremental learning includes attribute incremental learning and category incremental learning, wherein the attribute incremental learning is used to characterize the incremental learning of attribute characteristics of each cell attribute of the target new cell data, and the category incremental learning is used to characterize the incremental learning of category characteristics of the target new cell data.

[0013] In one embodiment of the present application, based on the aforementioned solution, performing incremental learning on the cell morphology inspection system output from the previous round according to the various cell attributes of the target new cell data includes:

[0014] Obtain the previous attribute learning memory library of the cell morphology inspection system output in the previous round;

[0015] Determining, based on each cell attribute of the target new cell data, a plurality of similarity weight data between each learned and memorized cell data in the previous attribute learning and memory library, to obtain a similarity weight data set;

[0016] For each cell attribute of the target new cell data, obtain a target cell attribute corresponding to the cell attribute in the previous attribute learning and memory library, and perform attribute feature migration and fusion on each of the cell attributes and the cell attributes according to the similarity weight data set to obtain a cell attribute fusion vector, wherein the target cell attribute is the cell attribute learned and memorized in the historical rounds;

[0017] Determine the attribute feature vector of the target new cell data according to each of the cell attribute fusion vectors;

[0018] A feature fusion vector is determined based on the attribute feature vector and the attribute feature vectors of each cell data in historical rounds, and the previous attribute learning memory library is updated based on the feature fusion vector to complete the attribute incremental learning of the cell morphology inspection system.

[0019] In one embodiment of the present application, based on the above scheme, the category incremental learning of the cell morphology inspection system is performed by the following steps:

[0020] Binding the feature fusion vector to the target new cell data to determine a mapping relationship between the feature fusion vector and the target new cell data;

[0021] The category features of the target new cell data are determined according to the mapping relationship, and the out-of-distribution sample detection module is updated according to the category features to complete the category incremental learning of the cell morphology inspection system.

[0022] In one embodiment of the present application, based on the aforementioned solution, the method of determining, based on each cell attribute of the target new cell data, a plurality of similarity weight data between each learned and memorized cell data in the previous attribute learning memory library, to obtain a similarity weight data set includes:

[0023] For each cell attribute of the target new cell data, calculating the Hamming distance between the cell attribute and the attribute feature of each cell data in the attribute learning memory library;

[0024] The similarity weight data corresponding to each of the cell attributes of the target new cell data are determined according to each of the Hamming distances, and the similarity weight data set is determined according to each of the similarity weight data.

[0025] In one embodiment of the present application, based on the above scheme, the new type of cell dataset is obtained by the following steps:

[0026] Obtaining the category probability and distance score of each of the image data in the first data set through an out-of-distribution sample detection module;

[0027] For each image data, if the product value of the category probability and the distance score of the image data is greater than a preset threshold, the cell data corresponding to the image data is determined to be the new cell data;

[0028] Wherein, each of the image data corresponds to single cell data.

[0029] In one embodiment of the present application, based on the aforementioned scheme, the cell morphology inspection system is trained according to a preset class balance loss function, a preset cosine distillation loss function, a preset feature enhancement loss function and a preset attribute space separation loss function, and the class balance loss function, the cosine distillation loss function, the feature enhancement loss function and the attribute space separation loss function are used for attribute incremental learning.

[0030] In one embodiment of the present application, based on the aforementioned scheme, the cell morphology inspection system is also trained according to a preset cross entropy loss function and a preset distillation loss function, and the cross entropy loss function and the distillation loss function are used for category incremental learning.

[0031] According to one aspect of an embodiment of the present application, an iterative device for a cell morphology inspection system based on incremental learning is provided, which is applied to a cell morphology inspection system. The cell morphology inspection system includes a preset out-of-distribution sample detection module and an attribute learning memory library. The attribute learning memory library stores a plurality of learned and memorized cell data. The device includes:

[0032] an acquisition unit, configured to acquire image data of target cells and construct a first data set based on the image data;

[0033] an input unit, configured to input the first data set into the out-of-distribution sample detection module to obtain a new category cell data set within the first data set, wherein the new category cell data set includes a plurality of new cell data, and the new cell data is used to represent cell data that has not been learned and memorized;

[0034] an incremental learning unit, configured to perform iterative rounds of incremental learning on the cell morphology inspection system according to each of the new cell data to update the attribute learning memory library and the out-of-distribution sample detection module, until each of the new cell data is marked as learned and memorized cell data, and output a final round of target cytological morphology inspection system;

[0035] The incremental learning of the current round of the cell morphology inspection system includes the following steps:

[0036] Obtaining the target new cell data of the current round, and performing incremental learning on the cell morphology inspection system outputted in the previous round according to each cell attribute of the target new cell data, until each cell attribute of the target new cell data is incrementally learned, thereby obtaining a feature fusion vector of the target new cell;

[0037] The attribute learning memory library and the out-of-distribution sample detection module are updated according to the feature fusion vector to output the cell morphology inspection system of the current round.

[0038] According to one aspect of an embodiment of the present application, a computer-readable storage medium is provided, on which a computer program is stored. The computer program includes executable instructions. When the executable instructions are executed by a processor, the method described in the above embodiment is implemented.

[0039] According to one aspect of an embodiment of the present application, an electronic device is provided, comprising: one or more processors; and a memory for storing executable instructions of the processors, wherein when the executable instructions are executed by the one or more processors, the one or more processors implement the method described in the above embodiments.

[0040] The beneficial effects of the present application are as follows: the present application obtains a new category of cell data set in the first data set through a preset out-of-distribution sample detection module, that is, an out-of-distribution class detection method, that is, screens out new cell data that has not been learned and memorized, and performs iterative rounds of incremental learning on the cell morphology inspection system based on each of the new cell data, wherein one round of incremental learning only needs to use one new cell data. At this time, it can be determined that the new cell data used in the current round of incremental learning is the target new cell data, and then the cell morphology inspection system output in the previous round can be incrementally learned according to each cell attribute of the target new cell data until each cell attribute is incrementally learned, thereby obtaining a feature fusion vector of the target new cell data, so that the entire cell morphology inspection system can complete incremental learning through the feature fusion vector, that is, the attribute learning memory library and the out-of-distribution sample detection module can be updated, so that the next round of incremental learning can be performed on the cell morphology inspection system output in the current round. While not forgetting the old cell data, incremental learning of new cell data is continuously realized, reducing catastrophic forgetting, thereby reducing the consumption of time and computing power, and thus improving the training efficiency of the cell morphology inspection system.

[0041] A target attribute discriminator and a target category classifier are obtained, and the cell attributes and cell categories of each new cell are obtained. The cell attributes and cell categories obtained through learning can be used to iteratively evolve the cytological morphology inspection system, that is, each new cell data is marked as old cell data that has been learned and remembered by the cytological morphology inspection system, and stored in a local memory database. New cell data of new categories are continuously learned without forgetting the old cell data of old categories, and the new cell data obtained by continuous learning and memory are stored in the attribute learning memory library and converted into old cell data, thereby completing the iterative update of the entire cytological morphology inspection system, and solving the problem of weak analysis and learning capabilities in the existing technology.

[0042] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The accompanying drawings are incorporated into and constitute a part of the specification, illustrating embodiments consistent with the present application and, together with the specification, explaining the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application, and those skilled in the art can derive other drawings based on these drawings without inventive effort. In the drawings:

[0044] Figure 1 Flowchart of an iterative method for a cell morphology inspection system based on incremental learning according to an embodiment of the present application;

[0045] Figure 2 This is an overall logic diagram of incremental learning in a cell morphology inspection system according to an embodiment of the present application;

[0046] Figure 3 1 is a logic diagram for calculating the product value of category probability and distance score according to an embodiment of the present application;

[0047] Figure 4 Flowchart for determining new and old cell data according to an embodiment of the present application;

[0048] Figure 5 Schematic diagram of an attribute learning memory library according to an embodiment of the present application;

[0049] Figure 6 Detailed schematic diagram of ILA incremental learning stages 2 and 3 according to an embodiment of the present application;

[0050] Figure 7 Detailed schematic diagram of ILA incremental learning stages one and two according to an embodiment of the present application;

[0051] Figure 8 Schematic diagram of target mapping parameter generation for ILA incremental learning stages one and two according to an embodiment of the present application;

[0052] Figure 9 A simple flowchart of incremental learning according to an embodiment of the present application is shown;

[0053] Figure 10 1 is a block diagram of an iterative cell morphology inspection system based on incremental learning according to an embodiment of the present application;

[0054] Figure 11 4 is a diagram showing the relationship between cell attributes and cell types according to an embodiment of the present application. DETAILED DESCRIPTION

[0055] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this application will be thorough and complete and will fully convey the concepts of the example embodiments to those skilled in the art.

[0056] In addition, described feature, structure or characteristic can be combined in one or more embodiments in any suitable manner.In the following description, many specific details are provided so as to provide a full understanding of the embodiments of the present application. However, it will be appreciated by those skilled in the art that the technical scheme of the present application can be put into practice without one or more of the specific details, or other methods, components, devices, steps etc. can be adopted. In other cases, known methods, devices, implementations or operations are not shown or described in detail to avoid blurring the various aspects of the application.

[0057] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically separate entities. That is, these functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or micro-controller node devices.

[0058] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, while others may be combined or partially combined. Therefore, the actual execution order may vary depending on the actual situation.

[0059] It should be noted that the term "plurality" used in this document refers to two or more. "And / or" describes a relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A alone, A and B together, or B alone. The character " / " generally indicates an "or" relationship between the associated objects.

[0060] The following is a detailed description of the implementation details of the technical solution of the embodiment of the present application:

[0061] According to one aspect of the present application, an iterative method for a cell morphology inspection system based on incremental learning is provided. Figure 1 This is a flowchart of an iterative method for a cell morphology inspection system based on incremental learning according to an embodiment of the present application. The iterative method for a cell morphology inspection system based on incremental learning includes at least steps S1 to S3, which are described in detail as follows:

[0062] In step S1 , image data of target cells are acquired, and a first data set is constructed based on the image data.

[0063] Specifically, during the inspection process, image data is acquired from a blood cell smear using a camera device, and a first data set D1 is constructed based on the acquired image data. The newly acquired first data set D1 is input into the cell morphology inspection system. This first data set D1 contains both old cell category data (i.e., the cell data that has been learned and memorized in the attribute learning memory bank of the cell morphology inspection system output in the previous round) and new cell category data (new cell data), where each new cell data represents a new category of cell. It should be noted that the new cell data includes multiple cell attributes, that is, it has multiple attribute features. The attribute features corresponding to some cell attributes in the new cell data may have been learned in incremental learning in previous rounds, while some cell attributes have not been learned. If some cell attributes have been learned in previous rounds, the learned target cell attributes are transferred and fused with the cell attributes of the current round. If they have not been learned, a high-dimensional extractor is dynamically expanded to extract the attribute features of these unlearned and unmemorized cell attributes.

[0064] It should be noted that the new cell data represents a new category of cell data, which includes multiple cell attribute features, some of which may have been learned, while others may not.

[0065] Therefore, the incremental learning of this application is actually divided into two categories. The first category is attribute incremental learning, which performs incremental learning on cell attributes, and the second category is category incremental learning, which performs incremental learning on category features for new cell data. The new cell data can be represented by t, wherein the incremental learning of cell attributes can be specifically as follows: if the cell attribute has not been learned and memorized in the incremental learning of historical rounds, the cell attribute is used as a new cell attribute. Traverse each cell attribute in the new cell data to obtain one or more new cell attributes, and then construct a corresponding high-dimensional extractor based on the set of these new cell attributes to perform feature extraction of the new cell attribute. If the cell attribute has been learned and memorized in the incremental learning of historical rounds, then find the target cell attribute that has been learned and memorized for the cell attribute in the historical rounds, and perform weighted fusion of these target cell attributes with the cell attributes of the current round to obtain a cell attribute fusion vector corresponding to the current round.

[0066] In step S2, the first data set is input into the out-of-distribution sample detection module to obtain a new category cell data set within the first data set, wherein the new category cell data set includes a plurality of new cell data, and the new cell data is used to represent cell data that has not been learned and memorized.

[0067] Specifically, the out-of-distribution sample detection module is a model network that uses a preset out-of-distribution class detection method for detection. The model network is an out-of-distribution class detection (OCD) based on attribute features (i.e., the cell attributes described in this application) and the KNN (K-NearestNeighbor) algorithm, hereinafter referred to as the OCD method, which can be Figure 2 As shown, the first data set D1 is processed by the OCD method, and the old category cell data and the new category cell data are screened out to obtain a data set D2 containing all the new cell category data. The data set D2 has been labeled with cell categories by the inspector during the morphological inspection process. The active learning method is used to sort the learning value of the data set D2, and the samples with the most learning value are screened out to obtain a number of classified and labeled new category data, and add them to the new category data set d. The new category data set d represents the new category cell data set in this application. The number of samples in the new category cell data set is greater than the threshold T, where T can be set to an arbitrary value. Among them, since the out-of-distribution sample detection module is updating in iterative rounds, each time a new cell data is incrementally learned, the cell category corresponding to the new cell data will be memorized, and the judgment accuracy of the next round of new and old category cell data can be improved.

[0068] In one embodiment of the present application, the new type of cell dataset is obtained by the following steps:

[0069] Obtaining the category probability and distance score of each of the image data in the first data set through an out-of-distribution sample detection module;

[0070] For each image data, if the product value of the category probability and the distance score of the image data is greater than a preset threshold, the cell data corresponding to the image data is determined to be the new cell data;

[0071] Wherein, each of the image data corresponds to single cell data.

[0072] Specifically, combined Figure 3 and Figure 4 As shown, the OCD method is described in detail:

[0073] Stage 1: Feature extraction and normalization

[0074] A convolutional neural network model is used, which has been pre-trained on a dataset of x blood cells. For each sample in the training set, validation set, and test set, the model will output the predicted probability of x categories, and extract the feature vectors output by different networks during the prediction process.

[0075] Feature extraction is divided into several steps:

[0076] Extract the output features of the last block of layer 2 and layer 3 in the model. These features represent the intermediate features learned after the input data undergoes convolution at different depths. Extract the output of the global average pooling layer. The global average pooling layer captures global information of the entire image and aggregates it into a single vector.

[0077] The feature vectors are normalized using the L2 norm to adjust the components of each feature vector to the same scale range, so that the extracted feature vectors have the same scale and improve the accuracy of distance calculation.

[0078] The OCD method stores the feature vectors of all samples (i.e., the image data described in this application) together with their corresponding sample indexes in a feature array. This helps to quickly retrieve features and identify the source of samples in subsequent steps. The feature array is divided into three, including feature vectors of the training set, as well as feature vectors of the validation set and test set, in preparation for subsequent distance calculation and OOD (Out-of-Distribution Detection) detection.

[0079] Phase 2: Distance calculation and scoring

[0080] The Faiss library is used to build an index. For each test sample, the Euclidean distance between the test sample and all training set samples is calculated. The Euclidean distance is used to measure the similarity between two feature vectors. The smaller the distance, the more similar the two samples are.

[0081] To reduce the impact of extreme values ​​on the distance score, after calculating the distances, all distances are sorted from smallest to largest. Then, the middle k distances are selected (excluding outliers from the front-end and back-end), inverted, and averaged to form the score for the test sample. A higher score indicates a lower similarity between the sample and the training set, making it more likely to be an out-of-order (OOD) sample (indicating unlearned and unremembered data).

[0082] Phase 3: OOD Detection

[0083] Before performing OOD detection, a threshold is preset to determine whether a sample is OOD. The last layer of the ResNet34 model is modified to output x+1 categories, where the x+1th category is the OOD category.

[0084] The samples in the validation set are predicted to obtain the probability that each sample belongs to the OOD category. These probabilities represent the confidence of the model that the sample is OOD. At the same time, for each sample in the validation set, the OCD method also calculates its distance score. Then, the OOD category probability of each sample is multiplied by its distance score to obtain a product value. Finally, these product values ​​are sorted, and the product values ​​corresponding to the bottom 5% of samples are selected as the threshold, which is used to distinguish between ID samples (representing samples that have been learned and memorized) and OOD samples.

[0085] For each test sample, the OCD method calculates the class probability and distance score of its OOD class according to the processing method of the validation set, and multiplies the two. If the product value of the test sample is greater than the set threshold, the sample is judged to be an OOD sample (new cell data); otherwise, it is judged to be an ID sample (old cell data).

[0086] In step S3, the cell morphology inspection system is subjected to iterative rounds of incremental learning according to each of the new cell data to update the attribute learning memory library and the out-of-distribution sample detection module until each of the new cell data is marked as learned and memorized cell data, and the target cytological morphology inspection system of the final round is output;

[0087] The incremental learning of the current round of the cell morphology inspection system includes the following steps:

[0088] Obtaining the target new cell data of the current round, and performing incremental learning on the cell morphology inspection system outputted in the previous round according to each cell attribute of the target new cell data, until each cell attribute of the target new cell data is incrementally learned, thereby obtaining a feature fusion vector of the target new cell;

[0089] The attribute learning memory library and the out-of-distribution sample detection module are updated according to the feature fusion vector to output the cell morphology inspection system of the current round.

[0090] The incremental learning includes attribute incremental learning and category incremental learning. The attribute incremental learning is used to represent the incremental learning of attribute features of each cell attribute of the target new cell data, and the category incremental learning is used to represent the incremental learning of category features of the target new cell data.

[0091] The incremental learning of the cell morphology inspection system output in the previous round according to each cell attribute of the target new cell data includes:

[0092] Obtain the previous attribute learning memory library of the cell morphology inspection system output in the previous round;

[0093] Determining, based on each cell attribute of the target new cell data, a plurality of similarity weight data between each learned and memorized cell data in the previous attribute learning and memory library, to obtain a similarity weight data set;

[0094] For each cell attribute of the target new cell data, obtain a target cell attribute corresponding to the cell attribute in the previous attribute learning and memory library, and perform attribute feature migration and fusion on each of the cell attributes and the cell attributes according to the similarity weight data set to obtain a cell attribute fusion vector, wherein the target cell attribute is the cell attribute learned and memorized in the historical rounds;

[0095] The feature fusion vector is determined according to each of the cell attribute fusion vectors, and the previous attribute learning memory library is updated according to the feature fusion vector to complete the attribute incremental learning of the cell morphology inspection system.

[0096] Specifically, the following combination Figure 5 、 6 , 7, 8 for detailed explanation:

[0097] First, this application adopts the ILA (Incremental learning based on cell Attributes) method to construct a progressive attribute-driven dynamic expansion network, designs an attention-weighted dynamic feature fusion network, and designs a cosine distillation loss to achieve efficient learning and classification of newly added category cells, while maintaining the model's discrimination performance on the attributes and categories of new and old category cells, reducing catastrophic forgetting, reducing time and computing power consumption, and thus improving training efficiency.

[0098] The following is a detailed explanation of the ILA method:

[0099] like Figure 5 The initialization phase shown is as follows: using the existing cell category data (i.e., the cell data that has been learned and memorized in the cell morphology inspection system output in the previous round) to initialize the low-dimensional attribute extractor L, high-dimensional attribute extractor H1, attribute discriminator F1 and category classifier C1, and construct / update the attribute learning memory library.

[0100] The network contains a shared L, which is composed of the first two residual blocks of a pretrained model and is responsible for extracting basic features from the input data. It also contains a high-dimensional attribute feature extractor H1, which is composed of the last two residual blocks of the pretrained model. Therefore, the low-dimensional attribute extractor L and the high-dimensional attribute extractor H are different parts of the same model network, and the output of the low-dimensional attribute extractor L is the input of the high-dimensional attribute extractor. Before processing the new cell classification task, the low-dimensional attribute extractor L and the high-dimensional attribute extractor H1 are trained based on the existing cell data to provide model support for subsequent incremental tasks.

[0101] Attribute discriminator F1 maps the output h1 of high-dimensional attribute extractor H1 through a fully connected layer to obtain A1. The final output of this mapping network is an n-dimensional attribute feature layer that can be divided into Q attribute combinations, namely, an attribute feature vector. During the initialization phase, since the model only has a single high-dimensional attribute extractor, attribute discriminator F1 does not need to perform attention-weighted dynamic fusion on the attribute feature vector A1.

[0102] The cell category classifier C1, based on the attribute discriminator F1, finally adds a layer of k-class classification layer as the output of the cell category classification. The training of the attribute classification network enables the network to classify cells directly by attribute features. It should be noted that after the attribute discriminator obtains the output (the output is the feature fusion vector described in this application), the output is used for training to achieve incremental learning at the attribute level (i.e., the attribute incremental learning described in this application). This is an independent training process, and the training of attribute incremental learning is carried out through the attribute incremental network. After the training is completed, the feature fusion vector output by the attribute incremental network is imported, and then the category discriminator is used for training to achieve category incremental learning, and category incremental learning is carried out through the category incremental network.

[0103] Specifically, if Figure 11 As shown, each cell category corresponds to multiple cell attributes, and each image data contains a cell category label (i.e. Figure 11 At the same time, based on the knowledge of blood cell morphology, several attributes are defined for each type of cell (each new cell data in this application represents a type of cell), and each attribute contains several attribute features (each attribute represents each cell attribute described in this application). Some data can be seen in the figure, which shows some attribute features of 15 types of cells. In the table, except for the first row (cell category) and the first column (cell attribute), the content of each cell represents the semantic feature value of its corresponding attribute.

[0104] Build an attribute learning memory library to record the attribute features (i.e., cell attributes) corresponding to each learned cell category. The present invention uses the form of DataFrame, where each row represents a cell category and each column corresponds to an attribute feature. This structure allows for easy access, query, and manipulation of data. When inputting cell data, first read the CSV file containing the cell data attributes and load it into a Pandas DataFrame. Set the cell category as the index of the DataFrame for subsequent queries. Save the constructed attribute learning memory library as a new CSV file format for loading and use in subsequent operations.

[0105] Phase 1: Import the attribute features (attribute features, i.e., the cell attributes described in this application) of the new category cell t (new category cell dataset d) into the attribute learning memory bank, obtain the record of cell attribute learning by searching the attribute learning memory bank, calculate the Hamming distance between the new category cell t and each old category cell (the old category cell is the cell data that has been learned and memorized) at the attribute feature level, and obtain the attribute similarity weight between the new category cell (new cell data) and each old category cell (old cell data).

[0106] The determining, based on each cell attribute of the target new cell data, a plurality of similarity weight data between each learned and memorized cell data in the previous attribute learning memory library to obtain a similarity weight data set includes:

[0107] For each cell attribute of the target new cell data, calculating the Hamming distance between the cell attribute and the attribute feature of each cell data in the attribute learning memory library;

[0108] The similarity weight data corresponding to each of the cell attributes of the target new cell data are determined according to each of the Hamming distances, and the similarity weight data set is determined according to each of the similarity weight data.

[0109] When a new cell type t is input, the morphological attribute features of the new cell type t are first imported and added to the attribute learning memory. Then, the attribute features of each old cell type in the attribute memory are traversed, that is, each old cell type is read row by row and the attributes corresponding to each cell are read column by column. The present invention uses the Hamming distance to calculate the distance between the attributes of the new cell type and the attributes of each old cell type, and performs a weighted summation on the distances of each attribute to obtain the attribute difference distance D between the new cell type and the i-th old cell type. i .

[0110] Specifically, for a new category cell t, its attribute features are extracted from the attribute learning memory bank and represented as M t, then traverse each old category cell c i Attribute characteristics And calculate the Hamming distance D, the formula is as follows:

[0111]

[0112] Where Q is the number of cell attributes defined in cell morphology. t [j] and Represent the feature labels of the new category and old category cells on the jth attribute respectively.

[0113] According to the Hamming distance D(t,c i ), the similarity between the new category cell and each old category cell at the attribute level is represented as the weight w(t,c i ), that is, the similarity weight dataset described in this application, the weight is inversely proportional to the distance, and the greater the distance, the smaller the weight.

[0114]

[0115] The weight w(t,c i ), used to characterize the c i The attribute correlation between the old category cell and the new category cell t, the larger the weight, the higher the correlation between the two cells, then in the subsequent fusion step, the high-dimensional extractor H corresponding to the old category cell t The larger the proportion of output, the more fully existing knowledge can be utilized, providing a basis for subsequent feature fusion and model updating.

[0116] Phase 2: When the t-th new cell data is input, the model dynamically expands a high-dimensional attribute extractor H according to the attribute learning situation t , the low-dimensional attribute extractor L and the high-dimensional attribute extractor {H 1, H 2, …,H t-1} parameters, and the output of all high-dimensional attribute extractors {h 1, h 2, …,h t}(output update parameters described in the embodiment of the present application).

[0117] The incremental learning of the cell morphology inspection system output in the previous round according to each cell attribute of the target new cell data includes:

[0118] Obtain the previous attribute learning memory library of the cell morphology inspection system output in the previous round;

[0119] Determining, based on each cell attribute of the target new cell data, a plurality of similarity weight data between each learned and memorized cell data in the previous attribute learning and memory library, to obtain a similarity weight data set;

[0120] For each cell attribute of the target new cell data, obtain a target cell attribute corresponding to the cell attribute in the previous attribute learning and memory library, and perform attribute feature migration and fusion on each of the cell attributes and the cell attributes according to the similarity weight data set to obtain a cell attribute fusion vector, wherein the target cell attribute is the cell attribute learned and memorized in the historical rounds;

[0121] Determine the attribute feature vector of the target new cell data according to each of the cell attribute fusion vectors;

[0122] A feature fusion vector is determined based on the attribute feature vector and the attribute feature vectors of each cell data in historical rounds, and the previous attribute learning memory library is updated based on the feature fusion vector to complete the attribute incremental learning of the cell morphology inspection system.

[0123] When the image data of the newly added cell of the tth category (i.e., the target new cell data of the tth category) is input, the ILA method first freezes the attribute extractor {L,H 1, H 2, …,H t-1} (i.e., the corresponding feature output in the cell morphology inspection system output by the historical rounds), that is, its gradient is closed to prevent the influence on the model weight of the old task during the training of the new task (i.e., the incremental learning of the current round). At the same time, by searching the attribute learning memory library, the attribute features of the new category cell t that have been learned by the extractor of the previous stage (the previous stage represents the historical round) (i.e., the target cell attributes) and the attribute features that have not been learned by the extractor of the previous stage (i.e., the cell attributes that have not been learned in the historical rounds) are obtained, and the number of unlearned attributes is recorded as N. At this time, the model dynamically expands N residual blocks as a new high-dimensional attribute extractor H t This approach will H t It is used to learn the attribute features of new cell data t, so that the model can effectively learn the features of new cell data t; on the other hand, according to the learning status of the attribute features, high-dimensional attribute extractors are dynamically introduced to avoid the increased complexity caused by the static introduction of fixed extractors, thereby reducing the loss of computing power.

[0124] At this time, each high-dimensional attribute extractor {H 1, H 2, …,H t}Parallel independent output model prediction feature vector h i , where hi is the output of the i-th high-dimensional attribute extractor, with dimension d. In general, the high-dimensional attribute extractor H t Compared with all high-dimensional attribute extractors in the previous stage {H 1, H 2, …,H t-1} parallel output of the feature vector {h 1, h 2, …,h t}.

[0125] Stage 3: All high-dimensional attribute extractor outputs {h1,h2,…,h t} Input to attribute feature discriminator F t , attribute discriminator F t It contains the following three functions: (1) transform the feature vector {h1,h2,…,h t}After mapping through the fully connected layer, the attribute feature vector {A1, A2, ..., A t}; (2) According to the attribute learning memory library, the learned attribute situation is obtained, and the learned attribute features in the attribute feature vector are transferred to A t (3) for the attribute feature vector {A1, A2, ..., A t}We perform weighted dynamic feature fusion based on the attention mechanism. Finally, the attribute increment network is trained using the designed loss function.

[0126] The following is a detailed description of the contents of Phase 3:

[0127] like Figure 6 As shown, h t is the feature currently output by the new cell data t, which includes multiple cell attributes (i.e., attribute features). Then, an attribute incremental learning is performed on each cell attribute, such as cell attribute feature j, Figure 7 As shown, in the incremental learning of historical rounds (i.e. h1,h2,…h t-1 ) to find out which cell attribute features j have been learned, Figure 7 In this example, h1 and h3 have already learned the target cell attribute j. Based on the similarity weight data from stage 1, they are fused with Aj from the current round of incremental learning At to obtain Aj-fusion, which is the cell attribute fusion vector for the jth cell attribute. These cell attribute fusion vectors are then fused to obtain the attribute feature vector At for the current round (i.e., the attribute feature vector described in stage 3). At corresponds to the new cell category t.

[0128] Updating the previous attribute learning memory base according to the feature fusion vector may be specifically as follows:

[0129] Combine the attribute feature vector At of the current round with the attribute feature vector set {A1, A2, ..., A t-1} to merge, such as Figure 8 As shown, for the current round of feature fusion vector Afusion, to complete the attribute incremental learning,

[0130] The output of the high-dimensional attribute extractor {h1,h2,…,h t}, input to the attribute feature discriminator. Each feature vector h i Map it to n dimensions through a fully connected layer alone to obtain the attribute feature vector A i At this time, through the feedback of the attribute learning memory bank, the attribute features of the new category cell t that have been learned by the previous stage extractor are obtained. For each learned attribute feature, the index and number M corresponding to the learned attribute and the corresponding high-dimensional attribute extractor number are recorded. According to the index range corresponding to each attribute, each attribute feature A can be intercepted i In the output A of one of the learned attribute features ij , which represents the output of the i-th high-dimensional attribute extractor that has learned the attribute feature j.

[0131] According to the number M of extractors corresponding to the learned attribute features, the cell attribute similarity weights obtained in stage 1 are imported, and the weights of the old category cells corresponding to the high-dimensional attribute extractors that have learned the attribute are screened out. These weights are normalized to facilitate subsequent feature fusion.

[0132]

[0133] By fusing the attribute feature A corresponding to the high-dimensional extractor that has learned the attribute feature ij , get the fusion feature vector Aj-fusion of attribute j, the calculation formula is as follows:

[0134]

[0135] Replace the newly expanded high-dimensional attribute extractor H with Aj-fusion (feature fusion vector) t The corresponding attribute feature vector A t The feature vector of attribute j in . This is equivalent to fusing the output of the model that has learned the attribute features, and using this fused output as the newly expanded high-dimensional attribute extractor H t The output of the attribute j. This operation traverses all the learned attributes of the newly added category cell t, and finally completes the A t Transfer of learned attribute feature outputs in .

[0136] The attribute incremental learning and category incremental learning of this application are performed using two different models, namely the attribute incremental network and the category incremental network. The attribute incremental network model uses the attention weighted dynamic fusion method to process the attribute feature vector {A 1, A 2, …,A t}, t attribute feature vectors from different high-dimensional attribute extractors are fused according to their importance. The present invention uses a feedforward neural network to generate an attention score β for each attribute feature vector. The structure of the feedforward network consists of two fully connected layers. For each attribute feature vector A i , the feedforward network first maps the feature vector to the hidden layer through the first fully connected layer:

[0137]

[0138] Among them, W1 is the weight matrix of the first fully connected layer, with a size of [d, e], is the attention score of the i-th attribute feature vector after passing through the first layer of attention network, b1 is the bias vector of this layer, with a size of [e], where e is the size of the hidden layer. Next, the activation function ReLu is introduced to increase the nonlinear expression ability of the model:

[0139]

[0140] After ReLU activation, the feedforward network further maps the output of the hidden layer to a scalar through the second fully connected layer:

[0141]

[0142] Among them, W2 is the weight matrix of the second fully connected layer, the size is [e, 1], b2 is the bias term, the size is [1]. i Is a scalar, representing the attribute feature vector A i Attention score. During network initialization, the weight matrices W1 and W2 are randomly generated using He initialization to ensure that the initial weight values ​​have an appropriate scale, which is conducive to the training stability of deep networks. Throughout the entire process, the weights W1 and W2 and biases b1 and b2 of the feedforward neural network are trainable parameters optimized through the backpropagation algorithm.

[0143] To score these attention i Converted to an attention weight α that can be applied directly i , we need to i Normalization is performed. The present invention introduces the Softmax function to normalize all scores to probability values ​​between 0 and 1, and the sum is 1.

[0144]

[0145] Among them, α i is the attribute feature vector A i The final attention weight of , indicates the importance of the attribute feature in weighted fusion. All attention weights α i Normalization ensures that their sum is 1, so that they can be used in the weighted sum operation.

[0146] Then, each attribute feature vector {A1,A2,…,A t}(attribute feature vector set) is weighted and summed by attention weight to obtain a fused feature representation Afusion. The specific formula is as follows:

[0147]

[0148] Among them, Afusion (target mapping parameter) is the fused attribute feature vector, which represents the comprehensive representation of all attribute feature vectors output from different high-dimensional attribute extractors. This fusion feature contains all the attribute feature vectors from {H1, H2, ..., H t}The key information of the advanced feature extractor is obtained, and the weight of each feature extractor is dynamically adjusted according to the different input situations through the attention mechanism, thereby achieving adaptive feature fusion and capturing the key information from all related extractors.

[0149] Finally, by applying the Softmax activation function, these raw prediction values ​​are converted into a distribution representing the probability of each attribute. Specifically, according to the index range corresponding to each attribute, each attribute is traversed. For attribute k, the calculation formula of the Softmax function is:

[0150]

[0151] Among them, A fusion-k is the output of the fused attribute feature vector Afusion for attribute k, and C is the number of features corresponding to an attribute. Through the above transformation, for each attribute k, the probability p(k) of its attribute feature is between 0 and 1, and for this attribute, the sum of its attribute feature probabilities is 1. Finally, all attribute prediction probabilities are saved in a two-dimensional array P, as follows: P = [p(1), p(2), …, p(Q)], where Q is the total number of attributes.

[0152] The training set S for each incremental step t t , the dataset U of the newly introduced category t and the old category sample set V stored in memory t-1 Composition, namely S t =U t ∪Vt-1 The old category samples in memory are obtained through random sampling, and each old category cell stores L samples. By combining new data with a small amount of old data, the model can learn the characteristics of the new category cell attributes in each incremental step while maintaining the ability to recognize the characteristics of the old category cell attributes.

[0153] The loss function of the training process is L old With L new The two loss functions are used alternately in each training step.

[0154] In one embodiment of the present application, the cell morphology inspection system is trained according to a preset class balance loss function, a preset cosine distillation loss function, a preset feature enhancement loss function and a preset attribute space separation loss function, and the class balance loss function, the cosine distillation loss function, the feature enhancement loss function and the attribute space separation loss function are used for attribute incremental learning.

[0155] L new It focuses on learning the features of new categories of blood cell data and consists of four parts:

[0156] L new =L class +λ1·L enhance +λ2·L att-sep +λ3·L cos-dist

[0157] L old It focuses on helping the model remember old categories of blood cell data. At the same time, it focuses on old categories with fewer samples by assigning higher penalties to incorrectly predicted samples. It consists of three parts:

[0158] L old =L class +λ4·L cos-dist +λ5·L att-sep

[0159] The following are the design ideas of the four loss functions:

[0160] Class balance loss L class :

[0161] A class-balanced focal loss is adopted to avoid severe forgetting caused by the imbalance of data amount between new and old classes.

[0162]

[0163] Where n is the dataset S t, p(k) represents the predicted probability of the kth attribute output by the model. β is a hyperparameter that controls class imbalance and is used to impose additional penalties on scarce old category data to ensure that they are not ignored during training. γ is a hyperparameter that controls the difficulty of the task and strengthens the learning of old category samples, allowing the model to better learn challenging features.

[0164] The present invention assigns higher penalties to old classes, L class It takes into account the class imbalance when learning features to better distinguish old and new classes. It avoids discarding samples that are valuable for learning accurate decision boundaries. old The hyperparameters β, γ and L in new The hyperparameters used in are different, so the former penalizes misclassification of old class samples more.

[0165] Cosine distillation loss L cos-dist :

[0166] The old knowledge of the model is retained by minimizing the difference in attribute prediction between the new model being trained and the old model imported from the previous stage. The loss function consists of two parts: attribute difference loss L att and cosine similarity loss L cos , the calculation formula is as follows:

[0167] L cos-dist =L att +L cos

[0168] Among them, the attribute difference loss L att Used to measure the component-by-component difference between the new and old models on each attribute; cosine similarity loss L cos , which is used to evaluate the similarity between the new and old models in the overall direction of the attribute prediction vector.

[0169] Attribute difference loss L att The design is as follows: first, the attribute feature of the new model is output A new And the attribute feature output A of the old model old Temperature softening is performed to obtain and The calculation formula is as follows:

[0170]

[0171] Among them, T is the temperature parameter, which is used to control the smoothness of attribute prediction, A old (k) represents the kth attribute feature output of the old model in the previous stage, Represents the kth attribute feature output of the new model. L att The calculation is as follows:

[0172]

[0173] Attribute difference loss L att , quantified the component-by-component difference between the new and old models in the characteristic values ​​of each attribute, reflecting the accuracy of the two models in predicting specific attributes, emphasizing that the model needs to maintain the accuracy of each attribute during the update process, ensuring that the model can not only adapt to new tasks during incremental learning, but also effectively retain and utilize the knowledge of old tasks.

[0174] Cosine similarity loss L cos This method measures the directional similarity between the attribute prediction vectors of the new and old models, emphasizing the consistency of the model's overall prediction trend. It evaluates the relative position of two vectors in space by calculating the angle between them, reflecting the model's overall understanding of the attribute features. When the model maintains similar prediction directions during the update process, even though the specific feature values ​​may vary, it ensures the consistency and integration of new and old knowledge. This enables the model to more smoothly transfer existing knowledge when faced with new tasks, reducing uncertainty in the learning process.

[0175]

[0176] Finally, the attribute difference loss L att and cosine similarity loss L cos By adding the ratio ε, we get the cosine distillation loss L cos-dist .

[0177]

[0178] Among them, the combination ratio ε is used to adjust the relative importance of attribute difference loss and cosine similarity loss in the final loss, and the value range is [0,1].

[0179] Feature enhancement loss L enhance

[0180] In order to learn the attribute characteristics of new categories of cells in a targeted manner, the present invention introduces the feature enhancement loss L enhance . L enhance With L class Both use the class balance loss calculation method, but L enhance Only focus on the newly expanded high-dimensional attribute extractor H t Output h t The attribute feature A obtained after mapping t .

[0181]

[0182] Among them, p t(k) represents attribute feature A t The predicted probability of the kth attribute in .

[0183] Attribute space separation loss L att-sep

[0184] The purpose of the attribute space separation loss is to ensure that new and old classes remain separated in attribute space by measuring the L2 distance between the new model's attribute predictions for new and old class data. This approach prevents the attribute predictions of new and old classes from being too close as the model learns new classes, reducing the interference of new class cells with old class cells, thereby alleviating the "forgetting" problem in incremental learning.

[0185]

[0186] in, and They are the newly expanded high-dimensional attribute extractor H t For dataset S t and dataset V t-1 Output attribute feature vector of the kth attribute. d min It is the preset minimum distance to ensure that the new and old category cells are sufficiently separated in the attribute space.

[0187] During model training, by alternating between L new and L old Calculate and optimize the loss, and finally train the high-dimensional attribute extractor H t , attribute discriminator F t Through this training mode, the network can continuously learn the attribute features of newly added categories while reducing the forgetting of the attribute features of old categories, thus realizing the incremental learning based on cell attributes of the present invention.

[0188] Phase 4: Load the model trained in Phase 3 (i.e., obtain Afusion), update the category classifier Ct based on the attribute incremental network structure, and use Ct to map the feature fusion vector Afusion to the current cell category B t (i.e., the mapping relationship between the feature fusion vector and the target new cell data described in this application), using the preset cross entropy loss function and the preset distillation loss function, train and optimize the category incremental network to achieve incremental learning at the cell category level (i.e., the category incremental learning described in this application).

[0189] Through the training of stage three, an attribute discriminator Ft is obtained that can classify the attribute features of new and old categories of cells. This application loads the model obtained in stage three, and updates the category classifier Ft at the same time, and modifies its fully connected layer output to t (because t-1 type cells have been learned before, it needs to be modified to t, and each cell category data can be identified in parallel). The parameter t changes dynamically with the number of new categories. The fused attribute feature vector Afusion output by the attribute discriminator Ft in the attribute increment network of the third stage is mapped to the category through the updated classifier Ct, realizing the mapping of the model from attribute to category, and obtaining the category feature vector B t . Use the Softmax function to transform the category feature vector B t Converted into a probability distribution p t , the processing is as follows, where t represents the total number of categories:

[0190]

[0191] Use the loss function to train the category feature vector p i , we obtain the category incremental network and realize the incremental learning of cell categories.

[0192] The loss function of the training process is L cls , which consists of two parts: L cls =L dist +L ce , helps the model learn new categories of cells while not forgetting the old categories of cells. The dataset is S t , using cell category labels. The following are the specific designs of the two functions:

[0193] Cross entropy loss L ce :

[0194]

[0195] Among them, q i is the true label of the cell category, p i The probability distribution predicted by the model. The cross-entropy loss function is used to measure the difference between the predicted probability distribution and the true label probability distribution, guiding model training to minimize the difference and thus improve classification accuracy.

[0196] Distillation loss L dist :

[0197]

[0198] Among them, T is the temperature parameter, which is used to adjust the smoothness of the probability distribution in the Softmax function. and The old model and the new model are respectivelyt The Softmax probability distribution of , the calculation process is as follows:

[0199]

[0200] Distillation loss L dist Measuring the difference between the output of the new model and the output of the old model allows the new model to learn and inherit the behavior of the old model, especially in terms of probability distribution. This allows the new model to absorb the deep knowledge and probabilistic relationships of the old model when dealing with different categories while maintaining a small model size. This reduces computational complexity while maintaining or improving the model's performance and generalization ability as much as possible.

[0201] Specifically, by storing the new cell data obtained through continuous learning and memory into the attribute learning memory bank, it becomes old cell data. Without forgetting the old cell data of the old category, new cell data of the new category is continuously learned, and multiple rounds of iterative updates of the cell morphology inspection system can be performed, thereby solving the problem of weak analysis and learning capabilities existing in the existing technology.

[0202] Ke Ru Figure 9 As shown, Figure 9 This is a flowchart of incremental learning in an embodiment of the present application. By inputting a cell sample data stream (a first data set D1), a new category cell data set d is screened out, and the new category cell samples in the new category cell data set d are labeled. The labeled samples are integrated into the model of the cell morphology inspection system through the ILA method to achieve system evolution.

[0203] Figure 10 This is a logic diagram of an iterative device for a cell morphology inspection system based on incremental learning according to an embodiment of the present application. Figure 10 This is a block diagram of an iterative device 300 for a cell morphology inspection system based on incremental learning according to an embodiment of the present application. According to an iterative device 300 for a cell morphology inspection system based on incremental learning according to an embodiment of the present application, the system 300 includes: an acquisition unit 301, an input unit 302, and an incremental learning unit 303.

[0204] An acquisition unit 301 is configured to acquire image data of target cells and construct a first data set based on the image data;

[0205] An input unit 302 is configured to input the first data set into the out-of-distribution sample detection module to obtain a new category cell data set within the first data set, wherein the new category cell data set includes a plurality of new cell data, and the new cell data is used to represent cell data that has not been learned and memorized;

[0206] An incremental learning unit 303 is configured to perform iterative rounds of incremental learning on the cell morphology inspection system according to each of the new cell data to update the attribute learning memory library and the out-of-distribution sample detection module until each of the new cell data is marked as learned and memorized cell data, and output a final round of target cytological morphology inspection system;

[0207] The incremental learning of the current round of the cell morphology inspection system includes the following steps:

[0208] Obtaining the target new cell data of the current round, and performing incremental learning on the cell morphology inspection system outputted in the previous round according to each cell attribute of the target new cell data, until each cell attribute of the target new cell data is incrementally learned, thereby obtaining a feature fusion vector of the target new cell;

[0209] The attribute learning memory library and the out-of-distribution sample detection module are updated according to the feature fusion vector to output the cell morphology inspection system of the current round.

[0210] As another aspect, the present application further provides a computer-readable storage medium having stored thereon a program product capable of implementing the methods provided above in this specification. In some possible implementations, various aspects of the present application may also be implemented in the form of a program product, which includes program code. When the program product is executed on a terminal device, the program code is used to cause the terminal device to perform the steps described in the "Example Method" section above in accordance with various exemplary embodiments of the present application.

[0211] According to an embodiment of the present application, a program product for implementing the above method can be a portable compact disc read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present application is not limited thereto. In this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0212] The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0213] A computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium that can transmit, propagate, or transfer a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0214] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0215] The program code for performing the operations of the present application can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0216] Furthermore, the above-mentioned figures are merely illustrative of the processes included in the methods according to exemplary embodiments of the present application and are not intended to be limiting. It is readily understood that the processes illustrated in the above-mentioned figures do not indicate or limit the temporal order of these processes. Furthermore, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0217] It should be understood that the present application is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes can be performed without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. An iterative method for cell morphology inspection system based on incremental learning, characterized in that: Applied to a cell morphology testing system, the cell morphology testing system includes a preset out-of-distribution sample detection module and an attribute learning memory library, the attribute learning memory library storing a plurality of learned and memorized cell data, the method comprising: S1, acquiring image data of target cells and constructing a first data set based on the image data; S2, inputting the first data set into the out-of-distribution sample detection module to obtain a new category cell data set within the first data set, wherein the new category cell data set includes a plurality of new cell data, and the new cell data is used to represent cell data that has not been learned and memorized; S3, performing iterative rounds of incremental learning on the cell morphology inspection system according to each of the new cell data to update the attribute learning memory library and the out-of-distribution sample detection module, until each of the new cell data is marked as learned and memorized cell data, and outputting a final round of target cytological morphology inspection system; The incremental learning of the current round of the cell morphology inspection system includes the following steps: Obtaining the target new cell data of the current round, and performing incremental learning on the cell morphology inspection system outputted in the previous round according to each cell attribute of the target new cell data, until each cell attribute of the target new cell data is incrementally learned, thereby obtaining a feature fusion vector of the target new cell; The attribute learning memory library and the out-of-distribution sample detection module are updated according to the feature fusion vector to output the cell morphology inspection system of the current round.

2. The iterative method for cell morphology inspection system based on incremental learning according to claim 1, characterized in that: The incremental learning includes attribute incremental learning and category incremental learning. The attribute incremental learning is used to represent the incremental learning of attribute features of each cell attribute of the target new cell data, and the category incremental learning is used to represent the incremental learning of category features of the target new cell data.

3. The iterative method for cell morphology inspection system based on incremental learning according to claim 2, characterized in that: The incremental learning of the cell morphology inspection system output in the previous round according to each cell attribute of the target new cell data includes: Obtain the previous attribute learning memory library of the cell morphology inspection system output in the previous round; Determining, based on each cell attribute of the target new cell data, a plurality of similarity weight data between each learned and memorized cell data in the previous attribute learning and memory library, to obtain a similarity weight data set; For each cell attribute of the target new cell data, obtain a target cell attribute corresponding to the cell attribute in the previous attribute learning and memory library, and perform attribute feature migration and fusion on each of the cell attributes and the cell attributes according to the similarity weight data set to obtain a cell attribute fusion vector, wherein the target cell attribute is the cell attribute learned and memorized in the historical rounds; Determine the attribute feature vector of the target new cell data according to each of the cell attribute fusion vectors; A feature fusion vector is determined based on the attribute feature vector and the attribute feature vectors of each cell data in historical rounds, and the previous attribute learning memory library is updated based on the feature fusion vector to complete the attribute incremental learning of the cell morphology inspection system.

4. The iterative method for cell morphology inspection system based on incremental learning according to claim 3, characterized in that: The incremental learning of the cell morphology inspection system is performed by the following steps: Binding the feature fusion vector to the target new cell data to determine a mapping relationship between the feature fusion vector and the target new cell data; The category features of the target new cell data are determined according to the mapping relationship, and the out-of-distribution sample detection module is updated according to the category features to complete the category incremental learning of the cell morphology inspection system.

5. The iterative method for cell morphology inspection system based on incremental learning according to claim 4, characterized in that: The determining, based on each cell attribute of the target new cell data, a plurality of similarity weight data between each learned and memorized cell data in the previous attribute learning memory library to obtain a similarity weight data set includes: For each cell attribute of the target new cell data, calculating the Hamming distance between the cell attribute and the attribute feature of each cell data in the attribute learning memory library; The similarity weight data corresponding to each of the cell attributes of the target new cell data are determined according to each of the Hamming distances, and the similarity weight data set is determined according to each of the similarity weight data.

6. The iterative method for cell morphology inspection system based on incremental learning according to claim 5, characterized in that: The new category cell dataset is obtained by the following steps: Obtaining the category probability and distance score of each of the image data in the first data set through an out-of-distribution sample detection module; For each image data, if the product value of the category probability and the distance score of the image data is greater than a preset threshold, the cell data corresponding to the image data is determined to be the new cell data; Wherein, each of the image data corresponds to single cell data.

7. The iterative method for cell morphology inspection system based on incremental learning according to claim 6, characterized in that: The cell morphology inspection system is trained according to a preset class balance loss function, a preset cosine distillation loss function, a preset feature enhancement loss function and a preset attribute space separation loss function. The class balance loss function, the cosine distillation loss function, the feature enhancement loss function and the attribute space separation loss function are used for attribute incremental learning.

8. The iterative method for cell morphology inspection system based on incremental learning according to claim 7, characterized in that: The cell morphology inspection system is also trained according to a preset cross entropy loss function and a preset distillation loss function, and the cross entropy loss function and the distillation loss function are used for category incremental learning.

9. An iterative device for a cell morphology inspection system based on incremental learning, characterized in that: Applied to a cell morphology inspection system, the cell morphology inspection system includes a preset out-of-distribution sample detection module and an attribute learning memory library, the attribute learning memory library stores a plurality of learned and memorized cell data, the device includes: an acquisition unit, configured to acquire image data of target cells and construct a first data set based on the image data; an input unit, configured to input the first data set into the out-of-distribution sample detection module to obtain a new category cell data set within the first data set, wherein the new category cell data set includes a plurality of new cell data, and the new cell data is used to represent cell data that has not been learned and memorized; an incremental learning unit, configured to perform iterative rounds of incremental learning on the cell morphology inspection system according to each of the new cell data to update the attribute learning memory library and the out-of-distribution sample detection module, until each of the new cell data is marked as learned and memorized cell data, and output a final round of target cytological morphology inspection system; The incremental learning of the current round of the cell morphology inspection system includes the following steps: Obtaining the target new cell data of the current round, and performing incremental learning on the cell morphology inspection system outputted in the previous round according to each cell attribute of the target new cell data, until each cell attribute of the target new cell data is incrementally learned, thereby obtaining a feature fusion vector of the target new cell; The attribute learning memory library and the out-of-distribution sample detection module are updated according to the feature fusion vector to output the cell morphology inspection system of the current round.

Images