Cell image recognition method and system based on model fine-tuning

Abstract

The present invention relates to the field of data processing technology, and discloses a cell image recognition method and system based on model fine-tuning, the method comprising: obtaining an image to be recognized including a cell image; inputting the image to be recognized into a trained cell clarity recognition model to obtain the corresponding cell clarity; inputting the image to be recognized into a trained cell existence region recognition model to obtain the corresponding image of the existing cell region; inputting the image of the existing cell region and the cell clarity into a trained cell classification model to obtain corresponding multiple cell regions and corresponding cell types; the cell clarity recognition model, the cell existence region recognition model and the cell classification model are obtained based on joint training and fine-tuning. It can be seen that the present invention can make full use of clarity factors and prior region recognition to accurately identify cell regions and types in images, thereby improving the accuracy and efficiency of cell recognition.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a cell image recognition method and system based on model fine-tuning. Background Art

[0002] With the development of image processing technology and the popularization of medical auxiliary imaging technology, more and more biological cell images are used for disease research or treatment purposes. How to effectively identify and segment cell areas in biological tissue images has become a core technical problem. However, when realizing cell recognition, the existing cell image recognition technology generally only relies on training common image recognition algorithm models based on cell annotated images for recognition. It does not further build a unique algorithm model for recognition based on different links and clarity influencing factors in the cell recognition process. Therefore, its cell recognition efficiency and accuracy are both lacking. It can be seen that the existing technology has defects that need to be solved urgently. Summary of the invention

[0003] The technical problem to be solved by the present invention is to provide a cell image recognition method and system based on model fine-tuning, which can make full use of clarity factors and prior region recognition to accurately identify cell regions and types in images, improve the accuracy and efficiency of cell recognition, and provide an accurate data basis for disease treatment or research.

[0004] In order to solve the above technical problems, the first aspect of the present invention discloses a cell image recognition method based on model fine-tuning, the method comprising:

[0005] Acquiring an image to be identified including a cell image;

[0006] Inputting the image to be identified into a trained cell definition recognition model to obtain the corresponding cell definition;

[0007] Inputting the image to be identified into a trained cell existing region recognition model to obtain a corresponding cell existing region image;

[0008] The image of the cell area and the cell clarity are input into a trained cell classification model to obtain corresponding multiple cell areas and corresponding cell types; the cell clarity recognition model, the cell area recognition model and the cell classification model are obtained based on joint training and fine-tuning.

[0009] As an optional embodiment, in the first aspect of the present invention, the cell clarity recognition model, the cell existence area recognition model and the cell classification model are obtained by joint training based on the same training data set; the training data set includes multiple training images and corresponding cell area annotations, cell type annotations and cell clarity annotations.

[0010] As an optional embodiment, in the first aspect of the present invention, the cell clarity recognition model, the cell existence region recognition model and the cell classification model are trained by the following steps:

[0011] Inputting the training data set into the cell clarity recognition model and the cell existence region recognition model for training until the first prediction accuracy of the two reaches a preset first accuracy threshold;

[0012] Freezing the model weights of the cell clarity recognition model and the cell existence region recognition model, inputting the training data set into the cell classification model for training until its second prediction accuracy reaches a preset second accuracy threshold; the second accuracy threshold is greater than the first accuracy threshold;

[0013] The model weights of the cell clarity recognition model and the cell existence area recognition model are unfrozen, and the training data set is simultaneously input into the cell clarity recognition model, the cell existence area recognition model and the cell classification model for joint training until the third prediction accuracy of the three reaches a preset third accuracy threshold, thereby obtaining the trained cell clarity recognition model, the cell existence area recognition model and the cell classification model; the third accuracy threshold is greater than the second accuracy threshold.

[0014] As an optional embodiment, in the first aspect of the present invention, the cell clarity recognition model is a convolutional neural network, which uses L1 loss as a loss function during training; the cell existence area recognition model and the cell classification model are both fully convolutional neural networks, which use cross entropy loss function as a loss function during training.

[0015] As an optional embodiment, in the first aspect of the present invention, when the training data set is simultaneously input into the cell clarity recognition model, the cell existence region recognition model and the cell classification model for joint training, the loss function used in the joint training is a weighted sum of the first loss function value, the second loss function value and the third loss function value; wherein:

[0016] The first loss function value is the difference between the predicted clarity output by the cell clarity recognition model and the corresponding cell clarity annotation, and the corresponding weight is proportional to the variance of the cell clarity annotations corresponding to all training images in the current input data;

[0017] The second loss function value is the cross entropy between the predicted cell existence region output by the cell existence region recognition model and all the corresponding cell region annotations, and the corresponding weight is proportional to the total area of ​​the region union of all cell region annotations corresponding to all training images in the current input data;

[0018] The value of the third loss function is the cross entropy between the predicted cell region and predicted type output by the cell classification model and the corresponding cell region annotation and cell type annotation, and its corresponding weight is proportional to the total number of types corresponding to all cell type annotations corresponding to all training images in the input data at that time.

[0019] As an optional embodiment, in the first aspect of the present invention, the cell type is PanCK+ cells, CD3+ cells, CD8- cells or CD8+ cells.

[0020] As an optional implementation, in the first aspect of the present invention, the training data set is obtained by the following steps:

[0021] Obtain multiple training images;

[0022] Determining the cell nucleus position annotation and cell type annotation corresponding to each of the training images based on the annotations of the operator;

[0023] For each of the cell nucleus position annotations, obtaining a cell type outline corresponding to the cell type annotation corresponding to the cell nucleus position annotation;

[0024] Based on the cell type contour, a contour is generated with the position of the cell nucleus position mark as the center to obtain the corresponding cell region mark;

[0025] According to the time taken by the operator to label each of the training images, the cell clarity labeling corresponding to each of the training images is determined.

[0026] As an optional embodiment, in the first aspect of the present invention, in the step of acquiring the training data set, determining the cell clarity annotation corresponding to each training image according to the time taken by the operator to annotate each training image includes:

[0027] For each of the training images, obtaining a movement trajectory and a corresponding movement speed of a marking cursor of the operator on the training image;

[0028] Screening out, from the plurality of image regions of the training image, a plurality of cell-dense regions whose corresponding cell region annotation quantity is greater than a preset quantity threshold;

[0029] Calculate the product of the length of the track portion of the marking cursor movement track in each of the cell-dense areas and the corresponding movement speed to obtain the area marking movement time corresponding to each of the cell-dense areas;

[0030] The weighted sum average of the moving times of the regional annotations corresponding to all the cell-dense areas is calculated to obtain the cell clarity annotations corresponding to the training image; wherein the weight corresponding to the moving time of the regional annotations corresponding to each of the cell-dense areas is inversely proportional to the number of annotations corresponding to the cell-dense area.

[0031] A second aspect of an embodiment of the present invention discloses a cell image recognition system based on model fine-tuning, the system comprising:

[0032] An acquisition module, used for acquiring an image to be identified including a cell image;

[0033] A first recognition module, used for inputting the image to be recognized into a trained cell definition recognition model to obtain the corresponding cell definition;

[0034] A second recognition module is used to input the image to be recognized into a trained cell existing region recognition model to obtain a corresponding cell existing region image;

[0035] The third recognition module is used to input the image of the cell area and the cell clarity into the trained cell classification model to obtain corresponding multiple cell areas and corresponding cell types; the cell clarity recognition model, the cell existence area recognition model and the cell classification model are obtained based on joint training and fine-tuning.

[0036] As an optional embodiment, in the second aspect of the present invention, the cell clarity recognition model, the cell existence area recognition model and the cell classification model are obtained by joint training based on the same training data set; the training data set includes multiple training images and corresponding cell area annotations, cell type annotations and cell clarity annotations.

[0037] As an optional embodiment, in the second aspect of the present invention, the cell clarity recognition model, the cell existence region recognition model and the cell classification model are trained by the following steps:

[0038] Inputting the training data set into the cell clarity recognition model and the cell existence region recognition model for training until the first prediction accuracy of the two reaches a preset first accuracy threshold;

[0039] Freezing the model weights of the cell clarity recognition model and the cell existence region recognition model, inputting the training data set into the cell classification model for training until its second prediction accuracy reaches a preset second accuracy threshold; the second accuracy threshold is greater than the first accuracy threshold;

[0040] The model weights of the cell clarity recognition model and the cell existence area recognition model are unfrozen, and the training data set is simultaneously input into the cell clarity recognition model, the cell existence area recognition model and the cell classification model for joint training until the third prediction accuracy of the three reaches a preset third accuracy threshold, thereby obtaining the trained cell clarity recognition model, the cell existence area recognition model and the cell classification model; the third accuracy threshold is greater than the second accuracy threshold.

[0041] As an optional embodiment, in the second aspect of the present invention, the cell clarity recognition model is a convolutional neural network, which uses L1 loss as a loss function during training; the cell existence area recognition model and the cell classification model are both fully convolutional neural networks, which use cross entropy loss function as a loss function during training.

[0042] As an optional embodiment, in the second aspect of the present invention, when the training data set is simultaneously input into the cell clarity recognition model, the cell presence region recognition model and the cell classification model for joint training, the loss function used in the joint training is a weighted sum of the first loss function value, the second loss function value and the third loss function value; wherein:

[0043] The first loss function value is the difference between the predicted clarity output by the cell clarity recognition model and the corresponding cell clarity annotation, and the corresponding weight is proportional to the variance of the cell clarity annotations corresponding to all training images in the current input data;

[0044] The second loss function value is the cross entropy between the predicted cell existence region output by the cell existence region recognition model and all the corresponding cell region annotations, and the corresponding weight is proportional to the total area of ​​the region union of all cell region annotations corresponding to all training images in the current input data;

[0045] The value of the third loss function is the cross entropy between the predicted cell region and predicted type output by the cell classification model and the corresponding cell region annotation and cell type annotation, and its corresponding weight is proportional to the total number of types corresponding to all cell type annotations corresponding to all training images in the input data at that time.

[0046] As an optional embodiment, in the second aspect of the present invention, the cell type is PanCK+ cells, CD3+ cells, CD8- cells or CD8+ cells.

[0047] As an optional implementation, in the second aspect of the present invention, the training data set is obtained by the following steps:

[0048] Obtain multiple training images;

[0049] Determining the cell nucleus position annotation and cell type annotation corresponding to each of the training images based on the annotations of the operator;

[0050] For each of the cell nucleus position annotations, obtaining a cell type outline corresponding to the cell type annotation corresponding to the cell nucleus position annotation;

[0051] Based on the cell type contour, a contour is generated with the position of the cell nucleus position mark as the center to obtain the corresponding cell region mark;

[0052] According to the time taken by the operator to label each of the training images, the cell clarity labeling corresponding to each of the training images is determined.

[0053] As an optional embodiment, in the second aspect of the present invention, in the step of acquiring the training data set, determining the cell clarity annotation corresponding to each training image according to the time taken by the operator to annotate each training image includes:

[0054] For each of the training images, obtaining a movement trajectory and a corresponding movement speed of a marking cursor of the operator on the training image;

[0055] Screening out, from the plurality of image regions of the training image, a plurality of cell-dense regions whose corresponding cell region annotation quantity is greater than a preset quantity threshold;

[0056] Calculate the product of the length of the track portion of the marking cursor movement track in each of the cell-dense areas and the corresponding movement speed to obtain the area marking movement time corresponding to each of the cell-dense areas;

[0057] The weighted sum average of the moving times of the regional annotations corresponding to all the cell-dense areas is calculated to obtain the cell clarity annotations corresponding to the training image; wherein the weight corresponding to the moving time of the regional annotations corresponding to each of the cell-dense areas is inversely proportional to the number of annotations corresponding to the cell-dense area.

[0058] The third aspect of the present invention discloses another cell image recognition system based on model fine-tuning, the system comprising:

[0059] A memory storing executable program code;

[0060] a processor coupled to the memory;

[0061] The processor calls the executable program code stored in the memory to execute part or all of the steps in the cell image recognition method based on model fine-tuning disclosed in the first aspect of the present invention.

[0062] The fourth aspect of the present invention discloses a computer storage medium, which stores computer instructions. When the computer instructions are called, they are used to execute some or all of the steps in the cell image recognition method based on model fine-tuning disclosed in the first aspect of the present invention.

[0063] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0064] The present invention can first identify the cell clarity corresponding to the image based on the trained cell clarity recognition model, and then identify the corresponding regional image with a priori high probability of the existence of cells through the trained cell existence region recognition model, so as to finally obtain the corresponding cell region and the corresponding cell type based on the regional image of the existing cells and the cell clarity recognition based on the trained cell classification model, thereby making full use of the clarity factor and prior region recognition to accurately identify the cell region and type in the image, improve the accuracy and efficiency of cell recognition, and provide an accurate data basis for disease treatment or research. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0066] Figure 1 It is a flow chart of a cell image recognition method based on model fine-tuning disclosed in an embodiment of the present invention.

[0067] Figure 2 It is a structural schematic diagram of a cell image recognition system based on model fine-tuning disclosed in an embodiment of the present invention.

[0068] Figure 3 It is a structural schematic diagram of another cell image recognition system based on model fine-tuning disclosed in an embodiment of the present invention.

[0069] Figure 4 It is a schematic diagram of the recognition effect of the recognition model disclosed in the embodiment of the present invention.

[0070] Figure 5 It is a schematic diagram of the statistical results of the correlation coefficient of the prediction results of the algorithm identification solution disclosed in the embodiment of the present invention. DETAILED DESCRIPTION

[0071] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0072] The terms "first", "second", etc. in the specification of the present invention and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, device, product or equipment that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or equipment.

[0073] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0074] The present invention discloses a cell image recognition method and system based on model fine-tuning, which can first identify the cell clarity corresponding to the image based on a trained cell clarity recognition model, and then identify the corresponding regional image with a high probability of existing cells through a trained cell existence region recognition model, so as to finally obtain the corresponding cell region and the corresponding cell type based on the regional image of the existing cells and the cell clarity recognition based on the trained cell classification model, so as to make full use of the clarity factor and the prior regional recognition to accurately identify the cell region and type in the image, improve the accuracy and efficiency of cell recognition, and provide an accurate data basis for disease treatment or research. The following are detailed descriptions.

[0075] Embodiment 1

[0076] See also Figure 1 , Figure 1is a flow chart of a cell image recognition method based on model fine-tuning disclosed in an embodiment of the present invention. Figure 1 The described cell image recognition method based on model fine-tuning can be applied to a data processing system / data processing device / data processing server (wherein the server includes a local processing server or a cloud processing server). Figure 1 As shown, the cell image recognition method based on model fine-tuning may include the following operations:

[0077] 101. Obtain an image to be identified including a cell image.

[0078] 102. Input the image to be identified into the trained cell clarity recognition model to obtain the corresponding cell clarity.

[0079] 103. Input the image to be identified into the trained cell existing region identification model to obtain the corresponding cell existing region image.

[0080] 104. Input the existing cell region image and cell clarity into the trained cell classification model to obtain corresponding multiple cell regions and corresponding cell types.

[0081] Optionally, a cell clarity recognition model, a cell existence region recognition model and a cell classification model are obtained based on joint training and fine-tuning.

[0082] It can be seen that the above-mentioned invention embodiment can first identify the cell clarity corresponding to the image based on the trained cell clarity recognition model, and then identify the corresponding regional image with a priori high probability of the existence of cells through the trained cell existence region recognition model, so as to finally obtain the corresponding cell region and the corresponding cell type based on the regional image of the existing cells and the cell clarity recognition based on the trained cell classification model, thereby making full use of the clarity factor and prior region recognition to accurately identify the cell region and type in the image, improve the accuracy and efficiency of cell recognition, and provide an accurate data basis for disease treatment or research.

[0083] As an optional embodiment, in the above steps, the cell clarity recognition model, the cell existence area recognition model and the cell classification model are jointly trained based on the same training data set; the training data set includes multiple training images and corresponding cell area annotations, cell type annotations and cell clarity annotations.

[0084] It can be seen that through the above-mentioned optional embodiments, the three models can be jointly trained based on the same training data set to obtain recognition models responsible for different functions, so as to achieve accurate cell recognition, assist in making full use of clarity factors and prior region recognition to accurately identify cell regions and types in images, improve the accuracy and efficiency of cell recognition, and provide an accurate data basis for disease treatment or research.

[0085] As an optional embodiment, in the above steps, the cell clarity recognition model, the cell existence region recognition model and the cell classification model are trained by the following steps:

[0086] Inputting the training data set into the cell clarity recognition model and the cell existence region recognition model for training until the first prediction accuracy of the two reaches a preset first accuracy threshold;

[0087] Freezing the model weights of the cell clarity recognition model and the cell existence region recognition model, and inputting the training data set into the cell classification model for training until its second prediction accuracy reaches a preset second accuracy threshold; optionally, the second accuracy threshold is greater than the first accuracy threshold;

[0088] The model weights of the cell clarity recognition model and the cell existence area recognition model are unfrozen, and the training data set is simultaneously input into the cell clarity recognition model, the cell existence area recognition model and the cell classification model for joint training until the third prediction accuracy of the three reaches a preset third accuracy threshold, thereby obtaining a trained cell clarity recognition model, a cell existence area recognition model and a cell classification model; the third accuracy threshold is greater than the second accuracy threshold.

[0089] It can be seen that through the above-mentioned optional embodiments, based on operations such as step-by-step training steps and fine-tuning model weight freezing, three recognition model region recognition models can be gradually trained to achieve accurate cell recognition, assist in making full use of clarity factors and prior region recognition to accurately identify cell regions and types in images, improve the accuracy and efficiency of cell recognition, and provide an accurate data basis for disease treatment or research.

[0090] As an optional embodiment, in the above steps, the cell clarity recognition model is a convolutional neural network, which uses L1 loss as the loss function in training; the cell existence area recognition model and the cell classification model are both fully convolutional neural networks, which use cross entropy loss function as the loss function in training.

[0091] It can be seen that through the above optional embodiments, the model algorithm details and training details of the three recognition models are clarified to achieve accurate cell recognition, assist in making full use of clarity factors and prior region recognition to accurately identify cell regions and types in images, improve the accuracy and efficiency of cell recognition, and provide an accurate data basis for disease treatment or research.

[0092] As an optional embodiment, in the above steps, when the training data set is simultaneously input into the cell clarity recognition model, the cell existence region recognition model and the cell classification model for joint training, the loss function used in the joint training is the weighted sum of the first loss function value, the second loss function value and the third loss function value; wherein:

[0093] The first loss function value is the difference between the predicted clarity output by the cell clarity recognition model and the corresponding cell clarity annotation, and its corresponding weight is proportional to the variance of the cell clarity annotations corresponding to all training images in the input data at that time;

[0094] The second loss function value is the cross entropy between the predicted cell existence region output by the cell existence region recognition model and all corresponding cell region annotations, and its corresponding weight is proportional to the total area of ​​the union of all cell region annotations corresponding to all training images in the input data at that time;

[0095] The value of the third loss function is the cross entropy between the predicted cell region and predicted type output by the cell classification model and the corresponding cell region annotation and cell type annotation, and its corresponding weight is proportional to the total number of types corresponding to all cell type annotations corresponding to all training images in the input data at that time.

[0096] It can be seen that through the above-mentioned optional embodiments, the loss function setting in the joint training process can be comprehensively determined based on the preset weight rules and the loss function values ​​of the three models respectively, so as to obtain a recognition model with greater correlation and coordination between the three prediction results through joint training, so as to achieve accurate cell recognition, assist in making full use of clarity factors and prior area recognition to accurately identify cell areas and types in images, improve the accuracy and efficiency of cell recognition, and provide an accurate data basis for disease treatment or research.

[0097] As an optional embodiment, in the above steps, the cell type is PanCK+ cells, CD3+ cells, CD8- cells or CD8+ cells.

[0098] It can be seen that through the above optional embodiments, the type of identified cells is limited for accurate cell identification related to tumor treatment, which helps to make full use of clarity factors and prior area recognition to accurately identify cell areas and types in images, improve the accuracy and efficiency of cell recognition, and provide an accurate data basis for disease treatment or research.

[0099] As an optional embodiment, in the above steps, the training data set is obtained by the following steps:

[0100] Obtain multiple training images;

[0101] Based on the operator's annotations, determine the cell nucleus position annotations and cell type annotations corresponding to each training image;

[0102] For each cell nucleus position annotation, obtain the cell type outline corresponding to the cell type annotation corresponding to the cell nucleus position annotation;

[0103] Based on the cell type contour, the contour is generated with the position of the cell nucleus position annotation as the center to obtain the corresponding cell area annotation;

[0104] According to the time taken by the operator to label each training image, the cell clarity labeling corresponding to each training image is determined.

[0105] It can be seen that through the above-mentioned optional embodiments, on the one hand, cell area annotations can be automatically generated based on the cell nucleus position annotations and the cell contours of the cell type, and on the other hand, the cell clarity annotations can be determined based on the annotation time, so as to comprehensively improve the work efficiency of the annotation personnel, reduce the time cost, and reduce the burden on the annotation personnel. In the future, it is also possible to make full use of clarity factors and prior area recognition to accurately identify cell areas and types in images, improve the accuracy and efficiency of cell recognition, and provide an accurate data basis for disease treatment or research.

[0106] As an optional embodiment, in the above steps, in the step of acquiring the training data set, determining the cell clarity labeling corresponding to each training image according to the time taken by the operator to label each training image includes:

[0107] For each training image, obtaining a movement trajectory and a corresponding movement speed of an operator's marking cursor on the training image;

[0108] Screening out, from the plurality of image regions of the training image, a plurality of cell-dense regions whose corresponding cell region annotation quantity is greater than a preset quantity threshold;

[0109] Calculate the product of the length of the track portion of the marking cursor movement track in each cell-dense area and the corresponding movement speed to obtain the area marking movement time corresponding to each cell-dense area;

[0110] The weighted sum average of the time taken to move the regional annotations corresponding to all cell-dense areas is calculated to obtain the cell clarity annotations corresponding to the training image; wherein the weight corresponding to the time taken to move the regional annotations corresponding to each cell-dense area is inversely proportional to the number of annotations corresponding to the cell-dense area.

[0111] It can be seen that through the above-mentioned optional embodiments, it is possible to more accurately characterize the visibility of the operator's annotations on the image based on the operator's annotation movement trajectory on the training image and the number of cell area annotations in different areas. The obtained calculation results can effectively characterize whether the training image is easy to label and then characterize its cell clarity. In the subsequent training, a more accurate recognition model can be obtained, and full use of clarity factors and prior area recognition can be made to accurately identify cell areas and types in the image, thereby improving the accuracy and efficiency of cell recognition and providing an accurate data basis for disease treatment or research.

[0112] In a specific embodiment, a recognition model is constructed based on the scheme of the present invention, and its main purpose is to identify tumor cells (PanCK+), CD3+ / CD8- cells and CD8+ cells in digital slices. Specifically, the annotation of cells in the training images is performed by a combination of manual and machine methods. Among them, CD3+ and CD8+ cells are manually labeled. Since some CD8+ cells will express CD3+ positivity at the same time, in order to ensure that each cell in the label has only one category, the two categories manually labeled in the specific annotation process are CD8+ cells and CD3+ / CD8- cells. CD8+ and CD3+ / CD8- cells are merged to form all CD3+ cells. In order to simplify the doctor's annotation workload, the annotation is in the form of point annotation, that is, the doctor only needs to mark the center point position and nuclear category of the cell in the form of a point.

[0113] Since there are too many tumor cells in the image, manual annotation is too time-consuming, so the annotation of PanCK+ cells is automatically annotated by machine. This scheme first uses a pre-trained Stardist network to segment the nucleus based on the image of the DAPI channel. It was found that the segmentation effect of this model in the tumor epithelial area with high cell density needs to be improved. Therefore, based on the prediction results of the existing cell nucleus segmentation model, this scheme manually corrected the nucleus contour of the tumor epithelial area, and retrained and fine-tuned the Stardist model.

[0114] On the other hand, this scheme calculates the proportion of PanCK-positive pixels in each cell nuclear contour area. The identification of PanCK-positive pixels is obtained by adaptive thresholding of the PanCK channel. If the proportion of PanCK-positive pixels within a cell nuclear contour is higher than 50%, the contour is automatically identified as a PanCK+ cell. If a cell nuclear contour identified as PanCK+ contains the center point of a CD8+ or CD3+ / CD8- cell manually annotated by a doctor, the category of the cell contour is preferentially set to the category manually annotated by the doctor. Finally, for all cells determined to be PanCK+, a center point is generated according to their cell nuclear contour as the final annotation result. Specifically, this scheme further generates clarity annotations for each training image based on the time analysis of the doctor's annotation trajectory, but does not adopt a scheme in which the doctor directly gives clarity opinions. This is because clarity opinions are subjective and cannot accurately characterize the cell clarity annotations of the image.

[0115] Based on the above manual and machine annotation method, this solution obtained 66 3000*3000 pixel images with relevant annotations, and used them as data to train the cell recognition model. The 66 images were divided into 54 training, 8 validation, and 4 testing. The images have 4 channels, namely DAPI, PanCK, CD3 and CD8. Each channel was truncated according to the 99% percentile value of the pixel value distribution and the maximum-minimum normalization method was used to convert the pixel value to between 0 and 1.

[0116] The cell recognition model used in this scheme contains three branch models. The first is the clarity recognition model, which is a convolutional network. The second is the region detection model, which is an 80-layer deep fully convolutional network (FCN) used to identify the region with high confidence that the cell nucleus exists, that is, the target prior region. The third branch is the cell classification branch, whose input is the clarity prediction output by the first branch and the high-confidence cell nucleus existence region output by the second branch, so that the classification model can focus on the cell existence region and adjust the cell region segmentation scheme based on the clarity prediction of the image itself without being affected by a large amount of background areas.

[0117] During training, 128*128 size patches are randomly selected from each image as input to the network. A variety of image enhancement techniques are used, including random flipping, Gaussian noise, brightness enhancement, and sharpening. For the three branches, this scheme uses a weighted cross entropy loss function. The entire training is divided into three stages. First, the first two branches are trained to maximize the performance of cell prior region detection and clarity prediction. Then, this scheme freezes the first two branches and trains the third branch. Finally, this scheme jointly trains the three branches to achieve refined detection and classification. The first and second stages are trained for 100 rounds respectively, and the third stage is trained for 200 rounds with a batch size of 500. This scheme uses the Adam optimizer and combines it with a cosine warm-up learning rate strategy. After each round of training, the validation set is evaluated, and the F1 score of each type of cell and the average F1 score of the three types are calculated. Finally, the model selection determines the best weight based on the highest average F1 score.

[0118] Figure 4 The figure shows the recognition effect of the recognition model trained by this scheme. Figure 4 The left side shows the segmentation result of the original pre-trained Stardist model, and the right side shows the segmentation result of the model after fine-tuning and joint training. It can be seen that the new fine-tuned model has significantly improved the segmentation effect of cell nuclei in the tumor epithelial area.

[0119] Figure 5 The correlation coefficient between the number of cells manually counted and the number of cells identified by the machine obtained by statistically analyzing the prediction results of this scheme is shown.

[0120] Finally, this scheme uses the above model to perform cell detection in the tumor area and tumor infiltration front area identified in each slice to obtain the center point position of PanCK+, CD3+ / CD8- and CD8+ cells for the next step of feature extraction, which can effectively improve the accuracy of subsequent cell feature extraction and related research.

[0121] Embodiment 2

[0122] See also Figure 2 , Figure 2 is a schematic diagram of the structure of a cell image recognition system based on model fine-tuning disclosed in an embodiment of the present invention. Figure 2 The described cell image recognition system based on model fine-tuning can be applied to a data processing system / data processing device / data processing server (wherein the server includes a local processing server or a cloud processing server). Figure 2 As shown, the cell image recognition system based on model fine-tuning may include:

[0123] The acquisition module 201 is used to acquire the image to be identified including the cell image.

[0124] The first recognition module 202 is used to input the image to be recognized into the trained cell clarity recognition model to obtain the corresponding cell clarity.

[0125] The second recognition module 203 is used to input the image to be recognized into the trained cell existing region recognition model to obtain the corresponding cell existing region image.

[0126] The third recognition module 204 is used to input the image of the existing cell region and the cell clarity into the trained cell classification model to obtain corresponding multiple cell regions and corresponding cell types.

[0127] Optionally, a cell clarity recognition model, a cell existence region recognition model and a cell classification model are obtained based on joint training and fine-tuning.

[0128] It can be seen that the above-mentioned invention embodiment can first identify the cell clarity corresponding to the image based on the trained cell clarity recognition model, and then identify the corresponding regional image with a priori high probability of the existence of cells through the trained cell existence region recognition model, so as to finally obtain the corresponding cell region and the corresponding cell type based on the regional image of the existing cells and the cell clarity recognition based on the trained cell classification model, thereby making full use of the clarity factor and prior region recognition to accurately identify the cell region and type in the image, improve the accuracy and efficiency of cell recognition, and provide an accurate data basis for disease treatment or research.

[0129] As an optional embodiment, a cell clarity recognition model, a cell existence region recognition model and a cell classification model are jointly trained based on the same training data set; the training data set includes multiple training images and corresponding cell region annotations, cell type annotations and cell clarity annotations.

[0130] It can be seen that through the above-mentioned optional embodiments, the three models can be jointly trained based on the same training data set to obtain recognition models responsible for different functions, so as to achieve accurate cell recognition, assist in making full use of clarity factors and prior region recognition to accurately identify cell regions and types in images, improve the accuracy and efficiency of cell recognition, and provide an accurate data basis for disease treatment or research.

[0131] As an optional embodiment, the cell clarity recognition model, the cell existence region recognition model and the cell classification model are trained by the following steps:

[0132] Inputting the training data set into the cell clarity recognition model and the cell existence region recognition model for training until the first prediction accuracy of the two reaches a preset first accuracy threshold;

[0133] Freezing the model weights of the cell clarity recognition model and the cell existence region recognition model, and inputting the training data set into the cell classification model for training until its second prediction accuracy reaches a preset second accuracy threshold; optionally, the second accuracy threshold is greater than the first accuracy threshold;

[0134] The model weights of the cell clarity recognition model and the cell existence area recognition model are unfrozen, and the training data set is simultaneously input into the cell clarity recognition model, the cell existence area recognition model and the cell classification model for joint training until the third prediction accuracy of the three reaches a preset third accuracy threshold, thereby obtaining a trained cell clarity recognition model, a cell existence area recognition model and a cell classification model; the third accuracy threshold is greater than the second accuracy threshold.

[0135] It can be seen that through the above-mentioned optional embodiments, based on operations such as step-by-step training steps and fine-tuning model weight freezing, three recognition model region recognition models can be gradually trained to achieve accurate cell recognition, assist in making full use of clarity factors and prior region recognition to accurately identify cell regions and types in images, improve the accuracy and efficiency of cell recognition, and provide an accurate data basis for disease treatment or research.

[0136] As an optional embodiment, the cell clarity recognition model is a convolutional neural network, which uses L1 loss as the loss function during training; the cell existence area recognition model and the cell classification model are both fully convolutional neural networks, which use cross entropy loss function as the loss function during training.

[0137] It can be seen that through the above optional embodiments, the model algorithm details and training details of the three recognition models are clarified to achieve accurate cell recognition, assist in making full use of clarity factors and prior region recognition to accurately identify cell regions and types in images, improve the accuracy and efficiency of cell recognition, and provide an accurate data basis for disease treatment or research.

[0138] As an optional embodiment, when the training data set is simultaneously input into the cell clarity recognition model, the cell existence region recognition model and the cell classification model for joint training, the loss function used in the joint training is the weighted sum of the first loss function value, the second loss function value and the third loss function value; wherein:

[0139] The first loss function value is the difference between the predicted clarity output by the cell clarity recognition model and the corresponding cell clarity annotation, and its corresponding weight is proportional to the variance of the cell clarity annotations corresponding to all training images in the input data at that time;

[0140] The second loss function value is the cross entropy between the predicted cell existence region output by the cell existence region recognition model and all corresponding cell region annotations, and its corresponding weight is proportional to the total area of ​​the union of all cell region annotations corresponding to all training images in the input data at that time;

[0141] The value of the third loss function is the cross entropy between the predicted cell region and predicted type output by the cell classification model and the corresponding cell region annotation and cell type annotation, and its corresponding weight is proportional to the total number of types corresponding to all cell type annotations corresponding to all training images in the input data at that time.

[0142] It can be seen that through the above-mentioned optional embodiments, the loss function setting in the joint training process can be comprehensively determined based on the preset weight rules and the loss function values ​​of the three models respectively, so as to obtain a recognition model with greater correlation and coordination between the three prediction results through joint training, so as to achieve accurate cell recognition, assist in making full use of clarity factors and prior area recognition to accurately identify cell areas and types in images, improve the accuracy and efficiency of cell recognition, and provide an accurate data basis for disease treatment or research.

[0143] As an optional embodiment, the cell type is PanCK+ cell, CD3+ cell, CD8- cell or CD8+ cell.

[0144] It can be seen that through the above optional embodiments, the type of identified cells is limited for accurate cell identification related to tumor treatment, which helps to make full use of clarity factors and prior area recognition to accurately identify cell areas and types in images, improve the accuracy and efficiency of cell recognition, and provide an accurate data basis for disease treatment or research.

[0145] As an optional embodiment, the training data set is obtained by the following steps:

[0146] Obtain multiple training images;

[0147] Based on the operator's annotations, determine the cell nucleus position annotations and cell type annotations corresponding to each training image;

[0148] For each cell nucleus position annotation, obtain the cell type outline corresponding to the cell type annotation corresponding to the cell nucleus position annotation;

[0149] Based on the cell type contour, the contour is generated with the position of the cell nucleus position annotation as the center to obtain the corresponding cell area annotation;

[0150] According to the time taken by the operator to label each training image, the cell clarity labeling corresponding to each training image is determined.

[0151] It can be seen that through the above-mentioned optional embodiments, on the one hand, cell area annotations can be automatically generated based on the cell nucleus position annotations and the cell contours of the cell type, and on the other hand, the cell clarity annotations can be determined based on the annotation time, so as to comprehensively improve the work efficiency of the annotation personnel, reduce the time cost, and reduce the burden on the annotation personnel. In the future, it is also possible to make full use of clarity factors and prior area recognition to accurately identify cell areas and types in images, improve the accuracy and efficiency of cell recognition, and provide an accurate data basis for disease treatment or research.

[0152] As an optional embodiment, in the step of acquiring the training data set, determining the cell clarity labeling corresponding to each training image according to the time taken by the operator to label each training image includes:

[0153] For each training image, obtaining a movement trajectory and a corresponding movement speed of an operator's marking cursor on the training image;

[0154] Screening out, from the plurality of image regions of the training image, a plurality of cell-dense regions whose corresponding cell region annotation quantity is greater than a preset quantity threshold;

[0155] Calculate the product of the length of the track portion of the marking cursor movement track in each cell-dense area and the corresponding movement speed to obtain the area marking movement time corresponding to each cell-dense area;

[0156] The weighted sum average of the time taken to move the regional annotations corresponding to all cell-dense areas is calculated to obtain the cell clarity annotations corresponding to the training image; wherein the weight corresponding to the time taken to move the regional annotations corresponding to each cell-dense area is inversely proportional to the number of annotations corresponding to the cell-dense area.

[0157] It can be seen that through the above-mentioned optional embodiments, it is possible to more accurately characterize the visibility of the operator's annotations on the image based on the operator's annotation movement trajectory on the training image and the number of cell area annotations in different areas. The obtained calculation results can effectively characterize whether the training image is easy to label and then characterize its cell clarity. In the subsequent training, a more accurate recognition model can be obtained, and full use of clarity factors and prior area recognition can be made to accurately identify cell areas and types in the image, thereby improving the accuracy and efficiency of cell recognition and providing an accurate data basis for disease treatment or research.

[0158] Embodiment 3

[0159] See also Figure 3 , Figure 3 It is another cell image recognition system based on model fine-tuning disclosed in an embodiment of the present invention. Figure 3The described cell image recognition system based on model fine-tuning is applied to a data processing system / data processing device / data processing server (wherein the server includes a local processing server or a cloud processing server). Figure 3 As shown, the cell image recognition system based on model fine-tuning may include:

[0160] A memory 301 storing executable program codes;

[0161] a processor 302 coupled to the memory 301;

[0162] The processor 302 calls the executable program code stored in the memory 301 to execute the steps of the cell image recognition method based on model fine-tuning described in the first embodiment.

[0163] Embodiment 4

[0164] An embodiment of the present invention discloses a computer-readable storage medium storing a computer program for electronic data exchange, wherein the computer program enables a computer to execute the steps of the cell image recognition method based on model fine-tuning described in the first embodiment.

[0165] Embodiment 5

[0166] An embodiment of the present invention discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to enable a computer to execute the steps of the cell image recognition method based on model fine-tuning described in Example 1.

[0167] The above describes specific embodiments of the present specification. In some cases, the actions or steps described can be performed in a different order than those in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily have to be performed in the specific order or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0168] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0169] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0170] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, the embodiments of this specification may be in the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the embodiments of this specification may be in the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0171] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0172] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0173] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0174] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0175] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0176] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0177] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0178] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0179] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0180] Finally, it should be noted that the cell image recognition method and system based on model fine-tuning disclosed in the embodiment of the present invention only discloses the preferred embodiment of the present invention, which is only used to illustrate the technical solution of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features therein can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A cell image recognition method based on model fine-tuning, characterized in that: The method comprises: Acquiring an image to be identified including a cell image; Inputting the image to be identified into a trained cell definition recognition model to obtain the corresponding cell definition; Inputting the image to be identified into a trained cell existing region recognition model to obtain a corresponding cell existing region image; The cell area image and the cell clarity are input into a trained cell classification model to obtain a plurality of corresponding cell areas and corresponding cell types; the cell clarity recognition model, the cell area recognition model and the cell classification model are obtained based on joint training and fine-tuning; the cell clarity recognition model, the cell area recognition model and the cell classification model are obtained by joint training based on the same training data set; the training data set includes a plurality of training images and corresponding cell area annotations, cell type annotations and cell clarity annotations; the cell clarity recognition model, the cell area recognition model and the cell classification model are obtained by training through the following steps: Inputting the training data set into the cell clarity recognition model and the cell existence region recognition model for training until the first prediction accuracy of the two reaches a preset first accuracy threshold; Freezing the model weights of the cell clarity recognition model and the cell existence region recognition model, inputting the training data set into the cell classification model for training until its second prediction accuracy reaches a preset second accuracy threshold; the second accuracy threshold is greater than the first accuracy threshold; The model weights of the cell clarity recognition model and the cell existence area recognition model are unfrozen, and the training data set is simultaneously input into the cell clarity recognition model, the cell existence area recognition model and the cell classification model for joint training until the third prediction accuracy of the three reaches a preset third accuracy threshold, thereby obtaining the trained cell clarity recognition model, the cell existence area recognition model and the cell classification model; the third accuracy threshold is greater than the second accuracy threshold.

2. The cell image recognition method based on model fine-tuning according to claim 1, characterized in that: The cell clarity recognition model is a convolutional neural network, which uses L1 loss as a loss function during training; the cell existence area recognition model and the cell classification model are both fully convolutional neural networks, which use cross entropy loss function as a loss function during training.

3. The cell image recognition method based on model fine-tuning according to claim 1, characterized in that: When the training data set is simultaneously input into the cell clarity recognition model, the cell existence region recognition model and the cell classification model for joint training, the loss function used in the joint training is a weighted sum of the first loss function value, the second loss function value and the third loss function value; wherein: The first loss function value is the difference between the predicted clarity output by the cell clarity recognition model and the corresponding cell clarity annotation, and the corresponding weight is proportional to the variance of the cell clarity annotations corresponding to all training images in the current input data; The second loss function value is the cross entropy between the predicted cell existence region output by the cell existence region recognition model and all the corresponding cell region annotations, and the corresponding weight is proportional to the total area of ​​the region union of all cell region annotations corresponding to all training images in the current input data; The third loss function value is the cross entropy between the predicted cell region and predicted type output by the cell classification model and the corresponding cell region annotation and cell type annotation, and its corresponding weight is proportional to the total number of types corresponding to all cell type annotations corresponding to all training images in the input data at that time.

4. The cell image recognition method based on model fine-tuning according to claim 1, characterized in that: The cell type is PanCK+ cell, CD3+ cell, CD8- cell or CD8+ cell.

5. The cell image recognition method based on model fine-tuning according to claim 1, characterized in that: The training data set is obtained by the following steps: Obtain multiple training images; Determining the cell nucleus position annotation and cell type annotation corresponding to each of the training images based on the annotations of the operator; For each of the cell nucleus position annotations, obtaining a cell type outline corresponding to the cell type annotation corresponding to the cell nucleus position annotation; Based on the cell type contour, a contour is generated with the position of the cell nucleus position mark as the center to obtain the corresponding cell region mark; According to the time taken by the operator to label each of the training images, the cell clarity labeling corresponding to each of the training images is determined.

6. The cell image recognition method based on model fine-tuning according to claim 5, characterized in that: In the step of acquiring the training data set, determining the cell clarity labeling corresponding to each training image according to the time taken by the operator to label each training image includes: For each of the training images, obtaining a movement trajectory and a corresponding movement speed of a marking cursor of the operator on the training image; Screening out, from the plurality of image regions of the training image, a plurality of cell-dense regions whose corresponding cell region annotation quantity is greater than a preset quantity threshold; Calculate the product of the length of the track portion of the marking cursor movement track in each of the cell-dense areas and the corresponding movement speed to obtain the area marking movement time corresponding to each of the cell-dense areas; The weighted sum average of the moving times of the regional annotations corresponding to all the cell-dense areas is calculated to obtain the cell clarity annotations corresponding to the training image; wherein the weight corresponding to the moving time of the regional annotations corresponding to each of the cell-dense areas is inversely proportional to the number of annotations corresponding to the cell-dense area.

7. A cell image recognition system based on model fine-tuning, characterized in that: The system comprises: An acquisition module, used for acquiring an image to be identified including a cell image; A first recognition module, used for inputting the image to be recognized into a trained cell definition recognition model to obtain the corresponding cell definition; A second recognition module is used to input the image to be recognized into a trained cell existing region recognition model to obtain a corresponding cell existing region image; The third recognition module is used to input the cell area image and the cell clarity into the trained cell classification model to obtain multiple corresponding cell areas and corresponding cell types; the cell clarity recognition model, the cell area recognition model and the cell classification model are obtained based on joint training and fine-tuning; the cell clarity recognition model, the cell area recognition model and the cell classification model are obtained by joint training based on the same training data set; the training data set includes multiple training images and corresponding cell area annotations, cell type annotations and cell clarity annotations; the cell clarity recognition model, the cell area recognition model and the cell classification model are trained through the following steps: Inputting the training data set into the cell clarity recognition model and the cell existence region recognition model for training until the first prediction accuracy of the two reaches a preset first accuracy threshold; Freezing the model weights of the cell clarity recognition model and the cell existence region recognition model, inputting the training data set into the cell classification model for training until its second prediction accuracy reaches a preset second accuracy threshold; the second accuracy threshold is greater than the first accuracy threshold; The model weights of the cell clarity recognition model and the cell existence area recognition model are unfrozen, and the training data set is simultaneously input into the cell clarity recognition model, the cell existence area recognition model and the cell classification model for joint training until the third prediction accuracy of the three reaches a preset third accuracy threshold, thereby obtaining the trained cell clarity recognition model, the cell existence area recognition model and the cell classification model; the third accuracy threshold is greater than the second accuracy threshold.

8. A cell image recognition system based on model fine-tuning, characterized in that: The system comprises: A memory storing executable program code; a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the cell image recognition method based on model fine-tuning as described in any one of claims 1-6.

Images