Cell image recognition method, device, equipment, medium and computer program product

Abstract

The present application discloses a cell image recognition method, apparatus, device, medium, and computer program product, relating to the field of deep learning. The method comprises: obtaining a sample data set; inputting the sample data set into a cell nucleus recognition model, outputting a segmentation prediction result of a first cell image and a coloring prediction result of a second cell image; obtaining a segmentation loss value based on the cell nucleus information of the first cell image and the segmentation prediction result; obtaining a coloring loss value based on the second cell image and the coloring prediction result; training the cell nucleus recognition model based on the segmentation loss value and the coloring loss value to obtain a target recognition model, which is used to identify cell nuclei in cell images. That is, the cell nucleus recognition model is input into the cell nucleus recognition model through the cell image, and the cell nucleus recognition model is trained based on the segmentation loss value and the coloring loss value to obtain a target recognition model, thereby improving the model accuracy and recognition efficiency of cell recognition.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of deep learning, and in particular to a cell image recognition method, apparatus, device, medium, and computer program product. Background Art

[0002] Histological examination is a method of screening for pathological tissues and a type of clinical case examination. It can help diagnose many diseases, especially some suspected malignant diseases, such as cervical cancer or pancreatic cancer.

[0003] In related technologies, cell nucleus segmentation is a key step in the automatic analysis of histopathological images in histological examinations. Cell nucleus segmentation often uses the method of threshold segmentation of the cell nucleus, and performs relevant preprocessing on the cell image through morphological operations to improve the quality of the cell image, thereby obtaining the cell nucleus segmentation result.

[0004] However, the cell nucleus segmentation results obtained by relying solely on morphological operations are often quite different from the actual cell nucleus state, and the segmentation accuracy is relatively low, which affects the diagnosis results. Therefore, there is a possibility of misdiagnosis and missed diagnosis. Summary of the Invention

[0005] The present invention provides a cell image recognition method, apparatus, device, medium, and computer program product that can improve the accuracy of cell nucleus recognition results. The technical solution is as follows:

[0006] In one aspect, a cell image recognition method is provided, the method comprising:

[0007] Acquire a sample data set, wherein the sample data set includes a first cell image and a second cell image, wherein the first cell image is annotated with cell nucleus information;

[0008] Inputting the sample data set into a cell nucleus recognition model, and outputting a segmentation prediction result of the first cell image and a coloring prediction result of the second cell image;

[0009] Obtaining a segmentation loss value based on the cell nucleus information of the first cell image and the segmentation prediction result;

[0010] Obtaining a coloring loss value based on the second cell image and the coloring prediction result;

[0011] The cell nucleus recognition model is trained based on the segmentation loss value and the coloring loss value to obtain a target recognition model, and the target recognition model is used to recognize cell nuclei in cell images.

[0012] In another aspect, a cell image recognition device is provided, comprising:

[0013] A sample acquisition module, configured to acquire a sample data set, wherein the sample data set includes a first cell image and a second cell image, wherein the first cell image is annotated with cell nucleus information;

[0014] an output module, configured to input the sample data set into a cell nucleus recognition model, and output a segmentation prediction result of the first cell image and a coloring prediction result of the second cell image;

[0015] a loss acquisition module, configured to obtain a segmentation loss value based on the cell nucleus information of the first cell image and the segmentation prediction result;

[0016] The loss acquisition module is further configured to obtain a coloring loss value based on the second cell image and the coloring prediction result;

[0017] A training module is used to train the cell nucleus recognition model based on the segmentation loss value and the coloring loss value to obtain a target recognition model, and the target recognition model is used to recognize cell nuclei in cell images.

[0018] On the other hand, a computer device is provided, comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the cell image recognition method as described in any of the above-mentioned embodiments of the present application.

[0019] On the other hand, a computer-readable storage medium is provided, wherein the storage medium stores at least one instruction, at least one program, code set or instruction set, and the at least one instruction, the at least one program, the code set or instruction set is loaded and executed by a processor to implement the cell image recognition method as described in any of the above-mentioned embodiments of the present application.

[0020] In another aspect, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the cell image recognition method described in any of the above embodiments.

[0021] The beneficial effects of the technical solutions provided in the embodiments of the present application include at least:

[0022] By inputting the first cell image and the second cell image in the sample data set into the cell nucleus recognition model for segmentation prediction training and coloring prediction training respectively, the segmentation prediction result corresponding to the first cell image with label and the coloring prediction result corresponding to the second cell image without label are obtained, and the corresponding segmentation loss value and coloring loss value are calculated. The cell nucleus recognition model is trained based on the segmentation loss value and the coloring loss value to improve the model accuracy of the cell nucleus recognition model, thereby improving the accuracy and recognition efficiency of cell nucleus recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0024] Figure 1 This is a flowchart of the overall solution implementation provided by an exemplary embodiment of the present application;

[0025] Figure 2 is a schematic diagram of an implementation environment of a cell image recognition method provided by an exemplary embodiment of the present application;

[0026] Figure 3 is a flow chart of a cell image recognition method provided by an exemplary embodiment of the present application;

[0027] Figure 4 is a flow chart of a cell image recognition method provided by another exemplary embodiment of the present application;

[0028] Figure 5 is a flow chart of a cell image recognition method provided by another exemplary embodiment of the present application;

[0029] Figure 6 is a schematic diagram of obtaining a grayscale image provided by an embodiment of the present application;

[0030] Figure 7 is a schematic diagram of a cell image recognition process provided by an exemplary embodiment of the present application;

[0031] Figure 8 is a schematic structural diagram of a cell image recognition device provided by an exemplary embodiment of the present application;

[0032] Figure 9 is a schematic structural diagram of a cell image recognition device provided by another exemplary embodiment of the present application;

[0033] Figure 10It is a structural diagram of a server provided by an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0034] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0035] First, a brief introduction to the terms involved in the embodiments of this application is given:

[0036] Artificial Intelligence (AI) is the theory, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that seeks to understand the essence of intelligence and produce new intelligent machines that can respond in a manner similar to human intelligence. AI also studies the design principles and implementation methods of various intelligent machines, enabling them to possess the capabilities of perception, reasoning, and decision-making.

[0037] Artificial intelligence (AI) technology is a comprehensive discipline encompassing a wide range of fields, encompassing both hardware and software technologies. Foundational AI technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0038] Machine Learning (ML) is a multidisciplinary field that encompasses probability theory, statistics, approximation theory, convex analysis, and algorithmic complexity theory. It specifically studies how computers can simulate or implement human learning behaviors to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is at the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications span all areas of AI. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and learning through demonstration.

[0039] Computer vision (CV) is the science of making machines "see." Specifically, it refers to using cameras and computers to replace the human eye in identifying, tracking, and measuring objects. It also performs image processing to create images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, aiming to build artificial intelligence systems that can extract information from images or multidimensional data. Computer vision technologies generally include image processing, image recognition, image semantic understanding, image retrieval, optical character recognition (OCR), video processing, video semantic understanding, video content / action recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping, and common biometric recognition technologies such as facial recognition and fingerprint recognition.

[0040] Hematoxylin-eosin staining (HE staining): HE staining is a fundamental and widely used technique in histology, embryology, and pathology teaching and research. It is commonly used in paraffin sections. Because hematoxylin is alkaline, it stains chromatin within the cell nucleus and nucleic acids within the cytoplasm purple-blue; while eosin, an acidic dye, stains components of the cytoplasm and extracellular matrix red.

[0041] Multi-Organ Nucleus Segmentation (MoNuSeg): A public nucleus segmentation dataset consisting of 40x magnified HE-stained images downloaded from The Cancer Genome Atlas (TCGA) archive. It was obtained by accurately annotating histopathological images of different tumor organs from multiple patients in multiple hospitals.

[0042] Grayscale image colorization: A method that uses neural networks to convert black-and-white images into color. In this application example, this method specifically involves colorizing black-and-white MRI, X-ray, and CT images in the medical field, converting them into color images and more fully displaying their features, helping doctors identify symptoms more quickly.

[0043] Histopathology plays a vital role in the diagnosis, prognosis and treatment decisions of cancer, among which cell nucleus segmentation is a key step in the automatic analysis of histopathology images. In related technologies, cell segmentation methods often use threshold segmentation methods to segment cells, and use a series of pre-processing methods such as blurring the cell image and enhancing the contrast through some morphological operations to assist in improving the quality of the cell image. However, the obtained cell nucleus segmentation results are often quite different from the true morphology of the cell nucleus, so the accuracy of threshold segmentation is poor. In the cell nucleus segmentation training of cell images using deep learning, the probability of cell nucleus and non-cell nucleus areas is obtained by semantically segmenting the cell image, which greatly improves the accuracy of cell nucleus segmentation.

[0044] Currently, most commonly used deep learning-based cell nucleus segmentation methods require pixel-level labeling for training. The quality of pixel-level labels directly determines the performance of the cell nucleus segmentation model. Manual pixel-level labeling of cell nuclei is also difficult and labor-intensive. Some cell nucleus segmentation models based on weak labels, due to incomplete labels, rely solely on weak labels obtained through point annotation or scribble annotation for training. The trained models produce results that differ from the true nucleus morphology during actual cell nucleus segmentation.

[0045] An embodiment of the present application provides a cell image recognition method, which inputs the first cell image and the second cell image in the sample data set into a cell nucleus recognition model for segmentation prediction training and coloring prediction training respectively, obtains the segmentation prediction result corresponding to the first cell image with a label and the coloring prediction result corresponding to the second cell image without a label, calculates the corresponding segmentation loss value and coloring loss value, and trains the cell nucleus recognition model based on the segmentation loss value and the coloring loss value to improve the model accuracy of the cell nucleus recognition model, thereby improving the accuracy and recognition efficiency of cell nucleus recognition.

[0046] The present application embodiment provides a cell image recognition method, schematically, Figure 1 This is a flowchart of the overall solution implementation provided by an embodiment of the present application, such as Figure 1 shown.

[0047] First, a sample dataset 100 is obtained. The sample dataset 100 includes a first cell image 101 and a second cell image 102. Both the first cell image 101 and the second cell image 102 include HE-stained pathological images. The first cell image 101 includes annotation information for annotating cell nucleus information corresponding to the HE-stained pathological image. The second cell image 102 may or may not include annotation information. In this embodiment, when the second cell image 102 includes annotation information, the image portion of the second cell image 102 needs to be applied during application.

[0048] By performing grayscale processing on the sample dataset 100, a grayscale image 110 corresponding to the sample dataset 100 is obtained. The grayscale image 110 is input into the cell nucleus recognition model 120. The cell nucleus recognition model 120 performs feature extraction on the grayscale image 110 to obtain intermediate features 121 corresponding to the grayscale image 110. The cell nucleus recognition model 120 also includes a segmentation sub-model 122 and a coloring sub-model 123. The intermediate features 121 are input into the segmentation sub-model 122 and the coloring sub-model 123, respectively, to obtain a segmentation prediction result 124 and a coloring prediction result 125 corresponding to the intermediate features 121. A loss value 130 is obtained based on the segmentation prediction result 124 and the coloring prediction result 125. A segmentation loss value 131 is obtained based on the segmentation prediction result 124 and the cell nucleus information of the first cell image, and a coloring loss value 132 is obtained based on the second cell image 102 and the coloring prediction result 125. The cell nucleus recognition model 120 is trained based on the segmentation loss value 131 and the coloring loss value 132, ultimately obtaining an object recognition model 140.

[0049] Next, the implementation environment involved in the embodiments of this application is described. For schematic illustration, please refer to Figure 2 The implementation environment involves a terminal 210 and a server 220 , and the terminal 210 and the server 220 are connected via a communication network 230 .

[0050] Terminal 210 is used to send histopathology images to server 220. Illustratively, terminal 210 is used by doctors, who, when diagnosing a condition based on histopathology images, use a cell nucleus recognition model for auxiliary diagnosis, thereby improving diagnostic accuracy. Alternatively, terminal 210 is used by users, such as patients themselves or their relatives, who send histopathology images to server 220 to obtain reference diagnostic results. Alternatively, terminal 210 is connected to a histopathology image scanning device, which scans and transmits the scanned histopathology images to terminal 210. After receiving the histopathology images, terminal 210 forwards the histopathology images to server 220 for auxiliary diagnosis.

[0051] The server 220 passes the above Figure 1 After training the cell nucleus recognition model 221 in the manner shown, and obtaining the trained target recognition model 222, the server 220 receives the tissue pathology image uploaded by the terminal 210 and performs cell nucleus recognition on the tissue pathology image using the target recognition model 222 to obtain a cell nucleus recognition result. The server 220 feeds the cell nucleus recognition result back to the terminal 210.

[0052] The above-mentioned terminal can be a mobile phone, tablet computer, desktop computer, portable notebook computer and other terminal devices in various forms, and the embodiments of the present application are not limited to this.

[0053] It is worth noting that the above-mentioned servers can be independent physical servers, or they can be server clusters or distributed systems composed of multiple physical servers. They can also be cloud servers that provide basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), as well as big data and artificial intelligence platforms.

[0054] Cloud technology refers to a hosting technology that unifies hardware, software, and network resources within a wide area network (WAN) or local area network (LAN) to enable data computing, storage, processing, and sharing. Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, and application technology, all based on the cloud computing business model. It can form a resource pool that can be used on demand with flexibility and convenience. Cloud computing technology will become a crucial support. Backend services for technical network systems, such as video websites, image websites, and more portals, require significant computing and storage resources. With the rapid development and application of the internet industry, every item will likely have its own unique identification mark and will need to be transmitted to backend systems for logical processing. Data of varying levels will be processed separately, and data from all industries will require a strong system backend, which can only be achieved through cloud computing.

[0055] In some embodiments, the aforementioned servers can also be implemented as nodes in a blockchain system. Blockchain is a novel application model for computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, a blockchain is a decentralized database, a series of data blocks generated using cryptographic methods. Each block contains information about a batch of network transactions, used to verify the validity of the information (to prevent counterfeiting) and generate the next block. A blockchain can include an underlying blockchain platform, a platform product and service layer, and an application service layer.

[0056] In combination with the above-mentioned noun introduction, the application scenarios involved in the embodiments of this application are illustrated by way of example:

[0057] First, the scenario where doctors use cell nucleus recognition models for auxiliary diagnosis.

[0058] That is, the doctor sends the tissue pathology image to the server through the terminal. The server uses the trained target recognition model to perform cell nucleus recognition on the tissue pathology image, obtains the cell nucleus recognition result corresponding to the tissue pathology image, and feeds back the cell nucleus recognition result to the terminal used by the doctor for display. The doctor then uses the cell nucleus recognition result to perform auxiliary diagnosis and analysis on the tissue pathology image and obtain the final diagnosis result.

[0059] Second, users make diagnoses through target recognition models.

[0060] The user (patient or relative of the patient) sends the tissue pathology image to the server, which includes a pathology diagnosis result database. The server uses a trained target recognition model to identify the cell nucleus of the tissue pathology image, obtains the cell nucleus recognition result corresponding to the tissue pathology image, and compares the cell nucleus recognition result with the pathology diagnosis result database in the server to obtain the corresponding preliminary diagnosis result. The preliminary diagnosis result is fed back to the user's application terminal for display. Based on the preliminary diagnosis result, the user first has a preliminary understanding of the abnormal life status, and then obtains a detailed diagnosis result through the doctor's diagnosis.

[0061] Combined with the above-mentioned noun introduction and application scenarios, the cell image recognition method provided by this application is described. Taking the application of this method in a server as an example, Figure 3 As shown, the method includes:

[0062] Step 301: Obtain a sample data set.

[0063] The sample data set includes a first cell image and a second cell image, wherein the first cell image is annotated with cell nucleus information, and the second cell image is annotated with or without cell nucleus information.

[0064] Among them, the first cell image is used to represent a class of images labeled with cell nucleus information, that is, the sample data set includes at least one first cell image; similarly, the second cell image is used to represent a class of images that do not need to be labeled with cell nucleus information, that is, the sample data set includes at least one second cell image.

[0065] In some embodiments, the sample dataset is obtained by at least one of the following methods:

[0066] 1. Obtain sample datasets from publicly available cell nucleus image collections from medical competitions (e.g., MoNuSeg dataset);

[0067] 2. Obtain sample datasets from cell nucleus image sets publicly discussed in medical lectures or cell nucleus image sets mentioned in public medical papers and make reasonable citations;

[0068] 3. Collect sample datasets from cell nucleus images that are publicly available in medical forums and that have been authorized for use;

[0069] 4. Obtain cell nucleus images from the hospital's tissue pathology image library for which use rights have been granted.

[0070] It is worth noting that the above-mentioned method for obtaining the sample data set is only an illustrative example, and there is no specific limitation on the method for obtaining the sample data set in the embodiments of the present application.

[0071] In some embodiments, the cell nucleus information is annotated by medical personnel based on medical knowledge and diagnostic experience after observing the cell image. After the medical personnel annotate the cell nucleus information on the cell image, they send the cell nucleus information to a server, which obtains the cell nucleus information corresponding to the cell image.

[0072] In the embodiment of the present application, the annotated cell nucleus information includes at least one of the following information: cell nucleus boundary information; or cell nucleus distribution characteristic information; or cell nucleus size information, etc., which are not limited here.

[0073] In some embodiments, the first cell image and the second cell image may be histopathological images of the same organ or the same body part; or, they may be histopathological images of different organs or different body parts.

[0074] In some embodiments, during the acquisition of a pathological tissue image, a sample is first collected from the organism and smeared onto a glass slide to obtain a smear, which is then used to capture the pathological tissue image from the smear using a film reading device (such as an optical microscope, an electron microscope, etc.). For example, in the case of a human organism, the collected pathological tissue sample includes at least one of naturally exfoliated cell tissue from human body fluids, mucosal cell tissue, and human tissue cells obtained by fine needle aspiration or ultrasound-guided puncture.

[0075] Step 302: Input the sample data set into the cell nucleus recognition model, and output the segmentation prediction result of the first cell image and the coloring prediction result of the second cell image.

[0076] In the embodiment of the present application, the cell nucleus recognition model identifies and analyzes each pixel of the cell image in the sample data set. Its recognition function may include at least one of the following forms:

[0077] 1. Used to distinguish the cell nucleus from the non-cell nucleus (such as cytoplasm, extracellular matrix, etc.) in a cell image, that is, to determine whether each pixel unit in the current input cell image is a cell nucleus; where a pixel unit is a pixel point, or a pixel unit is a pixel array composed of multiple pixels points;

[0078] 2. Used to obtain the distribution characteristics of cell nuclei in the cell images in the input sample data set, that is, the number of cell nuclei per unit area of ​​the image;

[0079] 3. Used to calculate the average size of cell nuclei in the input cell image, that is, the average area of ​​a single cell nucleus in the image area.

[0080] It is worth noting that the above description of the role of the cell nucleus recognition model is only an illustrative example, and the specific role of the cell nucleus recognition model is not limited in any way in the embodiments of the present application.

[0081] In an embodiment of the present application, the segmentation prediction result of the first cell image is the judgment result about the cell nucleus boundary information obtained by performing regional segmentation on the cell nucleus in the first cell image, and the coloring prediction result of the second cell image is to perform grayscale image coloring on the cell nucleus in the second cell image to obtain a color image of the second cell image, and obtain the boundary information about the cell nucleus in the second cell image based on the coloring result corresponding to its color image.

[0082] Step 303: Obtain a segmentation loss value based on the cell nucleus information of the first cell image and the segmentation prediction result.

[0083] In some embodiments, a segmentation loss value of the cell nucleus information of the first cell image and the segmentation prediction result is obtained based on a cross entropy loss function.

[0084] Optionally, the cross entropy loss function can refer to Formula 1:

[0085] Formula 1: L ce =-[y L ·logy sL +(1-y L )·log(1-y sL )]

[0086] Among them, L ce is the segmentation loss value, y L is the nucleus information of the first cell image, y sL is the segmentation prediction result of the first cell image. From Formula 1, it can be seen that when the segmentation prediction result of the first cell image is closer to its cell nucleus information, the smaller the segmentation loss value is, the more accurate the segmentation prediction result of the cell nucleus is.

[0087] In the embodiment of the present application, the segmentation loss value is used to describe the error between the segmentation prediction result of the first cell image and its corresponding cell nucleus information, such as edge error, size error, distribution error, etc.

[0088] Step 304 : Obtain a coloring loss value based on the second cell image and the coloring prediction result.

[0089] In some embodiments, a coloring loss value is obtained based on a squared residual between the second cell image and the coloring prediction result.

[0090] Optionally, the calculation formula for the shading loss value can refer to Formula 2:

[0091] Formula 2: L MSE =(y cU -r U )

[0092] Among them, L MSE Refers to the shading loss value, y cU Refers to the coloring prediction result of the second cell image, r U is the second cell image. As can be seen from Formula 2, when the residual square value between the second cell image and its coloring prediction result is smaller, the coloring loss value is smaller, which indicates that the coloring prediction result is more accurate.

[0093] In the embodiment of the present application, the coloring loss value is used to describe the error between the coloring prediction result of the second cell image and its corresponding cell image, such as: color brightness error, color distribution error, color edge error, etc.

[0094] Step 305 : Training the cell nucleus recognition model based on the segmentation loss value and the coloring loss value to obtain a target recognition model.

[0095] Among them, the target recognition model is used to identify cell nuclei in cell images.

[0096] In some embodiments, the weighted result of the segmentation loss value and the coloring loss value is determined to obtain a target loss value, that is, the product of the segmentation loss value and the first weight parameter is determined to obtain a first weighted part; the product of the coloring loss value and the second weight parameter is determined to obtain a second weighted part; the sum of the first weighted part and the second weighted part is determined as the target loss value, wherein the first weight parameter and the second weight parameter are preset parameters; and the model parameters of the cell nucleus recognition model are adjusted based on the target loss value.

[0097] Optionally, the target loss value can be calculated by referring to Formula 3:

[0098] Formula 3: L = α·L ce +β·L MSE

[0099] Among them, L is the target loss value, α and β are the first weight parameter corresponding to the segmentation loss value and the second weight parameter corresponding to the coloring loss value, respectively. When the weight parameter value is higher, it indicates that the weight of the loss value is greater, and the impact on the target loss value is greater. It can be seen from Formula 3 that the cell nucleus recognition model is affected by both the segmentation loss value and the coloring loss value.

[0100] The model parameters of the cell nucleus recognition model are adjusted by the target loss value. Optionally, the model parameters corresponding to the coloring prediction results are adjusted; or, the model parameters corresponding to the segmentation prediction results are adjusted; or, the model parameters corresponding to the coloring prediction results and the segmentation prediction results are adjusted, which is not limited here.

[0101] In some embodiments, the target recognition model is used to recognize cell nuclei in a cell image, and the recognition content includes at least one of the following:

[0102] 1. Identify the cell nucleus in the cell image, that is, identify the image area corresponding to the cell nucleus in the cell image;

[0103] 2. Identify the boundaries of the cell nucleus in the cell image, that is, obtain the coordinate values ​​of the pixel points corresponding to the cell nucleus in the cell image, and connect the coordinates of the pixel points corresponding to the cell nucleus to outline the range of the cell nucleus;

[0104] 3. Identify the size of the cell nucleus in the cell image, that is, obtain the size and area of ​​the cell nucleus in the cell image.

[0105] It is worth noting that the above-mentioned target recognition model is only an illustrative example of the recognition content of the cell nucleus in the cell image, and the specific content of the cell nucleus recognition is not limited in any way in the embodiments of the present application.

[0106] In summary, an embodiment of the present application provides a cell image recognition method, which inputs the first cell image and the second cell image in the sample data set into a cell nucleus recognition model for segmentation prediction training and coloring prediction training respectively, obtains the segmentation prediction result corresponding to the first cell image with a label and the coloring prediction result corresponding to the second cell image without a label, calculates the corresponding segmentation loss value and coloring loss value, and trains the cell nucleus recognition model based on the segmentation loss value and the coloring loss value to improve the model accuracy of the cell nucleus recognition model, thereby improving the accuracy and recognition efficiency of cell nucleus recognition.

[0107] In an optional embodiment, the cell nucleus recognition model includes a segmentation sub-model and a coloring sub-model, and also includes an encoder. When the sample data set is input into the cell nucleus recognition model, the segmentation prediction result is obtained by the segmentation sub-model, and the coloring prediction result is obtained by the coloring sub-model. For example, please refer to Figure 4 , which shows a flow chart of a cell image recognition method provided by an exemplary embodiment of the present application, the method comprising the following steps:

[0108] Step 401: Obtain a sample data set.

[0109] The sample data set includes a first cell image and a second cell image, and the first cell image is annotated with cell nucleus information.

[0110] In some embodiments, the cell nucleus information is obtained by medical personnel based on medical knowledge and diagnostic experience after observing the cell image. After the medical personnel annotate the cell nucleus information on the cell image, they send the cell nucleus information to the server, which obtains the cell nucleus information.

[0111] Step 402 : Downsample the cell images in the sample dataset through an encoder to obtain intermediate features of the cell images.

[0112] In some embodiments, the cell nucleus recognition model is a known network model, such as a U-net network (Convolutional Networks for Biomedical Image Segmentation), an FCN network (Fully Convolutional Networks for Semantic Segmentation), a DenseNet network (Densely Connected Convolutional Networks), etc. The cell nucleus recognition model includes at least one two-dimensional network model, or at least one three-dimensional network model, which is not limited here. In the embodiment of the present application, the U-net network is used to train the cell nucleus recognition model.

[0113] U-net network is a U-shaped structure network, uses fully convolutional network to carry out semantic segmentation, in the embodiment of the present application, there is encoder in the cell nucleus recognition model based on U-net network model, encoder is used for carrying out down sampling to the cell image in sample data set, optionally, encoder is used for carrying out at least two layers of down sampling to cell image.In the present embodiment, in the process of down sampling, four layers of down sampling are passed through, after each down sampling process is completed, the feature channel number of the cell image of input will increase, so that the feature image size corresponding to the cell image becomes smaller, for carrying out feature extraction to the cell image of input, as each down sampling gradually deepens, from shallow to deep, feature extraction is carried out to the cell image of input, the feature extracted includes cell nucleus distribution feature, cell nucleus edge contour feature, cell nucleus size area etc., are not limited here.In the down sampling process, the shallow structure (such as first layer down sampling) in down sampling structure can carry out the extraction of some simple features to the cell image of input, such as the boundary feature of cell nucleus, the color feature of cell nucleus etc., the deep structure (such as fourth layer down sampling) in down sampling structure can carry out the extraction of some abstract features to the cell image of input, finally obtain the corresponding intermediate feature of the cell image of input.

[0114] Steps 4031 to 4032 are the training process of the coloring sub-model and the segmentation sub-model, and the two are in a parallel relationship. That is, after the sample data set is input into the cell nucleus recognition model, the coloring sub-model and the segmentation sub-model in the cell nucleus recognition model simultaneously perform coloring and segmentation processing on the cell image to obtain the coloring prediction results and segmentation prediction results corresponding to the sample data set.

[0115] Step 4031: Input the sample data set into the cell nucleus recognition model, perform coloring on the sample data set through the coloring sub-model, and obtain a coloring prediction result of the sample data set.

[0116] Optionally, the coloring sub-model performs coloring processing on the cell images in the sample dataset, including at least one of the following processing methods:

[0117] 1. Colorize the cell nuclei in the input cell image, that is, the color prediction results of the output sample dataset include the coloring results of the cell nucleus area;

[0118] 2. Colorize the area other than the cell nucleus in the input cell image, that is, the color prediction results of the output sample data set include the coloring results of the non-cell nucleus area;

[0119] 3. Perform different coloring processing on the cell nucleus region and non-cell nucleus region in the input cell image, that is, the coloring prediction result of the output sample data set includes the coloring results of different regions of the cell image.

[0120] It is worth noting that the above-mentioned coloring processing method is only an illustrative example, and the specific coloring processing method in the embodiments of the present application is not limited in any way.

[0121] In some embodiments, the intermediate features are upsampled in color channels by a shading decoder in a shading sub-model to obtain shading features; and shading prediction is performed on the shading features to obtain shading prediction results of the sample data set.

[0122] In an embodiment of the present application, the coloring sub-model includes a coloring decoder, which is used to decode the intermediate features corresponding to the cell image output by the encoder, that is, the coloring sub-model performs a coloring process on the edge of the cell nucleus on the intermediate features corresponding to the input cell image (the cell image input to the cell nucleus recognition model is a grayscale image) to generate a color image corresponding to the color channel, wherein the coloring decoder downsamples the intermediate features four times, and the purpose of each downsampling is to restore the feature vector obtained by the previous downsampling process, and finally obtains a coloring feature map of the same size as the input cell image through four layers of downsampling. Based on the coloring feature map, a coloring prediction result is obtained. Schematically, the coloring prediction result is a feature map y of three channels (i.e., color channels) c∈Y c =[0,1] N×N×3 , where y c is the feature map, Y c is the coloring prediction result, and N is the channel.

[0123] Among them, the three-channel feature map is a feature map used to describe the edge information of the cell nucleus in the input cell image (that is, the cell nucleus area is the colored part); or, it is a feature map used to describe the non-cell nucleus area in the input cell image (that is, the non-cell nucleus area is the colored part); or, it is a feature map of the distribution area of ​​each tissue in the input cell image (that is, different tissues in the cell image are marked with different colors), which is not limited here.

[0124] Step 4032: Segment the sample data set using the segmentation sub-model to obtain a segmentation prediction result of the sample data set.

[0125] Schematically, the segmentation sub-model performs segmentation processing on the input cell image, and its processing method includes at least one of the following methods:

[0126] 1. The segmentation sub-model segments the cell nucleus region in the input cell image, that is, it segments the cell nucleus based on the edge of the cell nucleus to make the boundary of the cell nucleus clear and distinct;

[0127] 2. The segmentation sub-model segments the non-nuclear region in the input cell image, that is, based on the distribution of the non-nuclear region in the cell image, the non-nuclear region is segmented.

[0128] It is worth noting that the above description of the segmentation processing method is only an illustrative example, and the embodiments of the present application do not limit the specific processing method of the segmentation processing.

[0129] In some embodiments, the intermediate features are upsampled on the cell nucleus recognition channel by the segmentation decoder in the segmentation sub-model to obtain cell nucleus edge features; segmentation prediction is performed on the cell nucleus edge features to obtain a segmentation prediction result of the sample data set.

[0130] In an embodiment of the present application, the segmentation sub-model includes a segmentation decoder, which is used to decode the intermediate features output by the encoder, and finally obtain the segmentation prediction result corresponding to the input cell image, that is, the segmentation decoder performs an upsampling process on the intermediate features corresponding to the input cell image, wherein the segmentation decoder includes four layers of upsampling. It is worth noting that each layer of upsampling in the segmentation decoder corresponds to each layer of downsampling in the encoder. Therefore, in each layer of upsampling, the feature vector in the segmentation decoder is concatenated with the feature vector of the same size in the corresponding encoder at the same layer to obtain the feature vector for the next layer of upsampling. After the four-layer upsampling process, a cell nucleus edge feature map of the same size as the input cell image is obtained. Based on the segmentation feature map, the segmentation prediction result of the sample data set is obtained. Among them. The segmentation prediction result is a two-channel probability map y s ∈Y s =[0,1] N×N×？ (i.e., the probability result describing whether the region is a cell nucleus, with the cell nucleus region = 1 and the non-cell nucleus region = 0 as an example), where y s is the probability graph, Y s is the segmentation prediction result, and N is the channel.

[0131] Schematically, the two-channel probability map may be used to describe the probability that the input cell image is a cell nucleus region (i.e., the region is a cell nucleus region or the region does not belong to the cell nucleus region); or, it may also be used to describe the probability that the input cell image is a non-cell nucleus region, which is not limited here.

[0132] Step 4041: Obtain the segmentation prediction result of the first cell image from the segmentation prediction results of the sample data set.

[0133] The segmentation prediction result of the sample data set includes the segmentation prediction result of the first cell image and the segmentation prediction result of the second cell image, and the segmentation prediction result of the first cell image is obtained.

[0134] Step 4042: Obtain the coloring prediction result of the second cell image from the coloring prediction results of the sample data set.

[0135] The coloring prediction result of the sample data set includes the coloring prediction result of the first cell image and the segmentation prediction result of the second cell image, and the coloring prediction result of the second cell image is obtained.

[0136] Step 405 : Obtain a segmentation loss value based on the cell nucleus information of the first cell image and the segmentation prediction result.

[0137] The description of the segmentation loss value in step 405 has been discussed in detail in the above step 303 and will not be repeated here.

[0138] Step 406: Obtain a coloring loss value based on the second cell image and the coloring prediction result.

[0139] The description of the shading loss value in step 406 has been discussed in detail in the above step 304 and will not be repeated here.

[0140] Step 407 : Training the cell nucleus recognition model based on the segmentation loss value and the coloring loss value to obtain a target recognition model.

[0141] Among them, the target recognition model is used to identify cell nuclei in cell images.

[0142] In some embodiments, the segmentation loss value is used to train the segmentation sub-model, including adjusting the model parameters in the segmentation sub-model to improve the accuracy of the segmentation prediction results corresponding to the segmentation sub-model; the coloring loss value is not only used to train the coloring sub-model, but also to train the encoder. When the coloring loss value is used to train the coloring sub-model, the model parameters in the coloring sub-model can be adjusted to improve the accuracy of the coloring prediction results; by training the encoder with the coloring loss value, the accuracy of the encoder in extracting features from cell images during the downsampling process can be improved, that is, the feature extraction in the encoder is trained to improve the accuracy of the feature extraction in the encoder, that is, the encoder can extract more accurate image features from the input cell image during the downsampling process.

[0143] In summary, an embodiment of the present application provides a cell image recognition method, which inputs the first cell image and the second cell image in the sample data set into a cell nucleus recognition model for segmentation prediction training and coloring prediction training respectively, obtains the segmentation prediction result corresponding to the first cell image with a label and the coloring prediction result corresponding to the second cell image without a label, calculates the corresponding segmentation loss value and coloring loss value, and trains the cell nucleus recognition model based on the segmentation loss value and the coloring loss value to improve the model accuracy of the cell nucleus recognition model, thereby improving the accuracy and recognition efficiency of cell nucleus recognition.

[0144] In an optional embodiment, the sample data set input to the cell nucleus recognition model is a grayscale image of a cell image, so the cell image needs to be grayscale processed. Please refer to Figure 5 , which shows a flow chart of a cell image recognition method provided by an exemplary embodiment of the present application, the method comprising the following steps:

[0145] Step 501: extract target color features of cell images in a sample data set.

[0146] In some optional embodiments, the process of extracting the target color feature is to perform color channel decomposition on the cell image to obtain at least two color channels; and determine the target color channel corresponding to the target color feature among the at least two color channels.

[0147] In an embodiment of the present application, the cell images in the sample data set are HE-stained pathological tissue images, wherein the alkaline dye hematoxylin dyes the chromatin in the cell nucleus and the nuclei in the cytoplasm into purple-blue, and the acidic dye eosin dyes the components in the cytoplasm and extracellular matrix into red. That is, in the HE-stained image, the cell nucleus area is purple-blue and the non-cell nucleus area is red, and the color features of the hematoxylin components in the HE-stained image are extracted as the target color features.

[0148] Step 502 : Based on the target color feature in the cell image, grayscale processing is performed on the cell image to obtain a grayscale image.

[0149] In some optional embodiments, the grayscale processing process includes retaining a target color channel corresponding to the target color feature based on the target color feature in the cell image; and performing grayscale processing on the cell image based on the target color channel to obtain the grayscale image.

[0150] In the embodiment of the present application, the HE-stained pathological tissue image is a color image, that is, there are color channels (RGB channels). The color separation technology is used to map the color space corresponding to the color channel to the hematoxylin component space. Optionally, refer to Formula 4:

[0151] Formula 4:

[0152] The pseudo-inverse matrix described in Formula 4 is used in the staining separation process, where the RGB value of hematoxylin corresponding to the hematoxylin component is [0.644, 0.717, 0.267], and HE i，j is the hematoxylin and eosin channel value of pixel (i, j), RGB i，j are the red, green, and blue channel values ​​of pixel (i, j), and + represents the pseudo-inverse of the matrix.

[0153] That is, there are two color features in the HE stain: the color feature of the hematoxylin component and the color feature of the eosin component. After staining and separation, two corresponding color channels are obtained: the H channel (hematoxylin component channel) and the E channel (eosin component channel), which represent the staining intensity of the hematoxylin component and the eosin component, respectively. The color feature of the hematoxylin component in the cell image is retained, so the cell image corresponding to the H channel is obtained. In the cell image corresponding to the H channel, the color feature of the cell nucleus intersects with the corresponding cell nucleus color in the original cell image, and the contrast is enhanced, while the color feature of the non-nuclear region is reduced compared to the color feature of the non-nuclear region in the original cell image.

[0154] Grayscale processing is performed on the cell image corresponding to the H channel, wherein the grayscale processing includes at least one of the following methods:

[0155] 1. Use the component method to perform grayscale processing. That is, the brightness of the red, green, and blue components in the cell image corresponding to the H channel (in this case, a color image) is used as the grayscale values ​​of the three grayscale images. Select any one of the grayscale values ​​and obtain the grayscale image based on the grayscale value.

[0156] 2. Use the maximum value method for grayscale processing. That is, the maximum value of the brightness of the red, green, and blue components in the cell image corresponding to the H channel (in this case, a color image) is used as the grayscale value of the grayscale image, and the grayscale image is obtained based on this grayscale value;

[0157] 3. Use the average value method for grayscale processing. That is, the average value of the brightness of the red, green, and blue components in the cell image corresponding to the H channel (in this case, a color image) is used as the grayscale value of the grayscale image, and the grayscale image is obtained based on this grayscale value;

[0158] 4. Use the weighted average method for grayscale processing. According to the importance of the red, green, and blue components in the cell image corresponding to the H channel (this is a color image) and other indicators, perform weighted averaging on them with different weight values ​​to obtain the grayscale value after weighted average, and obtain the grayscale image based on the grayscale value.

[0159] It is worth noting that the above description of the grayscale processing method is only an illustrative example, and the specific method of grayscale processing is not limited in the embodiments of the present application.

[0160] For illustration, please refer to Figure 6 , which shows a schematic diagram of obtaining a grayscale image provided by an embodiment of the present application, such as Figure 6 As shown, the HE-stained cell image 601 is extracted by staining separation technology and grayscale processing is performed to obtain a grayscale image 602 of the HE-stained cell image corresponding to the hematoxylin component. Figure 6 It can be seen that the contrast of the cell nucleus region in the grayscale image with hematoxylin component extracted is more obvious, that is, the boundary of the cell nucleus is clearer.

[0161] Step 503: Acquire a sample data set, where the sample data set includes a first cell image and a second cell image.

[0162] The first cell image is annotated with cell nucleus information.

[0163] The description of the sample data set in step 503 is described in detail in the above step 301 and will not be repeated here.

[0164] Step 504 : Input the grayscale image into the cell nucleus recognition model, and output the segmentation prediction result of the first cell image and the coloring prediction result of the second cell image.

[0165] The description of the segmentation prediction result and the coloring prediction result in step 504 is described in detail in the above step 302 and will not be repeated here.

[0166] Step 505 : Obtain a segmentation loss value based on the cell nucleus information of the first cell image and the segmentation prediction result.

[0167] The description of the segmentation loss value in step 505 has been described in detail in the above step 303 and will not be repeated here.

[0168] Step 506: Obtain a coloring loss value based on the second cell image and the coloring prediction result.

[0169] The description of the shading loss value in step 506 has been described in detail in the above step 304 and will not be repeated here.

[0170] Step 507 : Training the cell nucleus recognition model based on the segmentation loss value and the coloring loss value to obtain a target recognition model.

[0171] Among them, the target recognition model is used to identify cell nuclei in cell images.

[0172] The description of the target recognition model in step 507 has been described in detail in the above step 305 and will not be repeated here.

[0173] In summary, an embodiment of the present application provides a cell image recognition method, which inputs the first cell image and the second cell image in the sample data set into a cell nucleus recognition model for segmentation prediction training and coloring prediction training respectively, obtains the segmentation prediction result corresponding to the first cell image with a label and the coloring prediction result corresponding to the second cell image without a label, calculates the corresponding segmentation loss value and coloring loss value, and trains the cell nucleus recognition model based on the segmentation loss value and coloring loss value to improve the model accuracy of the cell nucleus recognition model, thereby improving the recognition rate and recognition efficiency of cell nucleus segmentation.

[0174] In an optional embodiment, if Figure 7 As shown, it shows a schematic diagram of the recognition process of a cell image provided by an exemplary embodiment of the present application, as shown in FIG. Figure 7 As shown, the sample data set 701 is input into the cell nucleus recognition model 702. The sample data set 701 includes cell nucleus information Y L =(0,1) N×N The grayscale image X corresponding to the first cell image LThe grayscale image X corresponding to the second cell image U In the cell nucleus recognition model 702, the encoder downsamples the input cell image to obtain intermediate features 703. The cell nucleus recognition model also includes a segmentation sub-model and a coloring sub-model. The segmentation branch decoder in the segmentation sub-model 704 upsamples the intermediate features 703 to obtain a segmentation prediction result 706. The coloring branch decoder in the coloring sub-model 705 upsamples the intermediate features 703 to obtain a coloring prediction result 707. The segmentation loss is obtained based on the cell nucleus information of the first cell image and the segmentation prediction result 706. The coloring loss is obtained based on the second cell image and the coloring prediction result 707. The cell nucleus recognition model 702 is trained based on the segmentation loss and the coloring loss, and finally the target recognition model is obtained.

[0175] When testing the target recognition model, the hematoxylin component is first extracted from the HE-stained cell image and grayscale processed to obtain the corresponding grayscale image. The grayscale image is input into the target recognition model, and the segmentation prediction result and coloring prediction result of the grayscale image are obtained by the target recognition model. The segmentation prediction result is adjusted based on the coloring prediction result to obtain the final cell nucleus segmentation result corresponding to the HE-stained cell image.

[0176] Optionally, the manner of adjusting the segmentation prediction result based on the coloring prediction result includes at least one of the following manners:

[0177] 1. The three-channel feature map corresponding to the coloring prediction result is used as a color mask to cover the probability map corresponding to the segmentation prediction result. The labels in the probability map are modified based on the color mask to obtain the final cell nucleus segmentation result.

[0178] 2. Compare the three-channel feature maps corresponding to the coloring prediction results with the probability maps corresponding to the segmentation prediction results. Adjust the probability maps based on the nucleus edge features in the feature maps to obtain the final mint segmentation results.

[0179] 3. Mark the labels in the probability map corresponding to the segmentation prediction result in the three-channel feature map corresponding to the coloring prediction result, and obtain the overlapping part between the label and the coloring feature as the final cell nucleus segmentation result.

[0180] It is worth noting that the above-mentioned method of adjusting the segmentation prediction result based on the coloring prediction result is only an illustrative example and is not limited to it in the implementation of this application.

[0181] This application can effectively use a small amount of cell nucleus information (such as pixel-level segmentation labels) for supervised segmentation training, and use a large amount of unlabeled data for coloring training to assist in the segmentation task, thereby achieving more accurate cell nucleus recognition in HE-stained tissue pathology images. The specific usage process is as follows: on the offline side, a large number of HE-stained tissue pathology images are collected, and a few of them are fully labeled with cell nuclei. Then, all tissue pathology images are extracted with hematoxylin components, and the cell nucleus segmentation and coloring models are trained. On the server side, the input tissue pathology image is first extracted with hematoxylin components to obtain a grayscale image, and then the segmentation sub-model and the coloring sub-model are called to output the corresponding cell nucleus recognition result map.

[0182] This application can also be applied to the automated analysis of histopathology images. Subsequent calculations of cell nucleus recognition results can reveal characteristics such as average size, density, and arrangement of the nuclei, enabling clinical diagnosis and treatment of cancer, including grading different types of cancer and stratifying patient risk. Furthermore, this application can also be applied to other recognition tasks where targets within the same category have similar colors, while targets within different categories have significantly different colors.

[0183] In summary, the embodiment of the present application provides a cell image recognition method, which inputs the first cell image and the second cell image in the sample data set into the cell nucleus recognition model for segmentation prediction training and coloring prediction training respectively, obtains the segmentation prediction result corresponding to the first cell image with a label and the coloring prediction result corresponding to the second cell image without a label, calculates the corresponding segmentation loss value and coloring loss value, and trains the cell nucleus recognition model based on the segmentation loss value and coloring loss value, thereby improving the model accuracy of the cell nucleus recognition model and reducing the annotation burden of pathologists.

[0184] Figure 8 FIG. 1 is a schematic diagram of a cell image recognition device provided by an exemplary embodiment of the present application. Figure 8 As shown, the device includes:

[0185] A sample acquisition module 810 is configured to acquire a sample data set, wherein the sample data set includes a first cell image and a second cell image, wherein the first cell image is annotated with cell nucleus information;

[0186] An output module 820 is configured to input the sample data set into a cell nucleus recognition model and output a segmentation prediction result of the first cell image and a coloring prediction result of the second cell image;

[0187] a loss acquisition module 830, configured to obtain a segmentation loss value based on the cell nucleus information of the first cell image and the segmentation prediction result;

[0188] The loss acquisition module 830 is further configured to obtain a coloring loss value based on the second cell image and the coloring prediction result;

[0189] The training module 840 is used to train the cell nucleus recognition model based on the segmentation loss value and the coloring loss value to obtain a target recognition model, and the target recognition model is used to recognize cell nuclei in cell images.

[0190] In an optional embodiment, the training module 840 includes:

[0191] A determining unit 841 is configured to determine a weighted result of the segmentation loss value and the coloring loss value to obtain a target loss value;

[0192] The adjusting unit 842 is configured to adjust the model parameters of the cell nucleus recognition model based on the target loss value.

[0193] In an optional embodiment, the determination unit 841 is further used to determine the product of the segmentation loss value and a first weight parameter to obtain a first weighted part; determine the product of the shading loss value and a second weight parameter to obtain a second weighted part; and determine the sum of the first weighted part and the second weighted part as the target loss value, wherein the first weight parameter and the second weight parameter are preset parameters.

[0194] In an optional embodiment, the cell nucleus recognition model includes a segmentation sub-model and a coloring sub-model;

[0195] The output module 820 includes:

[0196] The coloring unit 821 is configured to input the sample data set into the cell nucleus recognition model, perform coloring processing on the sample data set through the coloring sub-model, and obtain a coloring prediction result of the sample data set.

[0197] The segmentation unit 822 is configured to perform segmentation processing on the sample data set using the segmentation sub-model to obtain a segmentation prediction result of the sample data set.

[0198] The acquiring unit 823 is configured to acquire the segmentation prediction result of the first cell image from the segmentation prediction result of the sample data set, and acquire the coloring prediction result of the second cell image from the coloring prediction result of the sample data set.

[0199] In an optional embodiment, the cell nucleus recognition model further includes an encoder;

[0200] The shading unit 821 is further configured to downsample the cell images in the sample dataset through the encoder to obtain intermediate features of the cell images.

[0201] The shading unit 821 is further configured to perform color channel upsampling on the intermediate features through the shading decoder in the shading sub-model to obtain shading features; and perform shading prediction on the shading features to obtain shading prediction results of the sample data set.

[0202] In an optional embodiment, the segmentation unit 822 is further used to perform cell nucleus recognition channel upsampling on the intermediate features through the segmentation decoder in the segmentation sub-model to obtain cell nucleus edge features; and perform segmentation prediction on the cell nucleus edge features to obtain a segmentation prediction result of the sample data set.

[0203] In an optional embodiment, the device further comprises:

[0204] Extraction module 801, used to extract target color features of cell images in the sample data set;

[0205] A processing module 802 is configured to perform grayscale processing on the cell image based on target color features in the cell image to obtain a grayscale image;

[0206] The output module 820 is further configured to input the grayscale image into the cell nucleus recognition model.

[0207] In an optional embodiment, the extraction module 801 is further configured to perform color channel decomposition on the cell image to obtain at least two color channels; and determine a target color channel corresponding to the target color feature among the at least two color channels.

[0208] The extraction module 801 is further configured to retain a target color channel corresponding to the target color feature in the cell image based on the target color feature; and perform grayscale processing on the cell image based on the target color channel to obtain the grayscale image.

[0209] In an optional embodiment, the loss acquisition module 830 is further configured to obtain a segmentation loss value of the cell nucleus information of the first cell image and the segmentation prediction result based on a cross entropy loss function.

[0210] The loss acquisition module 830 is further configured to obtain the coloring loss value based on the squared residual between the second cell image and the coloring prediction result.

[0211] In summary, the cell image recognition device provided in this embodiment inputs the first cell image and the second cell image in the sample data set into the cell nucleus recognition model for segmentation prediction training and coloring prediction training respectively, obtains the segmentation prediction result corresponding to the first cell image with a label and the coloring prediction result corresponding to the second cell image without a label, calculates the corresponding segmentation loss value and coloring loss value, trains the cell nucleus recognition model based on the segmentation loss value and the coloring loss value, improves the model accuracy of the cell nucleus recognition model, and thus improves the accuracy and recognition efficiency of cell nucleus recognition.

[0212] It should be noted that the cell image recognition device provided in the above embodiment is merely exemplified by the division of the aforementioned functional modules. In actual applications, the aforementioned functions can be assigned to different functional modules as needed, i.e., the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. Furthermore, the cell image recognition device provided in the above embodiment and the cell image recognition method embodiment are based on the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0213] Figure 10 The following is a schematic diagram showing the structure of a server provided by an exemplary embodiment of the present application. Specifically:

[0214] The server 1000 includes a central processing unit (CPU) 1001, a system memory 1004 including a random access memory (RAM) 1002 and a read-only memory (ROM) 1003, and a system bus 1005 connecting the system memory 1004 and the CPU 1001. The server 1000 also includes a mass storage device 1006 for storing an operating system 1013, application programs 1014, and other program modules 1015.

[0215] The mass storage device 1006 is connected to the central processing unit 1001 via a mass storage controller (not shown) connected to the system bus 1005. The mass storage device 1006 and its associated computer-readable media provide non-volatile storage for the server 1000. That is, the mass storage device 1006 may include a computer-readable medium (not shown) such as a hard disk or a compact disc read-only memory (CD-ROM) drive.

[0216] Without loss of generality, computer-readable media may include computer storage media and communication media. Computer storage media include volatile and non-volatile, removable and non-removable media implemented by any method or technology for storing information such as computer-readable instructions, data structures, program modules or other data. Computer storage media include RAM, ROM, Erasable Programmable Read Only Memory (EPROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other solid-state storage technologies, CD-ROM, Digital Versatile Disc (DVD) or other optical storage, tape cassettes, magnetic tape, disk storage or other magnetic storage devices. Of course, those skilled in the art will appreciate that computer storage media are not limited to the above-mentioned ones. The above-mentioned system memory 1004 and mass storage device 1006 can be collectively referred to as memory.

[0217] According to various embodiments of the present application, the server 1000 may also be connected to a remote computer on a network such as the Internet for operation. That is, the server 1000 may be connected to the network 1012 via the network interface unit 1011 connected to the system bus 1005, or the network interface unit 1011 may be used to connect to other types of networks or remote computer systems (not shown).

[0218] The memory also includes one or more programs, which are stored in the memory and configured to be executed by the CPU.

[0219] An embodiment of the present application also provides a computer device, which includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, code set or instruction set, and the at least one instruction, at least one program, code set or instruction set is loaded and executed by the processor to implement the cell image recognition method provided by the above-mentioned method embodiments.

[0220] An embodiment of the present application also provides a computer-readable storage medium, which stores at least one instruction, at least one program, code set or instruction set, and the at least one instruction, at least one program, code set or instruction set is loaded and executed by a processor to implement the cell image recognition method provided by the above-mentioned method embodiments.

[0221] Embodiments of the present application further provide a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the cell image recognition method described in any of the above embodiments.

[0222] Optionally, the computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a solid-state drive (SSD), or an optical disk. Among them, the random access memory may include a resistance random access memory (ReRAM) and a dynamic random access memory (DRAM). The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0223] Those skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or by a program to instruct the relevant hardware, and the program may be stored in a computer-readable storage medium, which may be a read-only memory, a disk, or an optical disk, etc.

[0224] The above description is merely an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A cell image recognition method, characterized in that: The method comprises: Acquire a sample data set, wherein the sample data set includes a first cell image and a second cell image, wherein the first cell image is annotated with cell nucleus information; Inputting the sample data set into a cell nucleus recognition model, and outputting a segmentation prediction result of the first cell image and a coloring prediction result of the second cell image; Obtaining a segmentation loss value based on the cell nucleus information of the first cell image and the segmentation prediction result; Obtaining a coloring loss value based on the second cell image and the coloring prediction result; The cell nucleus recognition model is trained based on the segmentation loss value and the coloring loss value to obtain a target recognition model, and the target recognition model is used to recognize cell nuclei in cell images.

2. The method according to claim 1, characterized in that The training of the cell nucleus recognition model based on the segmentation loss value and the coloring loss value includes: Determine a weighted result of the segmentation loss value and the coloring loss value to obtain a target loss value; The model parameters of the cell nucleus recognition model are adjusted based on the target loss value.

3. The method according to claim 2, characterized in that Determining a weighted result of the segmentation loss value and the coloring loss value to obtain a target loss value includes: Determine the product of the segmentation loss value and a first weight parameter to obtain a first weighted part; Determining the product of the shading loss value and a second weight parameter to obtain a second weighted part; The sum of the first weighted part and the second weighted part is determined as the target loss value, wherein the first weight parameter and the second weight parameter are preset parameters.

4. The method according to any one of claims 1 to 3, characterized in that: The cell nucleus recognition model includes a segmentation sub-model and a coloring sub-model; The step of inputting the sample data set into a cell nucleus recognition model and outputting a segmentation prediction result of the first cell image and a coloring prediction result of the second cell image includes: Inputting the sample data set into the cell nucleus recognition model, performing coloring processing on the sample data set using the coloring sub-model, and obtaining a coloring prediction result of the sample data set; Performing segmentation processing on the sample data set by using the segmentation sub-model to obtain a segmentation prediction result of the sample data set; The segmentation prediction result of the first cell image is obtained from the segmentation prediction result of the sample data set, and the coloring prediction result of the second cell image is obtained from the coloring prediction result of the sample data set.

5. The method according to claim 4, characterized in that The cell nucleus recognition model also includes an encoder; Before coloring the sample data set using the coloring sub-model to obtain a coloring prediction result of the sample data set, the method further includes: The encoder downsamples the cell images in the sample dataset to obtain intermediate features of the cell images.

6. The method according to claim 5, characterized in that The coloring process is performed on the sample data set by using the coloring sub-model to obtain a coloring prediction result of the sample data set, including: Performing color channel upsampling on the intermediate features by a shading decoder in the shading sub-model to obtain shading features; Perform coloring prediction on the coloring feature to obtain a coloring prediction result of the sample data set.

7. The method according to claim 5, characterized in that The segmentation processing of the sample data set by the segmentation sub-model to obtain the segmentation prediction result of the sample data set includes: Performing cell nucleus recognition channel upsampling on the intermediate features by the segmentation decoder in the segmentation sub-model to obtain cell nucleus edge features; Segmentation prediction is performed on the cell nucleus edge features to obtain a segmentation prediction result of the sample data set.

8. The method according to any one of claims 1 to 3, characterized in that: Before inputting the sample data set into the cell nucleus recognition model, the method further includes: Extracting target color features of cell images in the sample data set; Based on the target color feature in the cell image, performing grayscale processing on the cell image to obtain a grayscale image; The step of inputting the sample data set into a cell nucleus recognition model comprises: The grayscale image is input into the cell nucleus recognition model.

9. The method according to claim 8, characterized in that The extracting target color features of the cell images in the sample data set includes: performing color channel decomposition on the cell image to obtain at least two color channels; A target color channel corresponding to the target color feature is determined among the at least two color channels.

10. The method according to claim 9, characterized in that Based on the target color feature in the cell image, grayscale processing is performed on the cell image to obtain a grayscale image, comprising: Based on the target color feature in the cell image, retaining the target color channel corresponding to the target color feature; Based on the target color channel, grayscale processing is performed on the cell image to obtain the grayscale image.

11. The method according to any one of claims 1 to 3, characterized in that: The obtaining of a segmentation loss value based on the cell nucleus information of the first cell image and the segmentation prediction result includes: A segmentation loss value of the cell nucleus information of the first cell image and the segmentation prediction result is obtained based on a cross entropy loss function.

12. The method according to any one of claims 1 to 3, characterized in that: The obtaining of a coloring loss value based on the second cell image and the coloring prediction result includes: The coloring loss value is obtained based on the residual square of the second cell image and the coloring prediction result.

13. A cell image recognition device, characterized in that: The device comprises: A sample acquisition module, configured to acquire a sample data set, wherein the sample data set includes a first cell image and a second cell image, wherein the first cell image is annotated with cell nucleus information; an output module, configured to input the sample data set into a cell nucleus recognition model, and output a segmentation prediction result of the first cell image and a coloring prediction result of the second cell image; a loss acquisition module, configured to obtain a segmentation loss value based on the cell nucleus information of the first cell image and the segmentation prediction result; The loss acquisition module is further configured to obtain a coloring loss value based on the second cell image and the coloring prediction result; A training module is used to train the cell nucleus recognition model based on the segmentation loss value and the coloring loss value to obtain a target recognition model, and the target recognition model is used to recognize cell nuclei in cell images.

14. A computer device, characterized in that: The computer device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, code set or instruction set, and the at least one instruction, at least one program, code set or instruction set is loaded and executed by the processor to implement the cell image recognition method as described in any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that The storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the cell image recognition method as described in any one of claims 1 to 12.

16. A computer program product, characterized in that The method comprises a computer program or an instruction, which, when executed by a processor, implements the cell image recognition method according to any one of claims 1 to 12.

Images