Training method for ascites cell classification model and system for breast cancer prediction

Abstract

Training method for ascites cell classification model and system for breast cancer prediction. The system for breast cancer prediction comprises: an image acquisition component configured to acquire a sample image set of microparticles in an ascites sample liquid in response to the ascites sample liquid flowing through a flow tube of a flow cytometric microimaging device; a classification component configured to receive the sample image set as an input sample image set and obtain a classification result of each input sample image in the input sample image set based on a trained ascites cell classification model; a classification result statistics component configured to statistically analyze the classification results of all input sample images in the input sample image set to determine whether the ascites sample liquid is breast cancer positive; and a diagnosis component configured to make a prediction of breast cancer based on the determination of whether the ascites sample liquid is breast cancer positive.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence, and in particular to a training method for a pleural effusion and ascites cell classification model without manual annotation and a system for breast cancer prediction. BACKGROUND

[0002] According to the survey data of the International Agency for Research on Cancer, one fourth of the global female cancer incidence is breast cancer. And the incidence of breast cancer in China is increasing year by year. Although new treatment strategies and methods have emerged, the mortality rate of breast cancer has gradually decreased, but the screening method of breast cancer is still not very accurate and depends heavily on the experience of doctors, which requires expensive medical resources of the pathology department.

[0003] Currently, several methods have been proposed for breast cancer diagnosis. Clinical breast examination (CBE) is simple, easy to perform, and highly reproducible. According to current research, the proportion of early breast cancer discovered by CBE alone is still low, but it provides an opportunity for women to be vigilant about the occurrence of breast cancer, such as information on risk factors, genetic problems, and new diagnostic methods, thereby receiving indirect effects. However, its sensitivity is low and it is greatly affected by subjective factors. MAM is currently the preferred imaging method for diagnosing breast cancer, which can clearly show microcalcification. Microcalcification is another important sign of breast cancer in addition to mass and structural distortion, and sometimes it is the only sign. X-ray films of 30% to 50% of early breast cancer cases show fine granular calcification clusters. Calcification clusters are closely related to cancer lesions. Although MAM screening can reduce breast cancer mortality, it also has certain limitations, especially its inability to detect all breast cancer cases, and the breast cancer it detects may still have poor prognosis. Breast MRI technology has excellent soft tissue resolution and no radiation, and has unique advantages for breast examination. However, due to the lack of well-designed clinical trials, the long-term clinical effects of preoperative breast MRI are not very clear. There is no exact data to support that breast MRI can reduce the number of breast cancer surgeries or change the surgical approach, and its postoperative survival and tumor recurrence are also unknown. Digital breast tomosynthesis (DBT) is a high-level application based on flat panel detector technology, which is a new type of tomographic imaging technology developed on the basis of traditional tomographic principles combined with digital image processing technology. Its characteristics are small radiation dose, can obtain any layer image, can further process and display three-dimensional information, has higher sensitivity and specificity, and can reduce the review rate. However, the clinical experience of DBT is still insufficient. Computer-aided detection (CAD) uses computer-related software to denoise and enhance features of digital images, thereby extracting features with diagnostic value. Physicians can improve the detection rate of breast cancer by reading images combined with CAD, especially for lesions less than 1 cm in diameter. However, the false positive rate of CAD is high.

[0004] Cell recognition and classification has always been the focus of research in the fields of medical detection and judgment, drug development and research, and environmental detection. Flow cytometry has a high detection speed, which can reach thousands of cells per second, but the information dimension obtained by flow cytometry is small, and it is impossible to classify cells through high-dimensional information such as cell volume and shape, so the precision is not high, and flow cytometry can only be used for quantitative analysis and is often used for cell screening. Microscopy can observe cell images, but it needs to go through processes such as staining and film making, which is time-consuming, and the observation of cells generally relies on human recognition, which is extremely labor-intensive. Therefore, the observation method of staining and film making is generally used as the final observation method for distinguishing cells, and the detection sample is small, which cannot be used for large-scale detection.

[0005] With the development of deep learning technologies represented by convolutional neural networks in recent years, more and more deep learning methods have been used to assist medical examination and pathological diagnosis, but most of these methods need to rely on a large labeled database, and the annotation of the training set requires expensive costs.

[0006] Therefore, it is desirable to provide an improved method that can obtain cell images and accurately provide statistical data of cell populations while saving annotation costs. SUMMARY

[0007] The present disclosure is made in view of the above problems. One object of the present disclosure is to provide a training method for a pleural effusion and ascites cell classification model and a system for breast cancer prediction without manual annotation, and corresponding devices, electronic devices and computer readable storage media.

[0008] The embodiment of the present disclosure provides a training method for a pleural effusion cell classification model, which comprises: in response to pleural effusion breast cancer full negative sample liquid including first type particles, second type particles and third type particles flowing through a flow channel of a flow cytometry microscopic imaging device, acquiring images of the particles in the full negative sample liquid to obtain a full negative sample image set including a set of images of the first type particles, a set of images of the second type particles and a set of images of the third type particles; in response to pleural effusion breast cancer full positive sample liquid including first type particles, second type particles and fourth type particles flowing through a flow channel of a flow cytometry microscopic imaging device, acquiring images of the particles in the full positive sample liquid to obtain a full positive sample image set including a set of images of the first type particles, a set of images of the second type particles and a set of images of the fourth type particles; based on morphological differences of the first type particles, the second type particles, the third type particles and the fourth type particles, automatically labeling the full negative sample image set and the full positive sample image set, thereby generating a first training data set of images of the first type particles, a second training data set of images of the second type particles, a third training data set of images of the third type particles and a fourth training data set of images of the fourth type particles; and constructing a pleural effusion cell classification model based on a neural network, and training the pleural effusion cell classification model based on the first training data set, the second training data set, the third training data set and the fourth training data set to obtain a trained pleural effusion cell classification model for classifying pleural effusion cells.

[0009] For example, according to the method of the embodiment of the present disclosure, wherein the flow cytometry microscopic imaging device comprises a light source, a condenser lens group, a flow channel, a microscopic objective lens, a tube lens, and a camera, wherein the flow cytometry microscopic imaging device acquires images of particles in the pleural effusion sample liquid comprises: the flow cytometry microscopic imaging device controls the cells of the pleural effusion sample liquid to flow stably through the center of the flow channel by controlling the flow of the pleural effusion sample liquid; the light source is configured to emit a visible light beam; the condenser lens group is configured to condense the light beam and uniformly irradiate the light beam on the flow channel; the microscopic objective lens adopts an infinite objective lens, focuses on the center of the flow channel, and images the flowing cells; the tube lens converges the outgoing light of the infinite objective lens to form an image, and adjusting the focal length of the tube lens can adjust the magnification of the image; and the camera is located at the back focal plane of the tube lens, takes the enlarged image, and obtains the image of the particles.

[0010] For example, according to the method of the embodiment of the present disclosure, wherein the first type particles are impurities, the second type particles are lymphocytes, the third type particles are mesothelial cells, and the fourth type particles are breast cancer cells.

[0011] For example, the method according to the embodiments of the present disclosure, wherein the generating the first training data set of images of the first type of microparticles, the second training data set of images of the second type of microparticles, the third training data set of images of the third type of microparticles, and the fourth training data set of images of the fourth type of microparticles includes: for each of the sample images in the set of full negative sample images and the set of full positive sample images, determining a location of a microparticle region containing microparticles in the sample image; segmenting the microparticle region from the sample image according to the determined location of the microparticle region; calculating an area and a circularity of the microparticle region; classifying the sample image based on the calculated area and circularity and whether the sample image belongs to the set of full negative sample images or the set of full positive sample images; and labeling each of the sample images in the set of full negative sample images and the set of full positive sample images based on the classification result, thereby generating the first training data set of images of the first type of microparticles, the second training data set of images of the second type of microparticles, the third training data set of images of the third type of microparticles, and the fourth training data set of images of the fourth type of microparticles.

[0012] For example, the method according to the embodiments of the present disclosure, wherein the determining the location of the microparticle region containing microparticles in the sample image includes: normalizing pixels of the sample image, comparing each pixel in the normalized sample image with a predefined first threshold value; setting a value of the pixel to 1 when the value of the pixel is greater than or equal to the first threshold value, and setting the value of the pixel to 0 when the value of the pixel is less than the first threshold value; and determining a region in the sample image where the value of the pixel is 1 as the location of the microparticle region containing microparticles in the sample image.

[0013] For example, the method according to the embodiments of the present disclosure, wherein the classifying the sample image based on the calculated area and circularity and whether the sample image belongs to the set of full negative sample images or the set of full positive sample images includes: in response to the circularity being less than a second threshold value, determining that the sample image is an image of the first type of microparticles; in response to the circularity being greater than the second threshold value and the area being less than a third threshold value, determining that the sample image is an image of the second type of microparticles; in response to the circularity being greater than the second threshold value, the area being greater than the third threshold value, and the sample image belonging to the set of full negative sample images, determining that the sample image is an image of the third type of microparticles; and in response to the circularity being greater than the second threshold value, the area being greater than the third threshold value, and the sample image belonging to the set of full positive sample images, determining that the sample image is an image of the fourth type of microparticles.

[0014] For example, according to the method of an embodiment of the present disclosure, wherein the ascites cell classification model sequentially comprises an input layer, a hidden layer and an output layer, the hidden layer comprises 16 convolutional layers and 3 fully connected layers, and training the ascites cell classification model based on the training data set comprises: training the ascites cell classification model until the loss function of the ascites cell classification model converges to obtain a trained ascites cell classification model, wherein the loss function is a cross-entropy loss function, the optimization constraint algorithm is a stochastic gradient descent method, and the learning rate is 0.00003.

[0015] An embodiment of the present disclosure provides a training device for an ascites cell classification model, the device comprising: a training data set acquisition component configured to acquire images of particles in a full negative sample liquid comprising first particles, second particles and third particles of ascites breast cancer flowing through a flow channel of a flow cytometry microscopic imaging device to obtain a full negative sample image set; acquire images of particles in a full positive sample liquid comprising first particles, second particles and fourth particles of ascites breast cancer flowing through a flow channel of a flow cytometry microscopic imaging device to obtain a full positive sample image set; a label generation component configured to automatically annotate the full negative sample image set and the full positive sample image set based on the morphological differences of the first particles, the second particles, the third particles and the fourth particles, thereby generating a first training data set of images of the first particles, a second training data set of images of the second particles, a third training data set of images of the third particles and a fourth training data set of images of the fourth particles; and a training component configured to construct an ascites cell classification model based on a neural network, and train the ascites cell classification model based on the first training data set, the second training data set, the third training data set and the fourth training data set to obtain a trained ascites cell classification model for classifying ascites cells.

[0016] Embodiments of the present disclosure provide a system for breast cancer prediction, comprising: an image acquisition component configured to acquire a sample image set of particles in a pleural effusion sample liquid in response to the pleural effusion sample liquid flowing through a flow channel of a flow cytometry microscopic imaging device including first, second, third and / or fourth type of particles; a classification component configured to receive the sample image set as an input sample image set and obtain a classification result of each input sample image in the input sample image set based on a trained pleural effusion cell classification model; a classification result statistics component configured to statistically analyze the classification results of all input sample images in the input sample image set to determine whether the pleural effusion sample liquid is breast cancer positive; and a diagnosis component configured to make a prediction of breast cancer based on the determination of whether the pleural effusion sample liquid is breast cancer positive, wherein the trained pleural effusion cell classification model is trained according to the method of any one of the above.

[0017] For example, the method according to the embodiments of the present disclosure, wherein the system for breast cancer prediction further comprises a particle screening component configured to screen the third and fourth type of particles in the sample image set based on morphological differences of the first, second, third and fourth type of particles as the input sample image set.

[0018] For example, the method according to the embodiments of the present disclosure, wherein the particle screening component is further configured to: determine a location of a particle region including a particle in a sample image of the sample image set; segment the particle region from the sample image based on the determined location of the particle region; calculate an area and a circularity of the particle region; determine that the sample image is an image of the third type of particles or an image of the fourth type of particles based on the circularity being greater than a second threshold value and the area being greater than a third threshold value; and screen the images of the third type of particles and the images of the fourth type of particles in the sample image set as the input sample image set.

[0019] For example, the method according to the embodiments of the present disclosure, wherein the particle screening component is further configured to: normalize pixels of the sample image, compare each pixel in the normalized sample image with a predefined first threshold value; set the pixel value to 1 when the value of the pixel is greater than or equal to the first threshold value, and set the pixel value to 0 when the value of the pixel is less than the first threshold value; and determine a region in the sample image in which the value of the pixel is 1 as a location of a particle region including a particle in the sample image.

[0020] For example, the method according to an embodiment of the present disclosure, wherein the first type of microparticles are impurities, the second type of microparticles are lymphocytes, the third type of microparticles are mesothelial cells, and the fourth type of microparticles are breast cancer cells.

[0021] For example, the method according to an embodiment of the present disclosure, wherein the classification component is further configured to obtain a classification result indicating a positive probability score of the input sample image being a breast cancer cell image, and wherein the classification result statistics component is further configured to: statistically obtain the positive probability score of each input sample image in the input sample image set being a breast cancer cell image, respectively plot a scatter plot of the positive probability score with respect to each input sample image, a quantity histogram of the positive probability score with respect to the number of images, or a probability density histogram of the positive probability score with respect to the probability density.

[0022] For example, the method according to an embodiment of the present disclosure, wherein the classification result statistics component is further configured to: determine that the pleural effusion sample is breast cancer positive when the scatter plot shows that the positive probability score of more than a fourth threshold number of sample images is higher than a predetermined positive probability score value; determine that the pleural effusion sample is breast cancer positive when the quantity histogram shows that the positive probability score of more than a fourth threshold number of sample images is higher than a predetermined positive probability score value; and / or determine that the pleural effusion sample is breast cancer positive when the probability density histogram shows that the positive probability score of more than a fifth threshold proportion of sample images is higher than a predetermined positive probability score value.

[0023] Embodiments of the present disclosure also provide an electronic device comprising a memory and a processor, wherein the memory has stored thereon processor-readable program code which, when executed by the processor, performs the method according to any one of the above methods.

[0024] Embodiments of the present disclosure also provide an electronic device comprising a memory and a processor, wherein the memory has stored thereon processor-readable program code which, when executed by the processor, performs the following steps: in response to a pleural effusion sample liquid comprising first type of microparticles, second type of microparticles, third type of microparticles, and / or fourth type of microparticles flowing through a flow channel of a flow cytometric micro-imaging device, obtaining a sample image set of microparticles in the pleural effusion sample liquid; receiving the sample image set as an input sample image set and obtaining a classification result of each input sample image in the input sample image set based on a trained pleural effusion cell classification model; and statistically analyzing the classification results of all input sample images in the input sample image set to determine whether the pleural effusion sample liquid is breast cancer positive, wherein the trained pleural effusion cell classification model is trained by any one of the above methods.

[0025] Embodiments of the present disclosure also provide a computer-readable storage medium having stored thereon computer-executable instructions for performing the method according to any of the above methods.

[0026] Embodiments of the present disclosure also provide a computer-readable storage medium having stored thereon computer-executable instructions for performing the following steps: acquiring a sample image set of particles in a pleural effusion sample liquid in response to a liquid flow of the pleural effusion sample including first-type particles, second-type particles, third-type particles and / or fourth-type particles flowing through a flow channel of a flow cytometry microscopic imaging device; receiving the sample image set as an input sample image set, and obtaining a classification result of each input sample image in the input sample image set based on a trained pleural effusion cell classification model; and statistically analyzing the classification results of all input sample images in the input sample image set to determine whether the pleural effusion sample liquid is breast cancer positive, wherein the trained pleural effusion cell classification model is a pleural effusion cell classification model trained according to any of the above methods.

[0027] The method of the embodiments of the present disclosure utilizes flow microscopic imaging technology to perform single-cell imaging on a pleural effusion sample, and automatically annotates a large amount of imaging data, so as to perform feature extraction and feature learning on the automatically annotated imaging data based on a neural network, to obtain a trained pleural effusion cell classification model for classifying breast cancer cells. The breast cancer prediction system of the embodiments of the present disclosure utilizes the trained pleural effusion cell classification model, and finally realizes objective and accurate judgment on whether breast cancer cells exist in a sample and the positive probability of breast cancer cells, to realize the automation of breast cancer pathological diagnosis. BRIEF DESCRIPTION OF DRAWINGS

[0028] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings of the embodiments of the present disclosure will be briefly introduced below. Obviously, the drawings described below only relate to some embodiments of the present disclosure, but not limit the present disclosure.

[0029] Figure 1 A flow cytometry microscopic imaging device according to an embodiment of the present disclosure is shown;

[0030] Figure 2 A flow chart of a training method for a pleural effusion cell classification model according to an embodiment of the present disclosure is shown;

[0031] Figure 3 An effect diagram of classifying the full-negative sample image set and the full-positive sample image set according to step 203 in the method is shown; Figure 2

[0032] ​Figure 4 A structural schematic diagram of the VGG19 network is shown;

[0033] Figure 5 A process of training and testing a model for ascites cell classification based on labeled four training data sets is shown;

[0034] Figure 6 A more specific flowchart of automatically labeling the full negative sample image set and the full positive sample image set is shown;

[0035] Figure 7 An effect diagram of segmenting a cell sample image is shown;

[0036] Figure 8 A system for breast cancer prediction according to an embodiment of the present disclosure is shown;

[0037] Figure 9 A statistical chart of the positive probability score given by the trained ascites cell classification model to an unknown ascites sample according to an embodiment of the present disclosure is shown;

[0038] Figure 10 A schematic block diagram of a training device for an ascites cell classification model according to an embodiment of the present disclosure is shown;

[0039] Figure 11 A schematic block diagram of an image processing device of a training device for an ascites cell classification model according to another embodiment of the present disclosure is shown;

[0040] Figure 12 An architectural schematic diagram of an electronic device according to an embodiment of the present disclosure is shown; and

[0041] Figure 13 A schematic diagram of a storage medium according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0042] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings, and obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative labor also belong to the scope of protection of the present application.

[0043] The terms used in the specification are those general terms currently widely used in the art in consideration of the functions in relation to the present disclosure, but the terms can be changed according to the intention of those of ordinary skill in the art, precedents, or new technology in the art. Also, specific terms can be selected by the applicant, and in this case, the detailed meaning thereof will be described in the detailed description of the present disclosure. Therefore, the terms used in the specification should not be understood as simple names, but based on the meaning of the terms and the overall description of the present disclosure.

[0044] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, however, any number of different modules can be used and run on the user terminal and / or server. The modules are only illustrative, and different aspects of the system and method can use different modules.

[0045] Flowcharts are used in the present application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed in sequence. On the contrary, various steps can be processed in reverse order or simultaneously as needed. Meanwhile, other operations can also be added to these processes, or one or more steps of operations can be removed from these processes.

[0046] One aspect of the present disclosure proposes a flow cytometry microscopic imaging device, which realizes high-speed cell imaging detection, and has the advantages of both rapid detection and rich image information. Another aspect proposes a method of automatically annotating the cell image of the pleural effusion sample obtained by the flow cytometry microscopic imaging device, extracting features and learning features of the automatically annotated imaging data based on a neural network, obtaining a trained pleural effusion classification model for classifying breast cancer cells, and finally making an objective and accurate judgment on whether breast cancer cells exist in the sample and the positive probability of breast cancer cells.

[0047] Figure 1 A flow cytometry microscopic imaging device 100 according to an embodiment of the present disclosure is shown. As shown, the device 100 can include a light source 101, a condenser lens group 102, a flow tube 103, a microscope objective 104, a tube lens 105, and a camera 106. As those skilled in the art should understand, in addition to the above components, the flow cytometry microscopic imaging device 100 can also include other components, and is not limited thereto. Figure 1 As shown, the condenser lens group 102, the microscope objective 104, and the tube lens 105 can be located on the path of the light beam.

[0048] Figure 1 As shown, the condenser lens group 102, the microscope objective 104, and the tube lens 105 can be located on the path of the light beam.

[0049] ​According to an embodiment of the present disclosure, the light source 101 can be configured to emit a visible light beam for providing illumination. The visible light refers to white light or monochromatic light in the visible light band, for example, light between 0.4 and 0.8 μm, including laser light. The light source 101 can be a laser configured to emit pulsed laser light with modulated pulse width and pulse frequency. The light source 101 can further include other components, for example, a control component configured to determine the pulse width and the pulse frequency of the laser according to the flow rate of the flowing particles and the image acquisition frame rate of the image acquisition module (e.g., a camera), respectively, and generate a control signal for driving the laser and a synchronization trigger signal for the image acquisition module corresponding to the control signal according to the determined pulse width and pulse frequency; a driving component configured to generate a driving current according to the control signal to drive the laser to emit pulsed laser light at the determined pulse width and pulse frequency.

[0050] According to an embodiment of the present disclosure, the thoracic and abdominal fluid sample stream containing the particles can flow in the flow channel 103. The sheath fluid wrapping the thoracic and abdominal fluid sample stream enters the flow channel 103 as the sample stream, and the particles in the sample stream flow through the center of the flow channel 103 by controlling the flow of the sample stream in the flow channel 103.

[0051] The microscope objective 104 can adopt an infinite objective, and can be configured to focus on the center of the flow channel 103 to image the flowing particles.

[0052] According to an embodiment of the present disclosure, the condenser lens group 102 can be configured to condense the light beam and uniformly irradiate the light beam on the flow channel 103. The condenser lens group 102 can form Kohler illumination with the light source 101. The Kohler illumination can change the concentric light beam emitted by the light source into a plurality of parallel light beams, which uniformly irradiate on the target object (e.g., the flow channel 103 here, more specifically, the particles in the flow channel 103), so that the illumination can be uniform and the light efficiency is high, and the imaging effect of the imaging system is better. The condenser lens group 102 can include two lenses, a first lens arranged close to the light source 101 and a second lens arranged after the first lens (i.e., farther away from the light source 101 than the first lens), and the first lens condenses the light beam from the light source 101 on the front focal plane of the second lens. The condenser lens group 102 can further include more lenses as long as the light beam from the light source 101 can be uniformly emitted.

[0053] According to an embodiment of the present disclosure, the microscope objective 104 can be configured to focus on the center of the flow channel 103 to collect light passing through the particles in the flow channel 103, wherein the working distance of the microscope objective 104 is greater than the distance from the center of the flow channel 103 to the outer surface of the flow channel 103, and the numerical aperture (NA) of the microscope objective 104 is determined based on a predetermined resolution and the wavelength of the light beam.

[0054] For example, the objective lens 104 can include one or more lenses. Optionally, the objective lens 104 can be an infinity objective lens, and light passing through the infinity objective lens is emitted in parallel light beams to infinity. Applying the infinity objective lens to the objective lens 104 can reduce the influence of aberration on observation to a greater extent.

[0055] The tube lens 105 can be located between the objective lens 104 and the image plane, and can be configured to cooperate with the objective lens 104 to magnify the image of the microparticle and converge the magnified image of the microparticle on the image plane. That is, the objective lens 104 and the tube lens 105 together have a magnification function, and the magnification can be determined based on the focal length of the objective lens 104 and the tube lens 105.

[0056] For example, the tube lens 105 can include one or more lenses. The tube lens 105 can converge light emitted by the infinity objective lens to form an image on a finite image plane. Adjusting the focal length of the tube lens 105 can adjust the magnification of the image.

[0057] The camera 106 is disposed on the image plane and is configured to capture the image of the magnified microparticle on the image plane.

[0058] Based on the flow cytometry microscopic imaging device described above, the cell images of the cells passing through the flow channel can be captured one by one. Next, the cells in a sample of ascites can be quantitatively analyzed based on the analysis software, thereby providing statistical information of the sample and giving the probability of breast cancer, assisting in the screening and diagnosis of breast cancer. With the development of deep learning techniques represented by convolutional neural networks in recent years, more and more deep learning methods have been used to assist medical examination and pathological diagnosis, but most of the methods need to rely on a large amount of labeled database, and the labeling of the training set requires expensive cost. Therefore, the present disclosure proposes a method for automatically labeling and analyzing the cell images in the ascites breast cancer full-negative and full-positive samples obtained by the flow cytometry microscopic imaging device described above, which better solves the problem of high cost of manual labeling data, and can more accurately learn the characteristics of breast cancer cells in the ascites sample to accurately give the probability of breast cancer and assist in the screening and diagnosis of breast cancer.

[0059] At least one embodiment of the present disclosure provides a training method for an ascites cell classification model. For example, the method can be performed by a computer device integrated with a flow cytometry microscopic imaging device as shown in Figure 1 The computer device can be a single or multiple computer devices, and each computer processing device can have, for example, a central processing unit (CPU), a memory, a storage device, an input device, and an output device. Figure 12the architecture shown. For another example, the training method for a pleural effusion cell classification model of embodiments of the present disclosure can be implemented by a remote computer system in communication with a flow cytometric microscopy device, which can be a server cluster or a single / multiple computer devices, and the training method for a pleural effusion cell classification model can be executed by multiple servers arranged in a distributed manner, or can be distributedly executed by each computer device.

[0060] Figure 2 A flow chart of a training method 200 for a pleural effusion cell classification model according to embodiments of the present disclosure is shown. As described above, the method 200 includes steps S201-S204. Figure 2

[0061] First, in step S201, in response to a pleural effusion breast cancer full negative sample liquid including first particles, second particles and third particles flowing through a flow channel of a flow cytometric microscopy device, images of the particles in the full negative sample liquid are acquired to obtain a full negative sample image set including a set of images of the first particles, a set of images of the second particles and a set of images of the third particles.

[0062] Here, a pleural effusion sample of a patient not suffering from breast cancer is defined as a pleural effusion breast cancer full negative sample. As well known to those skilled in the art, all large cells in the cells of the pleural effusion breast cancer full negative sample are mesothelial cells.

[0063] For example, in embodiments of the present disclosure, the first particles are impurities, the second particles are lymphocytes and the third particles are mesothelial cells.

[0064] For example, sheath fluid wrapping the cell flow of the pleural effusion breast cancer full negative sample can be flowed into the flow channel 103 of the flow cytometric microscopy device 100 as a sample flow, and the sample flow can be kept in the center of the flow channel 103. In response to the particles passing through the flow channel, a picture of the cell sample flow at the time of taking a picture is acquired by the camera 106, i.e., from the camera 106. In this way, for the impurities, lymphocytes and mesothelial cells included in the pleural effusion sample flow of the full negative of breast cancer, a full negative sample image set including a set of images of the impurities, a set of images of the lymphocytes and a set of images of the mesothelial cells is obtained.

[0065] In step 202, in response to a pleural effusion breast cancer full positive sample liquid including first particles, second particles and fourth particles flowing through a flow channel of a flow cytometric microscopy device, images of the particles in the full positive sample liquid are acquired to obtain a full positive sample image set including a set of images of the first particles, a set of images of the second particles and a set of images of the fourth particles.

[0066] ​Here, the pleural effusion sample at a special stage where all mesothelial cells have been cancerized into breast cancer cells is defined as a pleural effusion breast cancer full-positive sample, and the full-positive sample can be obtained based on the pathologic detection by a doctor. As understood by those skilled in the art, all the large cells in the cells of the pleural effusion breast cancer full-positive sample are breast cancer cells.

[0067] For example, in an embodiment of the present disclosure, the first type of microparticles are impurities, the second type of microparticles are lymphocytes, and the fourth type of microparticles are breast cancer cells.

[0068] For example, the sheath fluid wrapping the cell stream of the pleural effusion breast cancer full-positive sample can be flowed into the flow channel 103 of the flow cytometry micro-imaging device 100 as a sample stream, and the sample stream can be kept in the center of the flow channel 103. In response to the passage of microparticles in the flow channel, the camera 106 can be used to take pictures, i.e., to obtain pictures of the cell sample stream at the time of taking pictures. In this way, for the impurities, lymphocytes, and breast cancer cells included in the pleural effusion sample stream of the full-positive breast cancer, a full-positive sample image set including a set of impurity images, a set of lymphocyte images, and a set of breast cancer cell images can be obtained.

[0069] After obtaining the full-negative sample image set and the full-positive sample image set respectively including the cells of the pleural effusion samples of the full-negative and the full-positive of breast cancer, in step S203, the full-negative sample image set and the full-positive sample image set are automatically annotated based on the morphological differences of the first type of microparticles, the second type of microparticles, the third type of microparticles, and the fourth type of microparticles, thereby generating a first training data set of images of the first type of microparticles, a second training data set of images of the second type of microparticles, a third training data set of images of the third type of microparticles, and a fourth training data set of images of the fourth type of microparticles.

[0070] In an embodiment, when the cells flowing through the channel are imaged, blank background pictures and multiple cells appearing at the same time can be captured, and therefore, before the full-negative sample set and the full-positive sample set are automatically annotated, the single-cell sample images initially obtained by the cell micro-imaging system can be selected and segmented based on image screening and segmentation techniques. For example, the selection here can be based on manual selection or based on the similarity between images with cells and blank pictures. The segmentation methods here can include threshold-based segmentation methods, region-based segmentation methods, edge-based segmentation methods, and segmentation methods based on specific theories, etc., and the present disclosure does not limit this.

[0071] The morphology of the particles in the pleural effusion and ascites samples is different based on the type of the particles, for example, the impurities usually have irregular shapes with many sharp edges and corners, while the cells are usually close to regular oval shapes with smooth edges. The process of how to realize the automatic labeling of the pleural effusion and ascites particle images based on the morphological characteristics of the particles in the full negative sample image set and the full positive sample image set will be described later in combination with Figure 6 are described in detail.

[0072] Figure 3 An effect diagram showing the classification of the full negative sample image set and the full positive sample image set based on the above step 203 is shown.

[0073] Figure 3 The left is the full negative sample image set and the full positive sample image set before classification, and the right is the first training data set of the images of the first type of particles, the second training data set of the images of the second type of particles, the third training data set of the images of the third type of particles, and the fourth training data set of the images of the fourth type of particles.

[0074] In step S204, a neural network-based pleural effusion and ascites cell classification model is constructed, and the pleural effusion and ascites cell classification model is trained based on the first training data set, the second training data set, the third training data set and the fourth training data set, to obtain a trained pleural effusion and ascites cell classification model for classifying pleural effusion and ascites cells.

[0075] For example, the number and proportion of the first training data set, the second training data set, the third training data set and the fourth training data set can be adjusted according to actual conditions, which is not limited by the present disclosure.

[0076] For example, in the construction of the neural network model, a neural network-based pleural effusion and ascites cell classification model can be constructed based on the basic VGG19 network according to the characteristics of the pleural effusion and ascites samples.

[0077] In the following description, the VGG19 network architecture is taken as an example for the pleural effusion and ascites cell classification model. It should be noted that the neural network-based pleural effusion and ascites cell classification model can also be constructed using other suitable network architectures, such as VggNet and ResNet architectures, which are not limited by the present disclosure.

[0078] Figure 4A structural diagram of a VGG19 network is shown, which includes an input layer, a hidden layer, and an output layer in turn. The hidden layer has a total of 19 layers, which are composed of 16 convolutional layers and 3 fully connected layers. The 16 convolutional layers are respectively: 2 layers of 3*3 convolutional layers (with a relu activation function followed by a 2*2 max pooling layer); 2 layers of 3*3 convolutional layers (with a relu activation function followed by a 2*2 max pooling layer); 4 layers of 3*3 convolutional layers (with a relu activation function followed by a 2*2 max pooling layer); 4 layers of 3*3 convolutional layers (with a relu activation function followed by a 2*2 max pooling layer); 4 layers of 3*3 convolutional layers (with a relu activation function followed by a 2*2 max pooling layer). The 3 fully connected layers are respectively: 1 layer of a fully connected layer with 4096 nodes (with a relu activation function); 1 layer of a fully connected layer with 4096 nodes (with a relu activation function); 1 layer of a fully connected layer with 1000 nodes (followed by a softmax function).

[0079] The VGG network is related work on ILSVRC 2014, which mainly proves that increasing the depth of the network can affect the final performance of the network to a certain extent. VGG has two structures, VGG16 and VGG19, which have no essential difference, only the network depth is different. In VGG, 3*3 convolution kernels are used instead of 7*7 convolution kernels, and 2*3 convolution kernels are used instead of 5*5 convolution kernels. The main purpose of this is to increase the depth of the network while ensuring the same receptive field, and to a certain extent, to improve the effect of the neural network. For example, the one-layer stacking of 3*3 convolution kernels with a step of 1 can be regarded as a receptive field of 7 (actually it means that 3*3 continuous convolution is equivalent to a 7*7 convolution), and the total number of parameters is 3*(9*C 2 ), if a 7*7 convolution kernel is used directly, the total number of parameters is 49*C 2 , where C represents the number of input and output channels. Obviously, 27*C 2 is less than 49*C 2 , that is, the parameters are reduced; and 3*3 convolution kernel is beneficial to better maintain the image properties.

[0080] Figure 5 A process for training and testing a model for classifying ascites cells based on four labeled training data sets is shown.

[0081] As shown in Figure 5 , an example model for classifying ascites cells includes an input layer, a hidden layer, and an output layer from left to right. The layers are connected in a fully connected structure. For example, the model here can be the VGG19 model described above, and the hidden layer here includes 19 layers, including 16 convolutional layers and 3 fully connected layers.

[0082] Figure 5 The training data set shown in the middle includes four types of microparticle image sets as described above, part of which is used as a training set to train the model for ascites cell classification according to the embodiments of the present disclosure, and the other part is used as a validation set to test the robustness of the trained model.

[0083] For example, in one specific implementation process of training and testing the trained ascites cell classification model based on the method according to the embodiments of the present disclosure, a data set of approximately 10W sub-images is used, and 75% of the same sample is used as a training set and 25% is used as a validation set. After multiple training, verification and testing, the final performance on the stranger data set is 97.4% in precision and 92.4% in recall.

[0084] As well known to those skilled in the art, the neural network training process is roughly as follows: first input the data, then the neural network performs forward propagation, calculates the output of the output layer, and then calculates the predefined loss, then performs error back propagation, and uses the pre-set optimization method to update the parameters in the network, such as weight parameters and threshold parameters. Then repeat the above iteration, stop training after reaching the maximum number of iterations (num_epoch) or the loss value meets a certain condition, so as to obtain a trained ascites cell classification model.

[0085] For example, the loss between the classification prediction value of the final output of the ascites cell classification model and the real label can be calculated based on the cross-entropy loss function. For example, the optimization constraint algorithm can use the stochastic gradient descent method to optimize the network parameters. For example, the learning rate can be set to 0.00003. It should be understood that any other suitable network parameters can also be set, which are not limited in the present disclosure.

[0086] Figure 6 A more specific flowchart of automatically annotating the full negative sample image set and the full positive sample image set based on the morphological differences of the first type of microparticles, the second type of microparticles, the third type of microparticles and the fourth type of microparticles in step S203 is shown. Figure 2

[0087] The full negative sample image set and the full positive sample image set here are image sets of microparticles captured in response to the flow of ascites breast cancer full negative sample and ascites breast cancer full positive sample through the flow channel of the flow cytometry device according to the embodiments of the present disclosure.

[0088] As shown in the figure, step S203 can include the following sub-steps S601-S605. Figure 6

[0089] ​​Specifically, in step S601, for each of the sample images in the set of full negative sample images and the set of full positive sample images, the position of the microparticle region containing the microparticle in the sample image is determined.

[0090] For example, taking the threshold-based method as an example, the method can include the following steps: normalizing the pixels of the sample image, comparing each pixel in the normalized sample image with a predefined first threshold; when the value of the pixel is greater than or equal to the first threshold, setting the pixel value to 1, and when the value of the pixel is less than the first threshold, setting the pixel value to 0; and determining the region in the sample image in which the value of the pixel is 1 as the position of the microparticle region containing the microparticle in the sample image.

[0091] For example, further, after obtaining the region in which the value of the pixel is 1, morphological processing of the image can also be performed on the binarized image to filter out noise in the background, and then the microparticle region is filled, so that the microparticle region with a more accurate position can be obtained in the binarized image.

[0092] For example, the morphological processing here is a corrosion and dilation algorithm. Corrosion performs a “shrinkage” or “refinement” operation on a binary image, and dilation performs a “lengthening” or “thickening” operation on a binary image. Thus, the noise in the background can be eliminated, and the holes in the target foreground can be filled.

[0093] It should be understood that the first threshold described above can be confirmed and set through a large number of orthogonal experiments, and the value of the first threshold is specifically limited as a basis for calculation and evaluation.

[0094] In step S602, the microparticle region is segmented from the sample image according to the determined position of the microparticle region.

[0095] After the position of the microparticle region is determined, the microparticle is segmented from the sample image. The region including the microparticle segmented from the sample image is defined as a second region. It should be understood that, in order not to damage the integrity of the microparticle, the second region segmented should be larger than the microparticle region.

[0096] For example, the second region can be a rectangular region. Specifically, based on the position of the microparticle region in the sample image, the smallest rectangular region surrounding the microparticle region in the sample image is determined. The smallest rectangle has the vertical distance between the two longitudinal end points of the microparticle as one side length of the rectangle, and the vertical distance between the two transverse end points of the microparticle as the second side length. Thus, it can be ensured that the smallest rectangle can encompass the region where the microparticle is located. The smallest rectangular region containing the microparticle is segmented from the second image, i.e., the second region defined above is obtained.

[0097] For example, the segmentation here can also be a threshold-based segmentation method, a region-based segmentation method, an edge-based segmentation method, and a segmentation method based on a specific theory, etc., and the present disclosure does not limit this.

[0098] Figure 7 An effect diagram of segmenting the cell sample image based on step S602 is shown.

[0099] As shown in Figure 7 The morphological differences between the segmented breast cancer cells and mesothelial cells are small, and both have the characteristics of large area, smooth surface and regular shape. The lymphocytes (such as the rightmost particle in the first row of the "other" category) have regular shape, smooth surface and small area, and the impurities have irregular shape, different size and sharp edges.

[0100] In step S603, the area and the circularity of the particle region are calculated.

[0101] For example, the area of the particle region can be obtained by summing the number of all pixels in the particle region, and the circumference of the particle region can be obtained by summing the number of all pixels on the boundary line, and the ratio of the area to the circumference is taken as the circularity. It should be understood that any suitable method of calculating the area and circularity of the image region can also be taken, and the present disclosure does not limit this.

[0102] In step S604, based on the calculated area and circularity and whether the sample image belongs to the full negative sample image set or the full positive sample image set, the sample image is classified.

[0103] Embodiments of the present disclosure classify sample images based on morphological differences of cells in pleural effusion cell samples.

[0104] Specifically, based on the circularity being less than a second threshold, it is determined that the sample image belongs to an impurity image; based on the circularity being greater than the second threshold and the area being less than a third threshold, it is determined that the sample image belongs to a lymphocyte image; based on the circularity being greater than the second threshold, the area being greater than the third threshold, and the sample image belonging to a positive sample image set, it is determined that the sample image belongs to a breast cancer cell image; and based on the circularity being greater than the second threshold, the area being greater than the third threshold, and the sample image belonging to a negative sample image set, it is determined that the sample image belongs to a mesothelial cell image.

[0105] It should be understood that the above-mentioned second threshold and third threshold can be confirmed and set through a large number of orthogonal experiments as a basis for calculation and evaluation, and the present disclosure does not specifically limit the values of the second and third thresholds.

[0106] Next, in step S605, each sample image in the set of full negative sample images and the set of full positive sample images is labeled based on the classification result, thereby generating a first training data set of images of the first type of microparticles, a second training data set of images of the second type of microparticles, a third training data set of images of the third type of microparticles, and a fourth training data set of images of the fourth type of microparticles.

[0107] The above data set labeling process does not need to be performed manually, and can only need to collect a large number of breast cancer full negative and full positive pleural effusion samples to continuously expand the four types of image data sets. Through these data sets, a model for pleural effusion cell classification based on a neural network can be trained.

[0108] In this way, when actually detecting a non-full negative and full positive pleural effusion sample of breast cancer, microparticles in the pleural effusion sample liquid can be presented based on the flow cytometry imaging device described above, and the image of the imaged non-full negative and full positive sample can be classified according to the trained pleural effusion cell classification model. According to whether the sample contains breast cancer cells or further according to the proportion of breast cancer cells, it can be used to determine whether the sample is a breast cancer positive sample.

[0109] Figure 8 A system 800 for breast cancer prediction is shown according to an embodiment of the present disclosure, specifically including an image acquisition component 801, a classification component 802, a classification result statistics component 803, and a diagnosis component 804. For example, the system 800 can classify and detect the type of pleural effusion cells based on the trained pleural effusion cell classification model obtained by the method for pleural effusion cell classification described above.

[0110] The image acquisition component 801 is configured to acquire a set of sample images of microparticles in a pleural effusion sample liquid in response to the pleural effusion sample liquid flowing through a flow channel of a flow cytometry imaging device including first type of microparticles, second type of microparticles, third type of microparticles, and / or fourth type of microparticles.

[0111] For example, the flow cytometry imaging device here is the flow cytometry imaging device 100 described above in combination with Figure 1 The flow cytometry imaging device 100 described above.

[0112] For example, the sheath fluid wrapping the pleural effusion sample cell flow can be flowed into the flow channel 103 of the flow cytometry imaging device 100 as a sample flow, and the sample flow can be kept in the center of the flow channel 103. In response to the microparticles passing through the flow channel, the camera 106 is used to take pictures, i.e., to acquire pictures of the cell sample flow flowing through the camera 106.

[0113] The classification component 802 is configured to receive the set of sample images as input sample images, and obtain a classification result of each input sample image in the set of input sample images based on the trained ascites cell classification model.

[0114] The trained ascites cell classification model herein is a model trained according to the training method for ascites cell classification model described above.

[0115] The classification result statistics component 803 is configured to statistically analyze the classification results of all input sample images in the set of input sample images to determine whether the ascites sample liquid is breast cancer positive.

[0116] For example, the classification result statistics component 803 is configured to obtain a classification result indicating a positive probability score of an input sample image being a breast cancer cell image. For example, the positive probability score herein can be a probability score of each input sample image being a breast cancer cell by the trained ascites cell classification model as described above.

[0117] For example, the classification result statistics component 803 is further configured to statistically analyze the positive probability scores of each input sample image being a breast cancer cell image in the set of input sample images, and respectively draw a scatter plot of the positive probability scores with respect to each input sample image, a number histogram of the positive probability scores with respect to the number of images, or a probability density histogram of the positive probability scores with respect to the probability density.

[0118] For example, the classification result statistics component 803 is further configured to determine that the ascites sample is breast cancer positive when the scatter plot shows that the positive probability scores of more than a fourth threshold number of sample images are higher than a predetermined positive probability score value, determine that the ascites sample is breast cancer positive when the number histogram shows that the positive probability scores of more than a fourth threshold number of sample images are higher than a predetermined positive probability score value, and / or determine that the ascites sample is breast cancer positive when the probability density histogram shows that the positive probability scores of more than a fifth threshold proportion of sample images are higher than a predetermined positive probability score value.

[0119] The diagnosis component 804 is configured to make a prediction of breast cancer based on the determination of whether the ascites sample liquid is breast cancer positive.

[0120] For example, if the ascites sample liquid is breast cancer positive, it is determined that the patient has breast cancer.

[0121] It should be understood that the fourth threshold and the fifth threshold described above can be confirmed and set through a large number of orthogonal experiments, and the values of the fourth threshold and the fifth threshold are not specifically limited in the disclosure as a basis for calculation and evaluation.

[0122] The following will be described in combination withFigure 9 A comparison chart between the results of the detection of a plurality of particle sample images in a sample acquired by the flow microscopic imaging device according to the embodiments of the present disclosure by the trained ascites cell classification model according to the embodiments of the present disclosure and the results of the pathological detection of the same sample by a doctor is introduced.

[0123] In order to acquire the comparison chart, the unknown sample is divided into two parts, one of which is given to the doctor for pathological analysis, and the other of which is used for statistical analysis by the system 800 for breast cancer prediction described above.

[0124] Figure 9 A statistical chart of the positive probability scores given by the trained ascites cell classification model according to the embodiments of the present disclosure to unknown ascites samples is shown.

[0125] As shown in Figure 9 each row, the three charts of each row are respectively a scatter plot of the sample positive probability score with respect to each input sample image, a number histogram of the positive probability score with respect to the number of images, or a probability density histogram of the positive probability score with respect to the probability density.

[0126] As shown in Figure 9 the left three samples are all samples diagnosed as positive by the doctor, and according to the analysis and statistics of the system 800, it can be seen that the positive probability of most of the images obtained is close to 1, and the top peak of the probability density histogram is on the right (close to 1). While the right three samples are all samples diagnosed as negative by the doctor, and according to the analysis and statistics of the system 800, it can be concluded that the positive probability of most of the images is close to 0, and the top peak of the probability density histogram is on the left (close to 0). It can be said that the analysis and statistics results of the system 800 are basically consistent with the doctor's diagnosis, and it is also confirmed that the statistical chart obtained can assist the doctor in the pathological detection of breast cancer.

[0127] In addition, since the main difficulty of ascites sample cell recognition lies in the differentiation and recognition between mesothelial cells and breast cancer cells with small morphological differences, after acquiring the sample image set, the image acquisition component 801 can further screen out mesothelial cells and breast cancer cells based on the morphological differences of the particles in the ascites sample.

[0128] According to one embodiment of the present disclosure, as shown in the dashed box in Figure 8 the system 800 further includes a particle screening component 805 configured to screen out the third type of particles and the fourth type of particles in the sample image set based on the morphological differences of the first type of particles, the second type of particles, the third type of particles and the fourth type of particles, as the input sample image set.

[0129] For example, the particle screening component 805 is further configured to: determine the position of the particle region including the particle in the sample image of the sample image set; segment the particle region from the sample image based on the determined position of the particle region; calculate the area and the circularity of the particle region; determine that the sample image is an image of the third type of particle or an image of the fourth type of particle based on that the circularity is greater than a second threshold value and the area is greater than a third threshold value; and screen out the image of the third type of particle and the image of the fourth type of particle in the sample image set as the input sample image set.

[0130] For example, the particle screening component 805 is further configured to: normalize the pixels of the sample image, compare each pixel in the normalized sample image with a predefined first threshold value; set the pixel value to 1 when the value of the pixel is greater than or equal to the first threshold value, and set the pixel value to 0 when the value of the pixel is less than the first threshold value; and determine the region with a pixel value of 1 in the sample image as the position of the particle region containing the particle in the sample image.

[0131] Based on the above embodiments, referring to Figure 10 As shown in FIG. 10, it is a schematic block diagram of the training device 1000 for the ascitic fluid cell classification model according to the embodiments of the present disclosure. The device 1000 at least includes a training data set acquisition component 1001, a label generation component 1002 and a training component 1003. In the embodiments of the present disclosure, the training data set acquisition component 1001, the label generation component 1002 and the training component 1003 can be integrated in one flow cytometry microscopic imaging device, or can be divided into multiple devices, connected and communicated with each other, composed of a system for use, etc. For example, the training data set acquisition component 1001 can be a part of the flow cytometry microscopic imaging device, and the label generation component 1002 and the training component 1003 can be a computer device in communication with the flow cytometry microscopic imaging device, etc.

[0132] It should be understood that the training device 1000 for the ascitic fluid cell classification model provided by the embodiments of the present disclosure can implement the foregoing training method 200 for the ascitic fluid cell classification model, and can also achieve similar technical effects to the foregoing training method 200 for the ascitic fluid cell classification model.

[0133] Specifically, the training data set acquisition component 1001 is configured to: in response to the ascitic fluid breast cancer full-negative sample liquid including the first type of particle, the second type of particle and the third type of particle flowing through the flow channel of the flow cytometry microscopic imaging device, acquire images of the particles in the full-negative sample liquid to obtain a full-negative sample image set; and in response to the ascitic fluid breast cancer full-positive sample liquid including the first type of particle, the second type of particle and the fourth type of particle flowing through the flow channel of the flow cytometry microscopic imaging device, acquire images of the particles in the full-positive sample liquid to obtain a full-positive sample image set.

[0134] The label generating component 1002 is configured to automatically label the full negative sample image set and the full positive sample image set based on the morphological differences of the first type of particles, the second type of particles, the third type of particles and the fourth type of particles, thereby generating a first training data set of images of the first type of particles, a second training data set of images of the second type of particles, a third training data set of images of the third type of particles and a fourth training data set of images of the fourth type of particles.

[0135] The training component 1003 is configured to construct a neural network-based ascites cell classification model, and train the ascites cell classification model based on the first training data set, the second training data set, the third training data set and the fourth training data set, to obtain a trained ascites cell classification model for classifying ascites cells.

[0136] For example, wherein the flow cytometry microscopic imaging device includes a light source, a condenser lens group, a flow channel, a microscope objective, a tube lens, a camera, wherein the flow cytometry microscopic imaging device acquires images of particles in the ascites sample liquid includes: the flow cytometry microscopic imaging device controls the cells of the ascites sample liquid to flow stably through the center of the flow channel by controlling the flow of the ascites sample liquid; the light source is configured to emit a visible light beam; the condenser lens group is configured to condense the light beam and uniformly irradiate the light beam on the flow channel; the microscope objective adopts an infinite objective, focuses on the center of the flow channel, and images the flowing cells; the tube lens converges the outgoing light of the infinite objective to form an image, and adjusting the focal length of the tube lens can adjust the magnification of the image; and the camera is located at the back focal plane of the tube lens, takes the enlarged image, and obtains the images of the particles.

[0137] For example, in the generation of the first training data set of images of the first type of microparticle, the second training data set of images of the second type of microparticle, the third training data set of images of the third type of microparticle, and the fourth training data set of images of the fourth type of microparticle, the following steps are included: for each sample image in the set of full negative sample images and the set of full positive sample images, determining the location of a microparticle region containing microparticles in the sample image; segmenting the microparticle region from the sample image according to the determined location of the microparticle region; calculating the area and circularity of the microparticle region; classifying the sample image based on the calculated area and circularity and whether the sample image belongs to the set of full negative sample images or the set of full positive sample images; and labeling each sample image in the set of full negative sample images and the set of full positive sample images based on the classification result, thereby generating the first training data set of images of the first type of microparticle, the second training data set of images of the second type of microparticle, the third training data set of images of the third type of microparticle, and the fourth training data set of images of the fourth type of microparticle.

[0138] For example, in the determination of the location of a microparticle region containing microparticles in a sample image, the following steps are included: normalizing the pixels of the sample image, comparing each pixel in the normalized sample image with a predefined first threshold value; setting the pixel value to 1 when the value of the pixel is greater than or equal to the first threshold value, and setting the pixel value to 0 when the value of the pixel is less than the first threshold value; and determining the region in the sample image where the value of the pixel is 1 as the location of the microparticle region containing microparticles in the sample image.

[0139] For example, in the classification of the sample image based on the calculated area and circularity and whether the sample image belongs to the set of full negative sample images or the set of full positive sample images, the following steps are included: in response to the circularity being less than a second threshold value, determining that the sample image is an image of the first type of microparticle; in response to the circularity being greater than the second threshold value and the area being less than a third threshold value, determining that the sample image is an image of the second type of microparticle; in response to the circularity being greater than the second threshold value, the area being greater than the third threshold value, and the sample image belonging to the set of full negative sample images, determining that the sample image is an image of the third type of microparticle; and in response to the circularity being greater than the second threshold value, the area being greater than the third threshold value, and the sample image belonging to the set of full positive sample images, determining that the sample image is an image of the fourth type of microparticle.

[0140] For example, in the chest and abdominal effusion cell classification model, the input layer, the hidden layer and the output layer are sequentially included, the hidden layer includes 16 convolutional layers and 3 fully connected layers, and the training of the chest and abdominal effusion cell classification model based on the training data set includes: training the chest and abdominal effusion cell classification model until the loss function of the chest and abdominal effusion cell classification model converges to obtain a trained chest and abdominal effusion cell classification model, wherein the loss function is a cross-entropy loss function, the optimization constraint algorithm is a stochastic gradient descent method, and the learning rate is 0.00003.

[0141] The embodiments of the present disclosure also provide an image processing device of a training device of a chest and abdominal effusion cell classification model, Figure 11 A schematic block diagram of an image processing device of a training device of a chest and abdominal effusion cell classification model according to another embodiment of the present disclosure is shown. For example, as shown in Figure 11 The image processing device 1100 of the training device of the chest and abdominal effusion cell classification model can include one or more processors 1101 and one or more memories 1102. The one or more memories 1102 store computer executable instructions which, when executed by the one or more processors 1101, can perform the training method for the chest and abdominal effusion cell classification model as described above. The one or more memories 1102 and the one or more processors 1101 can be interconnected by a bus system and / or other form of connection mechanism (not shown).

[0142] For example, the one or more memories 1102 and the one or more processors 1101 can be arranged in a single machine, can be arranged in a server, and can also be arranged in the cloud for executing one or more steps of the training method for the chest and abdominal effusion cell classification model as described above.

[0143] For example, the one or more processors 1101 can be a central processing unit (CPU), a digital signal processor (DSP), or other forms of processing units with data processing capability and / or program execution capability, such as a field programmable gate array (FPGA), etc. For example, the central processing unit (CPU) can be an X86 or ARM architecture, etc. The one or more processors 1101 can be a general purpose processor or a special purpose processor, and can control other components in the image processing device 1100 of the training device of the chest and abdominal effusion cell classification model to perform the desired functions.

[0144] For example, the one or more memories 1102 can include any combination of one or more computer program products having a set of instructions executable by the one or more processors 1101. The computer program product can be tangibly embodied in a non-transitory machine readable storage medium. The non-transitory machine readable storage medium can include, for example, a magnetic or optical disk storage, a magnetic tape, a Compact Disc Read Only Memory (CD-ROM), a flash memory, a USB memory, or the like. The computer program product can also be tangibly embodied in a computer memory or random access memory (RAM) associated with the one or more processors 1101. The computer program product can also include a set of instructions executable by the one or more processors 1101, which can be tangibly embodied in a non-transitory machine readable storage medium.

[0145] Furthermore, the method or the apparatus according to the embodiments of the present application can also be implemented by means of Figure 12 the architecture of the electronic device 1200 shown. Figure 12 The architecture of the electronic device is schematically shown. As Figure 12 shown, the electronic device 1200 can include a bus 1201, one or more CPUs 1202, a read only memory (ROM) 1203, a random access memory (RAM) 1204, a communication port 1205 connected to a network, an input / output component 1206, a hard disk 1207, etc. The storage device in the electronic device 1200, such as the ROM 1203 or the hard disk 1207, can store various data or files used in processing and / or communication of the method provided by the present application and program instructions executed by the CPU. The electronic device 1200 can also include a user interface. Of course, Figure 12 The architecture shown is only exemplary, and when implementing different devices, one or at least two components in the electronic device shown can be omitted or added according to actual needs. Figure 12 The architecture shown is only exemplary, and when implementing different devices, one or at least two components in the electronic device shown can be omitted or added according to actual needs.

[0146] The embodiments of the present disclosure also provide a computer readable storage medium. Figure 13 A schematic diagram 1300 of the storage medium according to the embodiments of the present disclosure is shown. As Figure 13 shown, the computer executable instructions 1301 are stored on the computer readable storage medium 1302. When the computer executable instructions 1301 are run by the processor, the method performed by the system for breast cancer prediction and the training method for the ascites cell classification model according to the embodiments of the present disclosure described with reference to the above figures can be executed. The computer readable storage medium includes but is not limited to, for example, volatile memory and / or non-volatile memory. The volatile memory can include, for example, random access memory (RAM) and / or cache memory, etc. The non-volatile memory can include, for example, read only memory (ROM), hard disk, flash memory, etc.

[0147] Embodiments of the present disclosure further provide a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the image processing method of the training apparatus of the ascitic fluid cell classification model according to the embodiments of the present disclosure.

[0148] Those skilled in the art can understand that the disclosed content of the present disclosure can have various modifications and improvements. For example, the various devices or components described above can be implemented by hardware, or by software, firmware, or a combination of some or all of the three.

[0149] In addition, although the present disclosure makes various references to certain units in the system according to the embodiments of the present disclosure, however, any number of different units can be used and run on the client and / or server. The units are only illustrative, and different aspects of the system and method can use different units.

[0150] Those skilled in the art can understand that all or part of the steps in the above method can be instructed by a program to complete by relevant hardware, and the program can be stored in a computer readable storage medium, such as a read-only memory, a magnetic disk or an optical disk, etc. Alternatively, all or part of the steps of the above embodiments can also be implemented using one or more integrated circuits. Accordingly, each module / unit in the above embodiments can be implemented in the form of hardware or in the form of a software functional module. The present disclosure is not limited to any specific form of combination of hardware and software.

[0151] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

[0152] The above is a description of the present disclosure and should not be considered as a limitation thereof. Although exemplary embodiments of the present disclosure are described, those skilled in the art will readily understand that many modifications can be made to the exemplary embodiments without departing from the novel teachings and advantages of the present disclosure. Accordingly, all such modifications are intended to be included within the scope of the present disclosure as defined in the claims. It is to be understood that the above is a description of the present disclosure and should not be considered as a limitation thereof. No limitation is intended to the particular embodiments disclosed, and modifications to the disclosed embodiments and other embodiments are intended to be included within the scope of the claims. The present disclosure is defined by the claims and their equivalents.

Claims

1. A training method for ascites cell classification model, the method comprising: obtaining images of microparticles in a breast ascites full-negative sample liquid flowing through a flow channel of a flow cytometry microscopic imaging device in response to the breast ascites full-negative sample liquid comprising microparticles of a first type, microparticles of a second type and microparticles of a third type, to obtain a full-negative sample image set comprising a set of images of microparticles of the first type, a set of images of microparticles of the second type and a set of images of microparticles of the third type, wherein the microparticles of the first type are impurities, the microparticles of the second type are lymphocytes, the microparticles of the third type are mesothelial cells and the microparticles of a fourth type are breast cancer cells; obtaining images of microparticles in a breast ascites full-positive sample liquid flowing through a flow channel of a flow cytometry microscopic imaging device in response to the breast ascites full-positive sample liquid comprising microparticles of a first type, microparticles of a second type and microparticles of a fourth type, to obtain a full-positive sample image set comprising a set of images of microparticles of the first type, a set of images of microparticles of the second type and a set of images of microparticles of the fourth type; automatically annotating the full-negative sample image set and the full-positive sample image set based on morphological differences of the microparticles of the first type, the microparticles of the second type, the microparticles of the third type and the microparticles of the fourth type, to generate a first training data set of images of microparticles of the first type, a second training data set of images of microparticles of the second type, a third training data set of images of microparticles of the third type and a fourth training data set of images of microparticles of the fourth type, wherein generating the first training data set of images of microparticles of the first type, the second training data set of images of microparticles of the second type, the third training data set of images of microparticles of the third type and the fourth training data set of images of microparticles of the fourth type comprises: for each sample image in the full-negative sample image set and the full-positive sample image set, determining a location of a microparticle region containing microparticles in the sample image, segmenting the microparticle region from the sample image according to the determined location of the microparticle region, calculating an area and a circularity of the microparticle region, classifying the sample image based on the calculated area and circularity and whether the sample image belongs to the full-negative sample image set or the full-positive sample image set, and annotating each sample image in the full-negative sample image set and the full-positive sample image set based on the classification result, to generate the first training data set of images of microparticles of the first type, the second training data set of images of microparticles of the second type, the third training data set of images of microparticles of the third type and the fourth training data set of images of microparticles of the fourth type; and constructing an ascites cell classification model based on a neural network, and training the ascites cell classification model based on the first training data set, the second training data set, the third training data set and the fourth training data set, to obtain a trained ascites cell classification model for classifying ascites cells.

2. The method of claim 1, wherein, the flow cytometry microscopic imaging device comprises a light source, a condenser lens group, a flow channel, a microscopic objective, a tube lens and a camera, and the flow cytometry microscopic imaging device obtains images of microparticles in an ascites sample liquid comprises: The flow cytometry imaging device controls the cells of the pleural effusion sample liquid to flow stably through the center of the flow channel by controlling the flow of the pleural effusion sample liquid; The light source is configured to emit a visible light beam; The condenser lens group is configured to condense the light beam and uniformly irradiate the light beam on the flow channel; The microscope objective is an infinite objective, focused on the center of the flow channel, and images the flowing cells; The tube lens converges the outgoing light of the infinite objective to form an image, and adjusting the focal length of the tube lens can adjust the magnification of the image; and The camera is located at the back focal plane of the tube lens, and captures the enlarged image to obtain the image of the microparticle.

3. The method of claim 1, wherein, Determining the location of the microparticle region containing microparticles in the sample image comprises: normalizing the pixels of the sample image, comparing each pixel in the normalized sample image with a predefined first threshold value; when the value of the pixel is greater than or equal to the first threshold value, the pixel value is set to 1, and when the value of the pixel is less than the first threshold value, the value of the pixel is set to 0; and determining the region in the sample image whose pixel value is 1 as the location of the microparticle region containing microparticles in the sample image.

4. The method of claim 3, wherein, Classifying the sample image based on the calculated area and roundness and whether the sample image belongs to the full negative sample image set or the full positive sample image set comprises: in response to the roundness being less than a second threshold value, determining that the sample image belongs to a first type of microparticle image; in response to the roundness being greater than a second threshold value and the area being less than a third threshold value, determining that the sample image belongs to a second type of microparticle image; in response to the roundness being greater than a second threshold value, the area being greater than a third threshold value, and the sample image belonging to the full negative sample image set, determining that the sample image belongs to a third type of microparticle image; and in response to the roundness being greater than a second threshold value, the area being greater than a third threshold value, and the sample image belonging to the full positive sample image set, determining that the sample image belongs to a fourth type of microparticle image.

5. The method of claim 1, wherein, The pleural effusion cell classification model sequentially comprises an input layer, a hidden layer and an output layer, the hidden layer comprises 16 convolutional layers and 3 fully connected layers, and Training the pleural effusion cell classification model based on the training data set comprises: training the pleural effusion cell classification model until the loss function of the pleural effusion cell classification model converges to obtain a trained pleural effusion cell classification model, wherein the loss function is a cross-entropy loss function, the optimization constraint algorithm is a stochastic gradient descent method, and the learning rate is 0.00003.

6. A training device for a pleural effusion cell classification model, the device comprising: a training data set obtaining component configured to, in response to a pleural effusion breast cancer fully negative sample liquid including first type of particles, second type of particles and third type of particles flowing through a flow channel of a flow cytometry micro-imaging device, obtain images of the particles in the fully negative sample liquid to obtain a fully negative sample image set, wherein the first type of particles are impurities, the second type of particles are lymphocytes, the third type of particles are mesothelial cells and the fourth type of particles are breast cancer cells; and in response to a pleural effusion breast cancer fully positive sample liquid including the first type of particles, the second type of particles and the fourth type of particles flowing through the flow channel of the flow cytometry micro-imaging device, obtain images of the particles in the fully positive sample liquid to obtain a fully positive sample image set; a label generating component configured to, based on morphological differences of the first type of particles, the second type of particles, the third type of particles and the fourth type of particles, automatically label the fully negative sample image set and the fully positive sample image set to generate a first training data set of images of the first type of particles, a second training data set of images of the second type of particles, a third training data set of images of the third type of particles and a fourth training data set of images of the fourth type of particles, wherein to generate the first training data set of images of the first type of particles, the second training data set of images of the second type of particles, the third training data set of images of the third type of particles and the fourth training data set of images of the fourth type of particles, the label generating component is configured to, for each sample image in the fully negative sample image set and the fully positive sample image set, determine a position of a particle region containing particles in the sample image; segment the particle region from the sample image according to the determined position of the particle region; calculate an area and a circularity of the particle region; classify the sample image based on the calculated area and circularity and whether the sample image belongs to the fully negative sample image set or the fully positive sample image set; and label each sample image in the fully negative sample image set and the fully positive sample image set based on the classification result to generate the first training data set of images of the first type of particles, the second training data set of images of the second type of particles, the third training data set of images of the third type of particles and the fourth training data set of images of the fourth type of particles; and a training component configured to construct a pleural effusion cell classification model based on a neural network, and train the pleural effusion cell classification model based on the first training data set, the second training data set, the third training data set and the fourth training data set to obtain a trained pleural effusion cell classification model for classifying pleural effusion cells.

7. A system for breast cancer prediction, comprising: an image obtaining component configured to, in response to a pleural effusion sample liquid including first type of particles, second type of particles, third type of particles and / or fourth type of particles flowing through a flow channel of a flow cytometry micro-imaging device, obtain a sample image set of particles in the pleural effusion sample liquid; a classification component configured to receive the set of sample images as a set of input sample images, and obtain a classification result of each input sample image in the set of input sample images based on the trained ascites cell classification model; a classification result statistics component configured to statistically analyze the classification results of all input sample images in the set of input sample images to determine whether the ascites sample liquid is breast cancer positive; and a diagnosis component configured to make a prediction of breast cancer based on the determination of whether the ascites sample liquid is breast cancer positive, wherein the trained ascites cell classification model is trained according to the method of any one of claims 1-5.

8. The system of claim 7, further comprising a particle screening component configured to screen out third type of particles and fourth type of particles in the set of sample images as the set of input sample images based on morphological differences between the first type of particles, the second type of particles, the third type of particles and the fourth type of particles.

9. The system of claim 8, wherein, The particle screening component is further configured to: determine a location of a particle region including a particle in a sample image of the set of sample images; segment the particle region from the sample image based on the determined location of the particle region; calculate an area and a circularity of the particle region; determine that the sample image is an image of the third type of particles or an image of the fourth type of particles based on that the circularity is greater than a second threshold value and the area is greater than a third threshold value; and screen out the image of the third type of particles and the image of the fourth type of particles in the set of sample images as the set of input sample images. The particle screening component is further configured to:

10. The system of claim 9, wherein, normalize pixels of the sample image, compare each pixel in the normalized sample image with a predefined first threshold value; set the pixel value to 1 when the pixel value is greater than or equal to the first threshold value, and set the pixel value to 0 when the pixel value is less than the first threshold value; and determine a region in the sample image where the pixel value is 1 as a location of a particle region including a particle in the sample image. The first type of particles are impurities, the second type of particles are lymphocytes, the third type of particles are mesothelial cells, and the fourth type of particles are breast cancer cells. The classification component is further configured to obtain a classification result indicating a positive probability score of the input sample image being a breast cancer cell image, and 11. The system of claim 7, wherein, The classification result statistics component is further configured to:

12. The system of claim 11, wherein, statistically analyze the positive probability scores of each input sample image in the set of input sample images, and plot a scatter plot of the positive probability scores against each input sample image, a number histogram of the positive probability scores against the number of images, or a probability density histogram of the positive probability scores against the probability density. The classification result statistics component is further configured to: determine that the ascites sample is breast cancer positive when the scatter plot shows that the positive probability scores of more than a fourth threshold number of sample images are higher than a predetermined positive probability score value.

13. The system of claim 12, wherein, ​ ​ determining the pleural effusion sample as breast cancer positive when the number histogram shows that a proportion of sample images exceeding a fourth threshold number have a positive probability score higher than a predetermined positive probability score value; and / or determining the pleural effusion sample as breast cancer positive when the probability density histogram shows that a proportion of sample images exceeding a fifth threshold have a positive probability score higher than a predetermined positive probability score value.

14. An electronic device comprising a memory and a processor, wherein, The memory has stored thereon processor-readable program code which, when executed by the processor, performs the method according to any one of claims 1-5.

15. An electronic device comprising a memory and a processor, wherein, The memory has stored thereon processor-readable program code which, when executed by the processor, performs the following steps: acquiring a set of sample images of particles in the pleural effusion sample liquid in response to a flow of the pleural effusion sample liquid comprising particles of the first type, the second type, the third type and / or the fourth type flowing through a flow channel of a flow cytometric micro-imaging device; receiving the set of sample images as a set of input sample images and obtaining a classification result of each input sample image in the set of input sample images based on a trained pleural effusion cell classification model; and statistically analyzing the classification results of all input sample images in the set of input sample images to determine whether the pleural effusion sample liquid is breast cancer positive, wherein the trained pleural effusion cell classification model is a pleural effusion cell classification model trained according to the method of any one of claims 1-5.

16. A computer-readable storage medium having stored thereon computer-executable instructions for performing the method according to any one of claims 1-5.

17. A computer-readable storage medium having stored thereon computer-executable instructions for performing the following steps: acquiring a set of sample images of particles in the pleural effusion sample liquid in response to a flow of the pleural effusion sample liquid comprising particles of the first type, the second type, the third type and / or the fourth type flowing through a flow channel of a flow cytometric micro-imaging device; receiving the set of sample images as a set of input sample images and obtaining a classification result of each input sample image in the set of input sample images based on a trained pleural effusion cell classification model; and statistically analyzing the classification results of all input sample images in the set of input sample images to determine whether the pleural effusion sample liquid is breast cancer positive, wherein wherein the trained pleural effusion cell classification model is a pleural effusion cell classification model trained according to the method of any one of claims 1-5.

Images