A system for classifying cervical cell images

Abstract

A system for classifying cervical cell images, comprising: a flow microscopic imaging device, which collects images of particles in a liquid sample to be detected including exfoliated cervical cells, and obtains one or more single particle images including only single particles; a screening device, which pre-classifies the acquired one or more single particle images using a pre-classification model based on a first-level neural network, so as to screen out cervical cell images from the one or more single particle images; an image segmentation device, which segments the cervical cell images using an image segmentation model based on a second-level neural network, so as to segment the cervical cell images into a first area, a second area, and a third area; and a classification device, which determines a plurality of feature parameters characterizing the morphological features of the cervical cells according to the segmented areas, and identifies the category of the cervical cells based on the plurality of feature parameters using a classification model based on a third-level neural network.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of image recognition technology, and in particular, to a system, electronic device, and computer-readable storage medium for classifying cervical cell images acquired based on flow cytometry microscopy imaging technology. Background Art

[0002] Cervical cancer is a common gynecological malignancy. The number of cervical cancer cases in my country is about 110,000 per year, accounting for more than one-fifth of the global total. Timely cervical cancer screening is conducive to controlling the number of cervical cancer cases and reducing the incidence and mortality of cancer.

[0003] TCT examination is the abbreviation of liquid-based thin-layer cytology examination, which is one of the basic methods for cervical cancer screening. The doctor collects the exfoliated cervical cells with a brush, adds them to the cell preservation solution, and then observes the cell morphology under a microscope after centrifugation and staining, and classifies and identifies the exfoliated cervical cells to determine whether the patient has cancer. The entire inspection process is time-consuming and labor-intensive, and the workload of doctors is large under large-scale inspections. The flow cytometer has a fast detection speed, which can be as high as thousands of cells per second, but the information obtained by its detection is of few dimensions, and it is impossible to classify cells by high-dimensional information such as cell volume morphology. Therefore, the accuracy is not high, and it can usually only be used for quantitative analysis and is often used for initial screening of cells.

[0004] Although some cervical cell classification algorithms can achieve cell classification in microscope images, this method still relies on microscope images and requires operations such as preparation of slides and photographing under a microscope.

[0005] In order to solve the above-mentioned problem of cervical exfoliated cell identification, an improved cervical cell classification system based on flow microscopy imaging device is proposed. Summary of the invention

[0006] The present disclosure is made in view of the above problems. One object of the present disclosure is to provide a system, an electronic device and a computer-readable storage medium for classifying cervical cell images.

[0007] An embodiment of the present disclosure provides a system for classifying cervical cell images, comprising: a flow microscopy imaging device, which collects images of particles in a liquid sample to be detected including exfoliated cervical cells, and obtains one or more single-particle images including only single particles; a screening device, which pre-classifies the acquired one or more single-particle images using a pre-classification model based on a first-level neural network, so as to screen out cervical cell images from the one or more single-particle images; an image segmentation device, which segments the cervical cell images using an image segmentation model based on a second-level neural network, so as to segment the cervical cell images into a first region, a second region, and a third region; and a classification device, which determines a plurality of feature parameters characterizing the morphological features of the cervical cells according to the segmented regions, and identifies the category of the cervical cells based on the plurality of feature parameters using a classification model based on a third-level neural network.

[0008] For example, according to the system of an embodiment of the present disclosure, the liquid sample to be detected is obtained by the following steps: centrifuging the original liquid to be processed to obtain multiple particles in the original liquid to be processed; removing the supernatant of the multiple particles obtained by the centrifugation operation, and resuspending the multiple particles from the supernatant removed in a particle fixing solution; and pipetting the resuspended cell fixing solution to obtain the liquid sample to be detected, wherein all the particles in the liquid sample to be detected are in a single particle state.

[0009] For example, according to the system of an embodiment of the present disclosure, the flow microscopy imaging device includes a light source, a focusing lens group, a flow pipe, a microscope objective, a tube lens, and a camera, and the flow microscopy imaging device controls the flow of the liquid sample to be detected to control the particles in the liquid sample to be detected to flow stably through the center of the flow pipe; the light source is configured to emit a visible light beam; the focusing lens group is configured to gather the light beam and irradiate the light beam uniformly on the flow pipe; the microscope objective uses an infinite objective lens, focuses on the center of the flow pipe, and images the particles flowing through; the tube lens converges the light emitted by the infinite objective lens to form an image, and the magnification of the image can be adjusted by adjusting the focal length of the tube lens; and the camera is located at the rear focal plane of the tube lens, and shoots the magnified image to obtain an image including the particles.

[0010] For example, in a system according to an embodiment of the present disclosure, a flow microscopy imaging device is used to capture images of particles in a liquid sample to be detected including exfoliated cervical cells, and obtaining one or more single-particle images including only single particles includes: acquiring an image including particles in response to particles passing through a flow channel of the flow microscopy imaging device; determining the location of one or more particle regions in the image including particles; and cutting out one or more single-particle images including only single particles from the image including particles based on the location of the one or more particle regions.

[0011] For example, in a system according to an embodiment of the present disclosure, determining the location of one or more particle regions in the image including particles includes: acquiring an image that does not include particles in response to no particles passing through the flow pipeline of the flow microscopy imaging device; subtracting the pixel values ​​of the image including particles from the image not including particles pixel by pixel to obtain a residual image; binarizing the residual image to obtain a binarized residual image; determining an area with a pixel value of 1 in the binarized residual image as the location of the one or more particle regions in the residual image; and determining a pixel area in the image including particles corresponding to the location of one or more particle regions in the residual image as the location of the one or more particle regions in the image including particles.

[0012] For example, in a system according to an embodiment of the present disclosure, based on the positioning of the one or more particle regions, cutting out one or more single-particle images including only a single particle from the image including the particles comprises: based on the positioning of the one or more particle regions in the image including the particles, determining one or more minimum rectangular regions surrounding the one or more particle regions in the image including the particles, and cutting out the one or more minimum rectangular regions from the image including the particles as the one or more single-particle images; or extending the one or more minimum rectangular regions by a predetermined number of pixels in all directions, and cutting out one or more extended rectangular regions from the image including the particles as the one or more single-particle images; or taking each of the one or more particle regions as the center, cutting out a region larger than each particle region and having a fixed size from the image including the particles as the one or more single-particle images.

[0013] For example, according to the system of an embodiment of the present disclosure, the first-level neural network is a pre-trained neural network-based binary classification model, which is trained to classify complete cervical cell images from impurity images and / or damaged cell images and / or clumped cell images, and to screen out complete cervical cell images.

[0014] For example, according to the system of an embodiment of the present disclosure, the second-level neural network is a pre-trained image segmentation model based on a neural network, wherein the image segmentation model based on the second-level neural network is used to perform image segmentation on the cervical cell image to segment the cervical cell image into a first region, a second region, and a third region, including: identifying and marking each pixel in the first region of the cervical cell image as a first region pixel, the first region being the background region of the cervical cell image, identifying and marking each pixel in the second region of the cervical cell image as a second region pixel, the second region being the cell region of the cervical cell image; and identifying and marking each pixel in the third region of the cervical cell image as a third region pixel, the third region being the cell nucleus region of the cervical cell image.

[0015] For example, according to the system of an embodiment of the present disclosure, the neural network-based image segmentation model is a Unet-based neural network model.

[0016] For example, according to the system of an embodiment of the present disclosure, the third-level neural network is a pre-trained fully connected neural network, the number of neurons in the input layer of the third-level neural network is equal to the number of feature parameters extracted based on the image segmentation, the number of neurons in the output layer of the fully connected neural network is equal to the number of cell categories, and the third-level neural network is trained to identify the categories of cervical cells based on the feature parameters extracted based on the image segmentation.

[0017] For example, in a system according to an embodiment of the present disclosure, the classification device determines a plurality of feature parameters characterizing the morphological features of the cervical cells according to the segmented areas, and based on the plurality of feature parameters, utilizes a classification model based on a third-level neural network to identify the category of the cervical cells, including: calculating a plurality of feature parameters characterizing the cervical cell image based on the second area and the third area after image segmentation of the cervical cell image; constructing a feature vector of the cervical cell image based on the plurality of feature parameters; and inputting the feature vectors of the plurality of cervical cell images into the third-level neural network to identify the category of the cervical cells.

[0018] For example, in the system according to an embodiment of the present disclosure, the categories of the cervical cells include one or more of normal cells, low-grade lesion cells, high-grade lesion cells, keratinous lesion cells and atypical cells.

[0019] For example, according to a system of an embodiment of the present disclosure, the multiple characteristic parameters include parameters regarding the morphological characteristics of the second region, including a first parameter indicating a cell area, a second parameter indicating a cell diameter, a third parameter indicating a cell circumference, and a fourth parameter indicating a cell roundness; the multiple characteristic parameters also include parameters regarding the morphological characteristics of the third region, including a fifth parameter indicating a cell nuclear area, a sixth parameter indicating a cell nuclear diameter, a seventh parameter indicating a cell nuclear circumference, and an eighth parameter indicating a cell nuclear roundness.

[0020] For example, in a system according to an embodiment of the present disclosure, based on the second area and the third area after image segmentation of the cervical cell image, calculating multiple characteristic parameters characterizing the cervical cell image includes: calculating the total number of all second area pixels as the first parameter; calculating the distance between the two second area pixels that are farthest apart as the second parameter; calculating the total number of second area pixels on the boundary of the second area as the third parameter; calculating the ratio of the first parameter to the second parameter as the fourth parameter; calculating the total number of all third area pixels as the fifth parameter; calculating the distance between the two third area pixels that are farthest apart as the sixth parameter; calculating the total number of third area pixels on the boundary of the third area as the seventh parameter; and calculating the ratio of the fifth parameter to the sixth parameter as the eighth parameter.

[0021] An embodiment of the present disclosure also provides an electronic device, including a memory and a processor, wherein the memory stores a program code readable by the processor, and when the processor executes the program code, the following steps are performed: enabling a flow microscopy imaging device to collect images of particles in a liquid sample to be detected including exfoliated cervical cells, and obtaining one or more single-particle images including only single particles; using a pre-classification model based on a first-level neural network to pre-classify the acquired one or more single-particle images, so as to filter out cervical cell images from the one or more single-particle images; using an image segmentation model based on a second-level neural network to segment the cervical cell images, so as to segment the cervical cell images into a first area, a second area and a third area; and determining a plurality of feature parameters characterizing the morphological features of the cervical cells according to the segmented areas, and based on the plurality of feature parameters, using a classification model based on a third-level neural network to identify the category of the cervical cells.

[0022] An embodiment of the present disclosure also provides a computer-readable storage medium having computer executable instructions stored thereon, the computer executable instructions being used to execute the following steps: causing a flow microscopy imaging device to perform image acquisition on particles in a liquid sample to be detected including exfoliated cervical cells, and obtain one or more single particle images including only single particles; using a pre-classification model based on a first-level neural network to pre-classify the acquired one or more single particle images, so as to filter out cervical cell images from the one or more single particle images; using an image segmentation model based on a second-level neural network to perform image segmentation on the cervical cell images, so as to segment the cervical cell images into a first region, a second region and a third region; and determining a plurality of feature parameters characterizing the morphological features of the cervical cells according to the segmented regions, and based on the plurality of feature parameters, using a classification model based on a third-level neural network to identify the category of the cervical cells.

[0023] Different from the traditional microscope shooting method, the present application uses a flow microscopy imaging device to collect cervical exfoliated cell images, which can realize the rapid shooting and cutting of cervical cell images, thereby obtaining a single particle image including only a single particle. After obtaining the single particle image including only a single particle, a pre-classification mechanism and an image segmentation mechanism based on deep learning are further proposed to achieve the filtering of impurities and the screening of complete cervical cells and the segmentation of the image area of ​​cervical cells, so as to obtain cell morphological characteristic parameters based on the segmented area and realize the rapid identification and differentiation of cervical cell types. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solution of the embodiment of the present disclosure, the following briefly introduces the drawings required for describing the embodiment. The drawings described below are only exemplary embodiments of the present disclosure.

[0025] Figure 1 A schematic diagram of a flow cytometry microscopy imaging device according to an embodiment of the present disclosure is shown;

[0026] Figure 2 A system 200 for classifying cervical cell images according to an embodiment of the present disclosure is shown;

[0027] Figure 3 The invention shows the steps of collecting images of particles in a liquid sample to be detected including exfoliated cervical cells using a flow microscopic imaging device, and obtaining one or more single particle images including only single particles;

[0028] Figure 4 A schematic diagram of the UNet network architecture is shown;

[0029] Figure 5The effect diagram obtained after the three-category segmentation of cervical cells based on the image segmentation model is shown;

[0030] Figure 6 (a) shows an exemplary structure of the third-level neural network of an embodiment of the present disclosure.

[0031] Figure 6 (b) shows five types of cell images obtained by classifying cervical exfoliated cell sample images based on the trained third-level neural network;

[0032] Figure 7 A schematic block diagram showing an electronic device for classifying cervical cells according to an embodiment of the present disclosure; and

[0033] Figure 8 The steps of classifying cervical cells performed by the electronic device for classifying cervical cells according to an embodiment of the present disclosure are shown;

[0034] Fig. 9 The architecture of an electronic device for classifying cervical cells according to an embodiment of the present disclosure is shown; and

[0035] Fig.10 A schematic diagram of a storage medium according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0036] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments of the present application, all other embodiments obtained by ordinary technicians in this field without creative work also fall within the scope of protection of the present application.

[0037] The terms used in this specification are those general terms currently widely used in the art in consideration of the functions of the present disclosure, but these terms may vary according to the intention of a person of ordinary skill in the art, precedents, or new technologies in the art. In addition, specific terms may be selected by the applicant, and in this case, their detailed meanings will be described in the detailed description of the present disclosure. Therefore, the terms used in the specification should not be understood as simple names, but rather as an overall description based on the meaning of the terms and the present disclosure.

[0038] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, any number of different modules can be used and run on the user terminal and / or server. The modules are only illustrative, and different aspects of the system and method can use different modules.

[0039] Flowcharts are used in the present application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, various steps may be processed in reverse order or simultaneously as required. At the same time, other operations may also be added to these processes, or a certain step or several steps of operations may be removed from these processes.

[0040] The following is a non-limiting description of a system for classifying cervical cell images provided according to at least one embodiment of the present disclosure through several examples or embodiments. As described below, different features in these specific examples or embodiments can be combined with each other without conflicting with each other to obtain new examples or embodiments, and these new examples or embodiments also fall within the scope of protection of the present disclosure.

[0041] Traditional TCT testing requires the collection of cervical exfoliated cells, which are then centrifuged and stained, and then the doctor observes the cell morphology under a microscope and classifies the cells according to the cell morphology. The entire cell identification process relies on manpower, which is time-consuming and inefficient. In order to solve the entire cervical exfoliated cell identification process, the present disclosure proposes a system for classifying cervical cell images.

[0042] According to the embodiment of the present disclosure, the system for classifying cervical cell images includes a flow microscopic imaging device, which has both shooting and cutting functions, and can quickly shoot the liquid sample to be tested containing cervical exfoliated cells to obtain an image including one or more particles (for example, cells and impurities, etc.), and realize the cutting of the particle area, so as to obtain a single particle image including only a single particle. After obtaining the single particle image including only a single particle, a pre-classification model based on deep learning is used to realize the screening of impurities, broken cells, and cells that are difficult to distinguish from clusters and intact cells. On this basis, the segmented area of ​​the cells in the single cell image is further obtained based on the image segmentation model, so as to obtain the cell morphological feature parameters based on the segmented area, so that the cervical cell classification model can realize the rapid identification and differentiation of cervical cell types based only on a small number of morphological parameters extracted in advance, instead of needing to identify and differentiate based on all features of the entire image as in the traditional cell classification model.

[0043] Before classification and identification, the doctor uses a brush to collect the patient's cervical exfoliated cells, adds them to the cell treatment solution, performs morphological treatment on the cells (such as staining, cell morphology fixation, etc.), and removes impurities through centrifugation and supernatant removal to obtain cell sample liquid. The liquid sample to be tested is then passed through a flow imaging system to collect cervical cells.

[0044] For example, the liquid sample to be detected is obtained by the following steps: centrifuging the original liquid after cell morphology treatment (such as staining, cell morphology fixation, etc.) to obtain multiple particles in the original liquid; removing the supernatant of the multiple particles obtained by the centrifugation operation, and resuspending the multiple particles from which the supernatant has been removed in a particle fixing solution; and pipetting the resuspended cell fixing solution to obtain the liquid sample to be detected, wherein all the particles in the liquid sample to be detected are in a single particle state.

[0045] For example, the centrifugal operation of the raw liquid to be processed may include: centrifuging the raw liquid to be processed at a predetermined temperature, with a predetermined centrifugal force, and within a predetermined time. The predetermined temperature is, for example, room temperature. Or the predetermined temperature is any reasonable temperature such as 18°C, 20°C, 23°C, 25°C, or 28°C. The predetermined centrifugal force is, for example, any reasonable value such as 2000g, 2500g, 3000g, or 3500g. The predetermined time is, for example, any reasonable time length such as 5 minutes, 6 minutes, 7 minutes, 8 minutes, 9 minutes, or 10 minutes.

[0046] For example, resuspending the plurality of cells from which the supernatant has been removed in a cell fixative may include: determining the actual volume of the cell fixative; mixing methanol and acetic acid in a predetermined ratio to prepare the cell fixative of the actual volume. The actual volume of the cell fixative is, for example, 3 ml, 4 ml, 5 ml, 6 ml, 8 ml, or 10 ml, or any other reasonable volume. The predetermined ratio is 3 to 1, 2 to 1, 4 to 1, or any other reasonable ratio.

[0047] For example, the pipetting of the resuspended cell fixative to obtain the liquid to be detected may include: using a pipette to pipette the resuspended cell fixative to obtain the liquid to be detected by breaking up multiple cells into a single cell state. Using a pipette to pipette the resuspended cell fixative includes: using a pipette to blow the resuspended cell fixative 15 times, 20 times, 25 times, etc.

[0048] Figure 1 FIG. 1 shows a flow cytometry microscopy imaging device 100 according to an embodiment of the present disclosure. Figure 1 As shown, the flow cytometry microscopic imaging device 100 may include a light source 101, a condenser lens group 102, a flow pipe 103, a microscope objective 104, a tube lens 105, and a camera 106. As those skilled in the art should understand, in addition to the above components, the flow cytometry microscopic imaging device 100 may also include other components, and is not limited thereto.

[0049] like Figure 1As shown, the condenser lens group 102, the microscope objective lens 104 and the tube lens 105 can be located on the path of the light beam.

[0050] According to an embodiment of the present disclosure, the light source 101 may be configured to emit a visible light beam for providing illumination. Visible light refers to white light or monochromatic light in the visible light band, for example, light between 0.4 and 0.8 μm, including lasers. The light source 101 may be a laser configured to emit a pulsed laser with a modulated pulse width and pulse frequency. The light source 101 may also include other components, for example, a control component configured to determine the pulse width and pulse frequency of the laser according to the flow velocity of the flowing particles and the image acquisition frame rate of the image acquisition module (for example, a camera), respectively, and generate a control signal for driving the laser and a synchronization trigger signal corresponding to the control signal for the image acquisition module according to the determined pulse width and pulse frequency; a driving component configured to generate a driving current according to the control signal to drive the laser to emit a pulsed laser at a determined pulse width and pulse frequency.

[0051] According to an embodiment of the present disclosure, a sample flow of a liquid to be detected containing particles may flow in the flow pipe 103 , and the particles in the sample flow may flow through the center of the flow pipe 103 by controlling the flow of the sample flow in the flow pipe 103 .

[0052] The microscope objective lens 104 may be an infinity objective lens, and may be configured to focus on the center of the flow channel 103 to image the particles flowing through.

[0053] According to an embodiment of the present disclosure, the focusing lens group 102 can be configured to gather the light beam and evenly illuminate the light beam on the flow pipe 103. The focusing lens group 102 can form Kohler illumination with the light source 101. Kohler illumination can make the concentric light beams emitted by the light source become multiple parallel light beams, which are evenly illuminated on the target object (for example, the flow pipe 103 here, more specifically, the particles in the flow pipe 103), so that the illumination can be uniform and the light efficiency is high, and the imaging effect of the imaging system is better. The focusing lens group 102 may include two lenses, the first lens is arranged close to the light source 101, and the second lens is arranged after the first lens (that is, farther away from the light source 101 than the first lens), and the first lens gathers the light beam from the light source 101 on the front focal plane of the second lens. The focusing lens group 102 may also include more lenses, as long as the light beam from the light source 101 can be evenly emitted.

[0054] According to an embodiment of the present disclosure, the microscope objective 104 can be configured to focus on the center of the flow pipe 103 and collect light passing through the particles in the flow pipe 103, wherein the working distance of the microscope objective 104 is greater than the distance between the center of the flow pipe 103 and the outer surface of the flow pipe 103, and the numerical aperture (NA) of the microscope objective 104 is determined based on a predetermined resolution and the wavelength of the light beam.

[0055] For example, the microscope objective lens 104 may include one or more lenses. Optionally, the microscope objective lens 104 may be an infinite objective lens, and the light passing through the infinite objective lens is emitted to infinity as a parallel beam. Applying the infinite objective lens to the microscope objective lens 104 can further reduce the influence of aberration on observation.

[0056] The tube lens 105 may be located between the microscope objective lens 104 and the image plane, and may be configured to magnify the image of the particles together with the microscope objective lens 104, and converge the magnified image of the particles on the image plane. That is, the microscope objective lens 104 and the tube lens 105 have a magnification function together, and the magnification factor may be determined based on the focal lengths of the microscope objective lens 104 and the tube lens 105.

[0057] For example, the tube lens 105 may include one or more lenses. The tube lens 105 may gather light emitted from the infinite distance objective lens and form an image on a finite distance image plane. The magnification of the image may be adjusted by adjusting the focal length of the tube lens 105.

[0058] The camera 106 is disposed on the image plane and is configured to acquire magnified images of the particles on the image plane.

[0059] As above reference Figure 1 The flow cytometry microscopic imaging device 100 according to the embodiment of the present disclosure uses a microscopic system to perform microscopic imaging of cells in the flow pipe, and continuously shoots the liquid in the flow pipe through a camera. Due to different cell concentrations, the captured image may appear to be an empty image without particles or contain multiple particles. At this time, the processing unit needs to use an image processing algorithm to identify and cut out single particles in the image containing particles obtained by the flow cytometry microscopic imaging device, so as to facilitate subsequent tasks such as classification or recognition based on single cell images.

[0060] Therefore, the flow cytometry microscopic imaging device 100 according to the embodiment of the present disclosure may further include a processing unit (not shown) for cutting an image containing multiple particles to obtain an image containing only a single particle. The processing unit may be implemented in hardware, firmware, and / or a combination of hardware and software. For example, the processing unit may have the following features: Fig. 9 The processing unit can be used with Figure 1 The flow cytometry imaging device shown is integrated and can also be used with Figure 1 The flow cytometry microscopy imaging device shown is physically separated and communicates via a wired or wireless connection. For example, the processing unit can be a single server / server cluster or a single / multiple computer devices.

[0061] The images including single particles acquired and processed by the flow cytometric microscopic imaging device 100 usually contain some impurities, broken cells or cells that are difficult to distinguish in clusters, which will affect the final cell identification and judgment.

[0062] Therefore, after obtaining a single particle image that only includes a single particle, the embodiment of the present disclosure proposes a pre-classification model based on deep learning to achieve the distinction and screening of impurities, broken cells, and indistinguishable cells from clusters and intact cervical cells. Because doctors mainly use the morphological characteristics of cells as the basis for cell classification when identifying the types of cervical exfoliated cells. Therefore, the embodiment of the present disclosure further proposes to automatically identify and segment different areas of cervical cells based on image segmentation, and extract morphological characteristic parameters of cervical cells based on each segmented area, so as to achieve rapid identification and differentiation of cell types in cervical exfoliated cells, instead of requiring identification and differentiation based on characteristic parameters of the entire image as in traditional cell classification models.

[0063] Figure 2 FIG. 2 shows a system 200 for classifying cervical cell images according to an embodiment of the present disclosure. Figure 2 As shown, the system 200 for classifying cervical cell images includes a flow microscopy imaging device 201, a screening device 202, an image segmentation device 203, and a classification device 204. As those skilled in the art should understand, in addition to the above components, the system 200 may also include other components or devices, which are not limited by the present disclosure.

[0064] For example, the flow microscopy imaging device 201 here is the same as that described above. Figure 1 An example of a flow cytometry microscopy imaging apparatus 100 is described.

[0065] The flow microscopic imaging device 201 is configured to collect images of particles in the liquid sample to be detected including exfoliated cervical cells, and obtain one or more single particle images including only single particles.

[0066] As an example, Figure 3 As shown, using the flow microscopic imaging device 201 to collect images of particles in the liquid sample to be detected including exfoliated cervical cells, and obtaining one or more single particle images including only single particles may include the following steps:

[0067] Step S301 : acquiring an image including the particles in response to the particles passing through the flow channel of the flow microscopy imaging device.

[0068] As described above, despite the cell processing and other operations, it is inevitable that some impurities, broken cells or cells that are difficult to distinguish in a cluster will be captured when the cell image is acquired using the flow microscopy imaging device 201, affecting the final cell judgment. For example, impurities, broken cells, clustered cells and single intact cervical cells are collectively referred to as particles.

[0069] In order to accurately determine the cell type later, the following steps S302-S303 are continued to implement the positioning and segmentation of particles in the image including particles preliminarily acquired by the flow microscopy imaging device 201. For example, steps S302-S303 can be performed by the processing unit integrated with or separated from the flow microscopy imaging device described above.

[0070] Step S302: determining the location of one or more particle regions in the image including the particles.

[0071] For example, determining the location of one or more particle regions in an image including particles may include: in response to no particles passing through a flow tube of a flow microscopy imaging device, acquiring an image that does not include particles; subtracting pixel values ​​of the image including particles from those of the image not including particles pixel by pixel to obtain a residual image; binarizing the residual image to obtain a binarized residual image; determining a region with a pixel value of 1 in the binarized residual image as the location of the one or more particle regions in the residual image; and determining a pixel region in the image including particles that corresponds to the location of the one or more particle regions in the residual image as the location of the one or more particle regions in the image including particles.

[0072] Specifically, performing binarization processing on the residual image to obtain a binarized residual image may include: performing binarization processing on the residual image, comparing each pixel in the residual image with a predefined binarization threshold, and when the pixel value is greater than or equal to the binarization threshold, setting the corresponding pixel value in the residual image to 1, and when the pixel value is less than the binarization threshold, setting the corresponding pixel value to 0. The binarization threshold here can be reasonably set based on actual conditions.

[0073] Furthermore, since the binary residual image may have problems such as noise and inaccurate cell regions, before determining the region with a pixel value of 1 in the binary residual image as the location of one or more particle regions in the residual image, the binary residual image can also be subjected to morphological processing to filter out the noise in the background and then fill the foreground region. For example, the morphological processing here is an erosion and dilation algorithm, in which erosion performs a "contraction" or "thinning" operation on the basis of the binary image, and dilation performs a "lengthening" or "thickening" operation on the basis of the binary image. This can eliminate noise in the background and fill the holes in the target foreground.

[0074] Step S303: based on the positioning of the one or more particle regions, one or more single particle images including only a single particle are cut out from the image including the particles.

[0075] After the regional position of each particle is determined, the particle is cut out from the image including the particle.

[0076] For the convenience of description, the region including the particle cut out from the image including the particle is defined as the cut region. It should be understood that in order not to destroy the integrity of the particle, the cut region needs to be larger than the region of the particle.

[0077] For example, the cutting area may be a rectangular area. Specifically, based on the position of the particle area in the image including the particle, the minimum rectangular area surrounding the particle area is determined in the image including the particle. The minimum rectangle uses the vertical distance between the two longitudinal end points of the particle as one side length of the rectangle, and uses the vertical distance between the two transverse end points of the particle as the second side length, thereby ensuring that the minimum rectangle can encompass the area where the particle is located. The minimum rectangular area containing the particle is segmented from the image including the particle, thereby obtaining the cutting area defined above.

[0078] For example, to ensure redundancy of the segmented area, the cutting area can also be a rectangular area slightly larger than the above minimum rectangular area. Specifically, after obtaining the above minimum rectangular area, the four sides of the minimum rectangular area can be expanded by several pixels (for example, 5 pixels) to the surrounding area, and the expanded area can be segmented from the image including the particles as the cutting area.

[0079] For example, the cutting region may also be a pre-set region with a fixed side length (e.g., a fixed length and width) that is larger than all particle regions. Specifically, after the particle region is determined, the particle region is taken as the center in the image including the particles and extended to the fixed side length in all directions, and the extended region is segmented from the image including the particles as the cutting region. It should be understood that all segmented images obtained in this way have a fixed size.

[0080] Those skilled in the art should understand that in order to ensure the integrity of the segmented cell region, other regions of appropriate size may also be set as cutting regions, and the present disclosure does not limit this.

[0081] Subsequently, the cut-out area can be input into a subsequent device as a single particle image including only a single particle to achieve final cervical cell classification.

[0082] The single particle image after cutting may include an image of a single impurity, an image of a single clumped cervical cell, an image of a single broken cervical cell, and an image of a single intact cervical cell. These impurities, broken cells, or clumped cells will affect the final cell judgment. Therefore, the present disclosure proposes to use a screening device in advance to screen out images of these impurities, broken cells, or clumped cells that affect cell judgment.

[0083] The system 200 for classifying cervical cell images according to the embodiment of the present disclosure utilizes the screening device 202 to remove these impurities and only retain the cervical cell images with complete morphology.

[0084] For example, the screening device 202 may pre-classify the acquired one or more single particle images using a pre-classification model based on a first-level neural network, so as to screen out complete cervical cell images from the one or more single particle images.

[0085] For example, the first-level neural network here is a pre-trained neural network-based binary classification model, which is trained to classify complete cervical cell images and impurity images, damaged cell images and / or clumped cell images, and screen out complete cervical cell images.

[0086] As a non-limiting example, the first-level neural network here can be a neural network structure based on MobileNet_v2. For example, when the inventors of the present disclosure train the neural network based on MobileNet_v2 to pre-classify single particle images, the loss function is set to the cross entropy loss function, the optimizer selects Adam, and the learning rate is set to 0.000001. The input of the network is a normalized image, and the final output layer of the network contains only two neurons, that is, this is a binary classification neural network. The complete cervical cell image and the impurity, clumped or broken cell image are manually selected in advance as the data set of the neural network, and the network model is continuously trained. Then the trained model is used for the classification of unlabeled new images, and the errors are manually corrected in the classification results. The corrected data set is then used as a supplement to the training data set, and the training-classification-supplementation data set is repeated. The final data set reaches more than 20,000, and the model achieves a classification accuracy of 96.7% on the unfamiliar data set.

[0087] In this way, the system 200 for classifying cervical cell images of the embodiment of the present disclosure can use the screening device 202 to pre-classify the single particle image obtained after shooting and cutting by the flow microscopy imaging device 201, obtain the cervical cell image and the impurity image, eliminate the impurity image, and retain the cervical cell image for the next step.

[0088] In clinical practice, doctors mainly classify the types of exfoliated cervical cells based on the morphological characteristics of the cells. Therefore, the present disclosure proposes to use an image segmentation network to segment the cell region and the cell nucleus region, and then calculate the feature quantity representing the cell morphological characteristics based on these regions, and classify the cells based on this.

[0089] The image segmentation device 203 performs image segmentation on the cervical cell image using an image segmentation model based on a second-level neural network, so as to segment the cervical cell image into a first region, a second region and a third region.

[0090] Image segmentation in this article refers to the machine automatically segmenting object areas from images and identifying the content therein. Image segmentation in this article is classification at the pixel level, and fine-grained reasoning is achieved by densely predicting and inferring each pixel, so that each pixel is labeled with the category of the closed object area in which it is located. Pixels belonging to the same category are classified into one category, so image segmentation understands the image at the pixel level. Simply put, the goal of segmentation is generally to take an RGB image (height × width × 3) or a grayscale image (height × width × 1) as input, and the output is a segmentation map, in which each pixel contains the label of its category (height × width × 1), and the resolution of the segmentation map matches the resolution of the original input.

[0091] For example, the second-level neural network here is a pre-trained neural network-based image segmentation model.

[0092] The UNet-based neural network model is one of the mainstream image segmentation models today. Figure 4 Figure 2 shows a schematic diagram of the UNet network architecture. Figure 4It can be seen that the two main features of the UNet network structure are: U-shaped network structure and skip layer connection. The UNet network structure is a symmetrical network structure, which includes two paths on the left and right. The path on the left can be regarded as an encoder, or it can be called an upsampling processing path. It includes five convolution submodules, each of which includes two convolution layers and a ReLU layer. The convolution layer structure here is unified as a 3×3 convolution kernel. After each submodule, there is a downsampling layer implemented by maximum pooling. The convolution submodule is used to extract features, and the maximum pooling layer is used to reduce the dimension. The resolution of the output feature image after each maximum pooling layer is halved. The feature map output by the last convolution submodule is not subjected to maximum pooling and is directly input to the decoder on the right. The path on the right can be regarded as a decoder, or it can be called a downsampling processing path. It includes a structure that is basically symmetrical with the encoder. It performs 3×3 convolution and upsampling on the input feature map to gradually repair the details and spatial dimensions of the object. In addition, feature fusion is also used in the network, such as Figure 4 As shown by the dotted arrows in the figure, the features of the previous down-sampling network are spliced ​​and fused with the features of the subsequent up-sampling through skip-layer connections to obtain more accurate context information and achieve better segmentation results. The image segmentation model finally outputs a segmentation map of the target object, in which the pixel value of each pixel can be a label representing its category.

[0093] This paper takes UNet as an example to implement the segmentation of cell regions. Of course, in addition to UNet, any suitable image segmentation architecture can also be used to segment the single cell image of the embodiment of the present disclosure, such as linknet, pspnet, and deeplab v3, and these image segmentation architectures can also be adjusted according to actual conditions.

[0094] For example, using the image segmentation model based on the second-level neural network to perform image segmentation on the cervical cell image to segment the cervical cell image into a first region, a second region, and a third region includes: identifying and marking each pixel in the first region of the cervical cell image as a first region pixel, identifying and marking each pixel in the second region of the cervical cell image as a second region pixel; and identifying and marking each pixel in the third region of the cervical cell image as a third region pixel.

[0095] For example, the first area is the background area of ​​the cervical cell image, the second area is the cell area of ​​the cervical cell image, and the third area is the cell nucleus area of ​​the cervical cell image.

[0096] For example, in the case of three-classification, the image segmentation model divides the cervical cell image into the cell nucleus region, the cytoplasm region and the background region in each processing. In this way, the cell region is the sum of the cells in the cell nucleus region and the cytoplasm region.

[0097] For example, two binary classification processes may be used, wherein in one process, the cervical cell image is segmented into a cell nucleus region and a background region, and in a second process, the cervical cell image is segmented into a cell region and a background region. In this way, the cell nucleus region, the cell region, and the background region may be identified and marked based on the two binary classification processes.

[0098] Figure 5 The figure shows the effect of three-class segmentation of cervical cells based on the image segmentation model. Figure 5 As shown, the smallest closed circle is the cell nucleus area, the annular area surrounding the closed circle is the cytoplasm area, and the rest is the background area.

[0099] Cervical exfoliated cells can be roughly divided into normal cells (including upper surface cells, middle surface cells, basal cells, etc.), low-grade diseased cells, high-grade diseased cells, keratinized diseased cells and other atypical cells (white blood cells that appear during inflammation, immature cells, etc.). Normal cells have a large cell area and a small ratio of nuclear area to cell area. If cancer occurs, the nucleus will become larger, the cell will shrink, and the ratio of nuclear to cell area will increase. The more severe the lesion, the more obvious it is. Keratinocytes will show cell keratinization and irregular shapes. Based on this classification principle, the present disclosure proposes to extract characteristic parameters that can characterize cells based on the morphological features of cervical cells to perform cell classification, without the need to identify and distinguish based on all features of the entire image as in traditional cell classification models.

[0100] Based on this, the system 200 for classifying cervical cell images of the embodiment of the present disclosure uses a classification device 204 to determine multiple feature parameters characterizing the morphological characteristics of cervical cells according to the segmented areas, and based on the multiple feature parameters, uses a classification model based on a third-level neural network to identify the category of cervical cells.

[0101] For example, after the image segmentation device 203 segments the cell region and the cell nucleus region in the cervical cell image, characteristic parameters that can characterize the cells are calculated based on the segmentation results, and these parameters are combined to form a characteristic vector used as data for cell classification.

[0102] For example, the plurality of characteristic parameters include parameters related to the morphological characteristics of the cell region, including parameters indicating the cell area A cell The first parameter, indicating the cell diameter D cell The second parameter, indicating the cell perimeter Ccell The third parameter, indicating cell roundness O cell The fourth parameter; the plurality of characteristic parameters also include parameters about the morphological characteristics of the cell nucleus region, including an indicator of the cell nucleus area A nucleus The fifth parameter, indicating the diameter of the cell nucleus D nucleus The sixth parameter, indicating the cell nucleus circumference C nucleus The seventh parameter indicates the roundness of the nucleus. nucleus The eighth parameter of

[0103] For example, based on the pixels of different regions identified and labeled, the eight parameters characterizing the morphological features of the cell regions above can be calculated as follows:

[0104] Cell area A cell : The sum of the number of pixels in the cell area;

[0105] Core area A nucleus : The sum of the number of pixels in the nuclear area;

[0106] Cell diameter D cell : Maximum width difference of cell area;

[0107] Core diameter D nucleus : Maximum width difference of the nuclear region;

[0108] Cell perimeter C cell : The sum of the number of boundary pixels in the cell area;

[0109] Nuclear circumference C nucleus : The sum of the number of boundary pixels in the nuclear area;

[0110] Cell roundness O cell : The roundness of the cell area

[0111] Nuclear roundness O nucleus : The roundness of the nuclear region

[0112] Figure 6 (a) shows an exemplary structure of the third-level neural network of the embodiment of the present disclosure. Figure 6 As shown in (a), an exemplary structure of the third-level neural network is a fully connected neural network structure, which includes an input layer, three hidden layers and an output layer from left to right, and a fully connected structure is used between each layer. The input layer has 8 neurons, representing the 8 feature parameters calculated above, the middle 3 hidden layers, the first hidden layer has 16 neurons, the second hidden layer has 32 neurons, the third hidden layer has 10 neurons, and the final output layer has 5 neurons, representing 5 types of cell classifications.

[0113] After designing the network structure, the types of cells are manually marked and the characteristic parameters of the extracted cervical cells are used as training labels and data sets. For example, each data in the data set is an 8-dimensional vector obtained by sequentially concatenating the above 8 characteristic parameters extracted from a cervical cell image. During training, these 8 parameters are input in parallel to the 8 neurons in the input layer as input data.

[0114] Specifically, the third-level neural network training process is as follows: first, input the data set in the input layer, then the neural network performs forward propagation in the hidden layer, and finally calculates the output of the output layer, and then calculates the pre-defined loss, then performs error back propagation, and uses the pre-set optimization method to update the parameters in the network, such as weight parameters and threshold parameters. Then repeat the above iterations, and stop training after reaching the maximum number of iterations (num_epoch) or the loss value meets a certain condition. Then input some data that is not in the training set as a test set for verification. After multiple training verifications and tests, the precision and recall rates on the unfamiliar data set finally meet the pre-set thresholds, so that a third-level neural network trained with a large amount of data can be obtained.

[0115] For example, here, the loss between the classification prediction value of the final output of the third-level neural network and the true label can be calculated based on the cross entropy loss function. For example, the optimization constraint algorithm can use the random steepest descent method to optimize the network parameters. For example, the learning rate can be set to 0.0005. It should be understood that any other suitable network parameters can also be set, and the present disclosure is not limited here.

[0116] Finally, the saved trained third-level neural network model can be used to classify any new cervical cell characteristic parameters. Figure 6 As shown in (b), five types of cell images are identified after classifying cervical exfoliated cell sample images based on the trained third-level neural network.

[0117] Different from the traditional image-based cell classification method, the disclosed embodiment proposes to extract feature vectors for classification model recognition based on the morphological features of cervical cells obtained after image segmentation. The extracted feature vector representing a cervical cell only includes a few (for example, eight) parameters. Since the cells contained in the cervical exfoliated cell sample are often in the tens of thousands, this method greatly reduces the speed of cell classification and recognition of the sample to be tested, and also greatly reduces the computer resources for operating the classification model.

[0118] Different from the traditional microscope shooting method, the system for classifying cervical cell images according to the embodiment of the present application uses a flow microscopy imaging device to collect cervical exfoliated cell images, which can realize rapid shooting and cutting of cell images, thereby obtaining a single particle image including only a single particle. After obtaining the single particle image including only a single particle, a pre-classification mechanism and an image segmentation mechanism based on deep learning are further proposed to achieve the filtering of impurities and the screening of complete cervical cells and the segmentation of the image area of ​​cervical cells, so as to obtain cell morphological characteristic parameters based on the segmented area and realize the rapid identification and differentiation of cervical cell types.

[0119] The embodiment of the present disclosure also provides an electronic device for classifying cervical cells. Figure 7 A schematic block diagram of an electronic device for classifying cervical cells according to an embodiment of the present disclosure is shown.

[0120] For example, Figure 7 As shown, the electronic device 700 for classifying cervical cells may include one or more processors 701 and one or more memories 702. The one or more memories 702 store computer executable instructions, and when the computer executable instructions are executed by the one or more processors 701, the following steps S801-S804 (such as Figure 8 shown):

[0121] S801, enabling a flow microscopic imaging device to collect images of particles in a liquid sample to be detected including exfoliated cervical cells, and obtain one or more single particle images including only single particles.

[0122] S802: Pre-classify the acquired one or more single particle images using a pre-classification model based on a first-level neural network to filter out cervical cell images from the one or more single particle images.

[0123] S803, performing image segmentation on the cervical cell image using an image segmentation model based on a second-level neural network, so as to segment the cervical cell image into a first region, a second region and a third region.

[0124] S804, determining a plurality of characteristic parameters characterizing the morphological characteristics of the cervical cells according to the segmented regions, and identifying the categories of the cervical cells based on the plurality of characteristic parameters using a classification model based on a third-level neural network.

[0125] The one or more memories 702 and the one or more processors 701 may be interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0126] For example, one or more memories 702 and one or more processors 701 may be disposed in a single machine, on a server, or in the cloud, to execute one or more steps performed by the system for classifying cervical cell images as described above.

[0127] For example, the one or more processors 701 may be a central processing unit (CPU), a digital signal processor (DSP), or other forms of processing units with data processing capabilities and / or program execution capabilities, such as a field programmable gate array (FPGA), etc.; for example, the central processing unit (CPU) may be an X86 or ARM architecture, etc. The one or more processors 701 may be a general-purpose processor or a dedicated processor, and may control the components in the system 200 for classifying cervical cells to perform desired functions.

[0128] For example, one or more memories 702 may include any combination of one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, erasable programmable read-only memory (EPROM), portable compact disk read-only memory (CD-ROM), USB memory, flash memory, etc.

[0129] In addition, the electronic device according to the embodiment of the present application can be Fig. 9 The architecture 900 shown in FIG. Fig. 9 As shown, the architecture 900 may include a bus 901, one or more CPUs 902, a read-only memory (ROM) 903, a random access memory (RAM) 904, a communication port 905 connected to a network, an input / output component 906, a hard disk 907, etc. The storage device in the architecture 900, such as the ROM 903 or the hard disk 907, may store various data or files used for processing and / or communication of the method provided in the present application and program instructions executed by the CPU. The architecture 900 may also include a user interface. Of course, Fig. 9 The architecture shown is only exemplary. When implementing different devices, it can be omitted or added according to actual needs. Fig. 9 One or at least two components of the architecture shown.

[0130] An embodiment of the present disclosure also provides a computer-readable storage medium. Fig.10 1000 is a schematic diagram of a storage medium according to an embodiment of the present disclosure. Fig.10As shown, the computer-readable storage medium 1002 stores computer-executable instructions 1001. When the computer-executable instructions 1001 are executed by the processor, the functions performed by the system for classifying cervical cells described above can be executed. The computer-readable storage medium includes, but is not limited to, for example, volatile memory and / or non-volatile memory. The volatile memory may include, for example, a random access memory (RAM) and / or a cache memory (cache), etc. The non-volatile memory may include, for example, a read-only memory (ROM), a hard disk, a flash memory, etc.

[0131] The embodiments of the present disclosure also provide a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device controls the components in the system 200 for classifying cervical cells to perform desired functions.

[0132] Those skilled in the art will appreciate that the contents disclosed in this disclosure may be subject to various modifications and improvements. For example, the various devices or components described above may be implemented by hardware, or by software, firmware, or a combination of some or all of the three.

[0133] In addition, although the present disclosure makes various references to certain units in the system according to embodiments of the present disclosure, any number of different units may be used and run on the client and / or server. The units are illustrative only, and different aspects of the systems and methods may use different units.

[0134] Those skilled in the art will appreciate that all or part of the steps in the above method can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk. Optionally, all or part of the steps in the above embodiment can also be implemented using one or more integrated circuits. Accordingly, each module / unit in the above embodiment can be implemented in the form of hardware or in the form of a software functional module. The present disclosure is not limited to any particular form of combination of hardware and software.

[0135] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present disclosure belongs. It should also be understood that terms such as those defined in common dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology and should not be interpreted in an idealized or highly formal sense, unless explicitly defined as such herein.

[0136] The above is an explanation of the present disclosure and should not be considered as a limitation thereof. Although several exemplary embodiments of the present disclosure have been described, it will be readily understood by those skilled in the art that many modifications may be made to the exemplary embodiments without departing from the novel teachings and advantages of the present disclosure. Therefore, all such modifications are intended to be included within the scope of the present disclosure as defined in the claims. It should be understood that the above is an explanation of the present disclosure and should not be considered as being limited to the specific embodiments disclosed, and modifications to the disclosed embodiments and other embodiments are intended to be included within the scope of the appended claims. The present disclosure is defined by the claims and their equivalents.

Claims

1. A system for classifying cervical cell images, comprising: A flow microscopic imaging device is used to collect images of particles in a liquid sample to be detected including exfoliated cervical cells, and obtain one or more single particle images including only single particles; A screening device, which pre-classifies the acquired one or more single particle images using a pre-classification model based on a first-level neural network, so as to screen out a cervical cell image from the one or more single particle images; An image segmentation device, which performs image segmentation on the cervical cell image using an image segmentation model based on a second-level neural network, so as to segment the cervical cell image into a first region, a second region and a third region; as well as a classification device, which determines a plurality of characteristic parameters characterizing the morphological characteristics of the cervical cells according to the segmented areas, and identifies the category of the cervical cells based on the plurality of characteristic parameters by using a classification model based on a third-level neural network, The flow microscopy imaging device is configured as follows: acquiring an image including the microparticle in response to the microparticle passing through a flow channel of the flow microscopy imaging device; determining the location of one or more particle regions in the image including the particles; as well as Based on the location of the one or more particle regions, one or more single particle images including only single particles are cut out from the image including the particles, The image segmentation device is configured as follows: Identify and mark each pixel in a first area of ​​the cervical cell image as a first area pixel, wherein the first area is a background area of ​​the cervical cell image, Identify and mark each pixel in a second area of ​​the cervical cell image as a second area pixel, the second area being a cell area of ​​the cervical cell image; as well as Each pixel in a third region of the cervical cell image is identified and marked as a third region pixel, and the third region is a cell nucleus region of the cervical cell image.

2. The system according to claim 1, wherein: The liquid sample to be detected is obtained by the following steps: Centrifuge the raw liquid to be processed to obtain a plurality of particles in the raw liquid to be processed; removing the supernatant of the plurality of microparticles obtained by the centrifugation operation, and resuspending the plurality of microparticles from which the supernatant has been removed in a microparticle fixing solution; as well as The resuspended cell fixative is pipetted to obtain a liquid sample to be detected, wherein all particles in the liquid sample to be detected are in a single particle state.

3. The system according to claim 1, wherein: The flow microscopic imaging device comprises a light source, a condenser lens group, a flow pipe, a microscope objective lens, a tube lens, and a camera. The flow microscopic imaging device controls the flow of the liquid sample to be detected to control the particles in the liquid sample to be detected to flow stably through the center of the flow channel; The light source is configured to emit a beam of visible light; The condenser lens group is configured to condense the light beam and irradiate the light beam uniformly on the flow pipe; The microscope objective lens adopts an infinite objective lens, focuses on the center of the flow channel, and images the particles flowing through; The tube lens focuses the light emitted from the infinite objective lens to form an image, and the magnification of the image can be adjusted by adjusting the focal length of the tube lens; and The camera is located at the rear focal plane of the tube lens and photographs the magnified image to obtain an image including particles.

4. The system according to claim 1, wherein: Determining the location of one or more particle regions in the image including the particles comprises: In response to the absence of microparticles passing through the flow conduit of the flow microscopy imaging device, acquiring an image that does not include microparticles; Subtracting pixel values ​​of the image including particles from pixel values ​​of the image not including particles pixel by pixel to obtain a residual image; Binarizing the residual image to obtain a binarized residual image; Determine the area with a pixel value of 1 in the binarized residual image as the location of the one or more particle areas in the residual image; and A pixel region in the particle-including image corresponding to the location of the one or more particle regions in the residual image is determined as the location of the one or more particle regions in the particle-including image.

5. The system according to claim 1, wherein: Based on the positioning of the one or more particle regions, cutting out one or more single particle images including only a single particle from the image including the particles comprises: Determining one or more minimum rectangular regions surrounding the one or more particle regions in the particle-containing image based on the positioning of the one or more particle regions in the particle-containing image, and cutting out the one or more minimum rectangular regions from the particle-containing image as the one or more single particle images; or Extending the one or more minimum rectangular areas by a predetermined number of pixels in all directions, and cutting out one or more extended rectangular areas from the image including the particle as the one or more single particle images; or Taking each particle region of the one or more particles as the center, a region larger than each particle region and having a fixed size is cut out from the image including the particles to serve as the one or more single particle images.

6. The system according to any one of claims 1 to 3, wherein: The first-level neural network is a pre-trained binary classification model based on a neural network, which is trained to classify complete cervical cell images from impurity images and / or damaged cell images and / or clumped cell images, and screen out complete cervical cell images.

7. The system according to claim 1, wherein: The neural network-based image segmentation model is a neural network model based on Unet.

8. The system according to any one of claims 1 to 3, wherein: The third-level neural network is a pre-trained fully connected neural network, the number of neurons in the input layer of the third-level neural network is equal to the number of feature parameters extracted based on the image segmentation, the number of neurons in the output layer of the fully connected neural network is equal to the number of cell categories, and the third-level neural network is trained to identify the categories of cervical cells based on the feature parameters extracted based on the image segmentation.

9. The system according to claim 8, wherein: The classification device determines a plurality of characteristic parameters characterizing the morphological characteristics of the cervical cells according to the segmented areas, and based on the plurality of characteristic parameters, uses a classification model based on a third-level neural network to identify the category of the cervical cells, including: Calculating a plurality of characteristic parameters characterizing the cervical cell image based on the second region and the third region after the image segmentation of the cervical cell image; constructing a feature vector of the cervical cell image based on the multiple feature parameters; The feature vectors of the plurality of cervical cell images are input into the third-level neural network to identify the category of the cervical cells.

10. The system according to claim 9, wherein: The types of cervical cells include one or more of normal cells, low-grade lesion cells, high-grade lesion cells, keratinized lesion cells and atypical cells.

11. The system according to claim 9, wherein: The multiple characteristic parameters include parameters regarding the morphological characteristics of the second region, including a first parameter indicating cell area, a second parameter indicating cell diameter, a third parameter indicating cell circumference, and a fourth parameter indicating cell roundness; the multiple characteristic parameters also include parameters regarding the morphological characteristics of the third region, including a fifth parameter indicating cell nuclear area, a sixth parameter indicating cell nuclear diameter, a seventh parameter indicating cell nuclear circumference, and an eighth parameter indicating cell nuclear roundness.

12. The system according to claim 11, wherein: Based on the second region and the third region after the image segmentation of the cervical cell image, calculating a plurality of characteristic parameters characterizing the cervical cell image includes: Calculating the total number of all pixels in the second area as the first parameter; Calculate the distance between two pixels in the second area that are farthest apart from each other, and use it as the second parameter; Calculating the total number of second area pixels on the boundary of the second area as the third parameter; Calculate the ratio of the first parameter to the second parameter as the fourth parameter; Calculating the total number of pixels in all third regions as the fifth parameter; Calculating the distance between two pixels in the third region that are farthest apart from each other as the sixth parameter; Calculating the total number of third area pixels on the boundary of the third area as the seventh parameter; and The ratio of the fifth parameter to the sixth parameter is calculated as the eighth parameter.

13. An electronic device comprising a memory and a processor, wherein: The memory stores a program code readable by a processor. When the processor executes the program code, the following steps are performed: The flow microscopic imaging device is used to collect images of particles in the liquid sample to be detected including the exfoliated cervical cells, and obtain one or more single particle images including only single particles; Pre-classifying the acquired one or more single particle images using a pre-classification model based on a first-level neural network to filter out a cervical cell image from the one or more single particle images; Performing image segmentation on the cervical cell image using an image segmentation model based on a second-level neural network to segment the cervical cell image into a first region, a second region, and a third region; as well as Determine a plurality of characteristic parameters characterizing the morphological characteristics of the cervical cells according to the segmented regions, and identify the category of the cervical cells based on the plurality of characteristic parameters using a classification model based on a third-level neural network, The method of using a flow microscopy imaging device to collect images of particles in a liquid sample to be detected including exfoliated cervical cells and obtaining one or more single particle images including only single particles includes: acquiring an image including the microparticle in response to the microparticle passing through a flow channel of the flow microscopy imaging device; determining the location of one or more particle regions in the image including the particles; and Based on the location of the one or more particle regions, cutting out one or more single particle images including only single particles from the image including the particles, and The second-level neural network is a pre-trained neural network-based image segmentation model, wherein the image segmentation of the cervical cell image is performed using the image segmentation model based on the second-level neural network to segment the cervical cell image into a first area, a second area, and a third area, including: Identify and mark each pixel in a first area of ​​the cervical cell image as a first area pixel, wherein the first area is a background area of ​​the cervical cell image, Identify and mark each pixel in a second region of the cervical cell image as a second region pixel, the second region being a cell region of the cervical cell image; and Each pixel in a third region of the cervical cell image is identified and marked as a third region pixel, and the third region is a cell nucleus region of the cervical cell image.

14. A computer-readable storage medium having computer-executable instructions stored thereon, the computer-executable instructions being used to perform the following steps: The flow microscopic imaging device is used to collect images of particles in the liquid sample to be detected including the exfoliated cervical cells, and obtain one or more single particle images including only single particles; Pre-classifying the acquired one or more single particle images using a pre-classification model based on a first-level neural network to filter out cervical cell images from the one or more single particle images; Performing image segmentation on the cervical cell image using an image segmentation model based on a second-level neural network to segment the cervical cell image into a first region, a second region, and a third region; as well as Determine a plurality of characteristic parameters characterizing the morphological characteristics of the cervical cells according to the segmented regions, and identify the category of the cervical cells based on the plurality of characteristic parameters using a classification model based on a third-level neural network, The method of using a flow microscopy imaging device to collect images of particles in a liquid sample to be detected including exfoliated cervical cells and obtaining one or more single particle images including only single particles includes: acquiring an image including the microparticle in response to the microparticle passing through a flow channel of the flow microscopy imaging device; determining the location of one or more particle regions in the image including the particles; and Based on the location of the one or more particle regions, cutting out one or more single particle images including only single particles from the image including the particles, and The second-level neural network is a pre-trained neural network-based image segmentation model, wherein the image segmentation of the cervical cell image is performed using the image segmentation model based on the second-level neural network to segment the cervical cell image into a first area, a second area, and a third area, including: Identify and mark each pixel in a first area of ​​the cervical cell image as a first area pixel, wherein the first area is a background area of ​​the cervical cell image, Identify and mark each pixel in a second region of the cervical cell image as a second region pixel, the second region being a cell region of the cervical cell image; and Each pixel in a third region of the cervical cell image is identified and marked as a third region pixel, and the third region is a cell nucleus region of the cervical cell image.

Images