Methods and related products for detecting living cell morphology based on deep neural networks

Abstract

The present invention relates to a method for detecting cell morphology and related products, wherein the method comprises: identifying and locating a captured image containing cells to be detected to extract a single-cell image; segmenting the single-cell image to obtain characteristic regions of the single cell; and determining morphological parameters of the single cell based on the characteristic regions. The method of the present invention enables non-destructive, accurate, and rapid detection of cell morphology, facilitating clinical applications and research of the detected cells.

Description

Technical Field

[0001] The present invention generally relates to the field of neural network technology. More specifically, the present invention relates to a method for detecting the morphology of living cells based on a deep neural network and related products. Background Art

[0002] Cell morphology detection is of great significance for evaluating cell status and cell quality. However, current methods for cell morphology detection are mostly based on images of fixed and stained cells, and cannot achieve online detection of living cells, especially active living cells.

[0003] Taking sperm cells as an example, the current conventional method for sperm morphology testing involves pre-processing a semen sample through centrifugation, smearing, and staining. The sperm smear is then placed under a microscope for manual observation and classification based on the examiner's experience, or using computer-assisted classification techniques to classify the sperm morphology. However, the fixation and staining process and methods can affect the morphological structure of sperm, potentially affecting the accuracy of morphological testing results. Furthermore, pre-processing sperm cells through smearing and staining can disrupt sperm physiological functions and DNA, causing sperm inactivation and rendering the tested sperm unusable in clinical practice. Therefore, these methods have significant application limitations. For example, in vitro fertilization (IVF) requires the use of live sperm cells. Due to the lack of methods for live cell morphology testing, the screening of live sperm required for IVF still relies on manual labor by clinical personnel. This process is highly dependent on clinical experience, resulting in high subjectivity, inconsistent standards, and low efficiency. Therefore, achieving non-destructive, rapid, and accurate testing of live cell morphology is a pressing technical challenge. Summary of the Invention

[0004] In view of the technical problems mentioned above, the technical solution of the present invention provides a method, device, equipment, system and computer storage medium for performing living cell morphology detection based on deep neural network in multiple aspects.

[0005] In a first aspect of the present invention, a method for detecting the morphology of living cells based on a deep neural network is provided, comprising: using a target detection model based on a deep neural network to identify and locate collected images to be detected containing living cells, so as to extract living single-cell images; using a cell segmentation model based on a deep neural network to segment the living single-cell images, so as to obtain characteristic parts of the living single cells; and analyzing and determining the morphological parameters of the living single cells based on the characteristic parts.

[0006] According to one embodiment of the present invention, before using the target detection model to identify and locate the image to be detected, the method also includes: obtaining a large sample of living cell images and performing a first labeling on individual cells in the living cell images; and using the living cell images with the first labeling to train a first deep neural network model to obtain the target detection model.

[0007] According to another embodiment of the present invention, before using the cell segmentation model to segment the living single-cell image, the method also includes: performing a second annotation on the characteristic parts of the single cells in the acquired living cell image; and training a second deep neural network model using the living cell image with the second annotation to obtain the cell segmentation model.

[0008] According to another embodiment of the present invention, when training the first deep neural network model, the method includes performing image data enhancement processing on the living cell image, wherein the image data enhancement processing includes at least one of spatial transformation processing, scaling processing, and image brightness adjustment processing on the image.

[0009] According to one embodiment of the present invention, when training the second deep neural network model, the method includes performing image data enhancement processing on the living cell image, wherein the image data enhancement processing includes at least one of spatial transformation processing, scaling processing, and image brightness adjustment processing on the image.

[0010] According to another embodiment of the present invention, the output part of the cell segmentation model adopts a single-branch multi-class segmentation structure or a multi-branch single-class segmentation structure.

[0011] According to yet another embodiment of the present invention, the living cell includes a living sperm, and the characteristic portion includes at least one of a sperm head, a vacuole, a midpiece, and a tail.

[0012] According to one embodiment of the present invention, before using the cell segmentation model to segment the living single-cell image, the method also includes: performing focal plane imaging classification on the living single-cell image to filter out the single-cell image within the focal plane range; and the segmentation of the living single-cell image includes: segmenting the single-cell image within the focal plane range.

[0013] According to another embodiment of the present invention, performing focal plane imaging classification on the living single-cell images to screen out single-cell images located within the focal plane range includes: classifying the sample images of the collected cell samples at different focal planes and using them as a focal plane image sample dataset; using the focal plane image sample dataset to train a third deep neural network model to obtain a focal plane classification model; and using the focal plane classification model to perform focal plane imaging classification on the living single-cell images to screen out single-cell images located within the focal plane range.

[0014] According to another embodiment of the present invention, when training the third deep neural network model, the method includes applying image data enhancement processing to the focus plane image sample data set, wherein the image data enhancement processing includes at least one of spatial transformation processing, scaling processing, and image brightness adjustment processing of the image.

[0015] According to one embodiment of the present invention, it further includes: before using the target detection model, the cell segmentation model or the focal plane classification model, at least one of the target detection model, the cell segmentation model or the focal plane classification model is accelerated by using network structure acceleration, model inference acceleration and / or model pruning acceleration.

[0016] According to another embodiment of the present invention, the analyzing and determining the morphological parameters of the living single cell includes: performing morphological analysis on the characteristic parts of the living single cell obtained by segmentation to obtain geometric parameters of the characteristic parts; performing clarity measurement on the living single cell image to further screen out single cell images with clear morphology; and determining the morphological parameters of the living single cell based on the geometric parameters and the clarity measurement results.

[0017] According to yet another embodiment of the present invention, performing clarity measurement on the in vivo single-cell image includes: using one or more focus evaluation operators to evaluate the clarity of the in vivo single-cell image.

[0018] According to one embodiment of the present invention, determining the morphological parameters of the living single cell based on the geometric parameters and the clarity measurement results includes: performing a first sorting on the living single cell images based on the size of the geometric parameters; performing a second sorting on the living single cell images based on the size of the clarity; and based on the sorting results, screening out one or more images that are at the top of both the first sorting and the second sorting, and taking the average value of the geometric parameters of the one or more images as the morphological parameters of the living single cell.

[0019] According to another embodiment of the present invention, the geometric parameter includes at least one of length, width, area, ellipticity, number and position.

[0020] According to one embodiment of the present invention, the image to be detected includes at least one of a differential interference contrast image, a phase contrast image, a bright field image, and a dark field image.

[0021] In a second aspect of the present invention, a device for performing morphological detection of living cells based on a deep neural network is provided, comprising: a positioning module configured to use a target detection model based on a deep neural network to identify and locate a collected image to be detected containing living cells, so as to extract a living single-cell image; a segmentation module configured to use a cell segmentation model based on a deep neural network to segment the living single-cell image, so as to obtain characteristic parts of the living single cell; and a morphological analysis module configured to analyze and determine the morphological parameters of the living single cell based on the segmentation result.

[0022] According to one embodiment of the present invention, it also includes: a focal plane classification module, which is configured to perform focal plane imaging classification on the living single-cell image to filter out the single-cell image within the focal plane range; and the segmentation module is also configured to segment the single-cell image within the focal plane range.

[0023] In a third aspect of the present invention, a device for performing live cell morphology detection based on a deep neural network is provided, comprising at least one processor; and a memory storing program instructions, wherein when the program instructions are executed by the at least one processor, the device executes the method according to any one of the first aspects of the present invention.

[0024] In a fourth aspect of the present invention, a computer-readable storage medium is provided, which stores a program for living cell morphology detection. When the program is executed by a processor, the method according to any one of the first aspects of the present invention is executed.

[0025] In a fifth aspect of the present invention, a system for detecting the morphology of living cells based on a deep neural network is provided, comprising: an image acquisition unit for acquiring images to be detected containing living cells; a control terminal connected to the image acquisition unit and for receiving the images to be detected sent by the image acquisition unit; and an apparatus as described in the third aspect of the present invention, connected to the control terminal, for receiving the images to be detected sent by the control terminal to detect the images to be detected, and sending the detection results to the control terminal.

[0026] According to one embodiment of the present invention, the device includes an inference engine.

[0027] Through the above description of the technical solution of the present invention and its multiple embodiments, those skilled in the art can understand that the method of the present invention for performing living cell morphology detection based on a deep neural network is to locate and extract living single cells in the image to be detected by using a target detection model, and to segment the living single cells using a cell segmentation model and to analyze the characteristic parts obtained by segmentation to determine the morphological parameters of the living single cells. According to the method of the present invention, the activity of the detected cells can be guaranteed, and the non-destructive, accurate and rapid detection of the morphology of living cells can be achieved, which is beneficial to the clinical application and research of the detected cells and has important significance and application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The above and other objects, features and advantages of the exemplary embodiments of the present invention will become readily understood by reading the detailed description below with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present invention are shown in an exemplary and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts.

[0029] Figure 1 is a flow chart generally illustrating a method for performing in vivo cell morphology detection based on a deep neural network according to the present invention;

[0030] Figure 2 1 is a schematic diagram showing a clear field image of a living sperm in different postures according to an embodiment of the present invention;

[0031] Figure 3 It shows the Figure 2 Schematic diagram of the localization results of living sperm shown in;

[0032] Figure 4 A detailed flow chart of a method for detecting living cell morphology based on a deep neural network according to an embodiment of the present invention is shown;

[0033] Figure 5a is a schematic diagram illustrating a single-branch multi-class segmentation structure according to an embodiment of the present invention;

[0034] Figure 5b is a schematic diagram showing a multi-branch single-class segmentation structure according to an embodiment of the present invention

[0035] Figure 6 is another flow chart illustrating a method for detecting living cell morphology based on a deep neural network according to an embodiment of the present invention;

[0036] Figure 7 is another detailed flow chart illustrating a method for detecting living cell morphology based on a deep neural network according to an embodiment of the present invention;

[0037] Figure 8is a schematic diagram showing a device for performing living cell morphology detection based on a deep neural network according to an embodiment of the present invention; and

[0038] Figure 9 is a schematic diagram of a system for performing living cell morphology detection based on a deep neural network according to an embodiment of the present invention. DETAILED DESCRIPTION

[0039] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work shall fall within the scope of protection of the present invention.

[0040] It should be understood that the terms "first," "second," "third," and "fourth," etc. in the claims, description, and drawings of the present invention are used to distinguish different objects, rather than to describe a specific order. The terms "comprise" and "comprising" used in the description and claims of the present invention indicate the presence of the described features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or combinations thereof.

[0041] It should also be understood that the terminology used in this specification is for the purpose of describing specific embodiments only and is not intended to limit the present invention. As used in the specification and claims, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly indicates otherwise. It should further be understood that the term "and / or" as used in the specification and claims refers to any and all possible combinations of one or more of the associated listed items, including and including these combinations.

[0042] As used in this specification and claims, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0043] The implementation of live cell morphology detection faces many technical difficulties: for example, live cells are not static targets, making their positioning difficult; cells often move out of focus during activity, and out-of-focus images are blurry and cannot be used for morphology detection; and the contrast of live cell images is much lower than that of stained images, which increases the difficulty of detecting cell morphology parameters. The present invention addresses the shortcomings of the existing technology and provides a new and feasible solution. In particular, the method of the present invention for live cell morphology detection based on a deep neural network can achieve non-destructive detection of the morphology of live cells while ensuring their activity by performing steps such as identification and positioning and feature segmentation on images to be detected containing live cells. From the following description, those skilled in the art will understand that the present invention also provides methods for further improving detection accuracy and efficiency in multiple embodiments, such as performing focal plane imaging classification on live single cell images before segmentation, and accelerating at least one of the target detection model, cell segmentation model, or focal plane classification model to further improve detection speed and efficiency. Specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0044] Figure 1 FIG. 1 is a flow chart generally showing a method for detecting morphology of living cells according to the present invention. Figure 1 As shown in FIG, at step 102, method 100 may use a deep neural network-based target detection model to identify and locate the captured image containing living cells to extract a live single-cell image. In one embodiment, the deep neural network-based target detection model may employ at least one of the Yolo and Faster R-CNN models. The image to be detected may be acquired using a device such as a microscope or a camera. In one embodiment, the image to be detected may include at least one of a differential interference contrast image, a phase contrast image, a brightfield image, and a darkfield image. The image to be detected may contain images of one or more living cells. According to another embodiment of the present invention, the living cells may include, for example, living sperm cells or cancer cells. Identifying and locating the image to be detected may include identifying and locating each target living cell in the image to be detected to extract the live single-cell image and eliminate the influence of factors such as impurities. In one embodiment, assuming that the target living cells are living sperm cells, method 100 may identify which of the live sperm cells in the image to be detected, locate them, and extract them, eliminating the influence of other cell types or impurities. In another embodiment, the method 100 can extract a living single cell image containing living single cells with deformed morphologies such as polycephaly.

[0045] According to one embodiment of the present invention, method 100 can extract a single living cell image from an image to be detected. According to another embodiment of the present invention, method 100 can extract multiple single living cell images from an image to be detected, and each of these images can be individually detected and analyzed in subsequent steps. In one embodiment, when an image to be detected contains multiple living cells, method 100 can number the multiple living cells, thereby enabling tracking of living cells in different frames of the image to be detected.

[0046] Method 100 can identify and locate living single cells in the image to be detected by identifying the entire living single cell or by identifying the characteristic parts of the living single cell. Figure 2 and Figure 3 Provide explanation.

[0047] Figure 2 Schematic diagram showing the clear field images of the same living sperm in different postures according to an embodiment of the present invention. Figure 2 As shown in a, b and c in the figure, since live sperm can swim and flip, when collecting images to be detected, multiple images of the same live sperm in different postures are often collected, which increases the difficulty of identifying and locating live sperm. Figure 2 By identifying and locating the live sperm images in different positions shown in FIG. Figure 3 The positioning results are shown.

[0048] Figure 3 a, b and c in the figure are Figure 2 According to the positioning results of images a, b and c in FIG, according to this embodiment, method 100 can identify and locate live sperm by identifying the characteristic parts of live sperm (such as the head in the figure). Figure 3 As shown in , in one embodiment, method 100 can display the positioning results using a callout box. The method according to the present invention can identify and locate images containing living cells to be detected, and can achieve the recognition and positioning of living cells in different postures, and can significantly improve the accuracy and reliability of positioning.

[0049] Return below Figure 1 Continue to describe, such as Figure 1As shown in , at step 104, method 100 can use a cell segmentation model based on a deep neural network to segment the living single cell image to obtain the characteristic parts of the living single cell. In one embodiment, the cell segmentation model based on a deep neural network may include at least one of the models such as U-net, FCN, DeepLab, E-net, etc. The segmentation results of the living single cell image by method 100 may be different depending on the type of living single cell, that is, the characteristic parts to be segmented can be determined according to the type of living single cell. The characteristic parts of the living single cell may include one or more. For example, in one embodiment, the living cell may include a living sperm, and the characteristic parts may include at least one of the head, vacuole, midsection and tail of the sperm.

[0050] According to another embodiment of the present invention, before using the target detection model or the cell segmentation model, method 100 can accelerate at least one of the target detection model and the cell segmentation model by using, for example, network structure acceleration, model inference acceleration and / or model pruning acceleration.

[0051] The network structure acceleration mentioned above can be achieved by using simpler deep neural networks, such as MobileNet and ShuffleNet, lightweight neural networks suitable for mobile devices. Compared to conventional convolutional neural networks, MobileNet uses depthwise separable convolution to reduce the number of model parameters. Model inference acceleration can be achieved by optimizing and refactoring the network structure and reducing parameter precision. This optimization and refactoring can include eliminating unused output layers to reduce computational complexity, vertically integrating the network structure (e.g., fusing the convolutional layer, batch normalization (BN), and rectified linear unit (RLU) layers of the main neural network into a single constant bitrate (CBR) structure), and horizontally integrating the network structure (e.g., combining layers with the same structure but different weights into a wider layer). Reducing parameter precision can accelerate model inference by converting floating-point numbers (Float32) to half-precision floating-point numbers (Float16) or integers (INT8) during model inference. Lower data precision reduces memory usage and latency, resulting in smaller models. Model pruning acceleration can be achieved by obtaining the output of each layer and each neuron during model inference. Since units with outputs of 0 or approximately 0 are ineffective during inference, they can be pruned to reduce the amount of computation required for the inference process, thereby accelerating the model.

[0052] Next, the process proceeds to step 106, where method 100 can analyze and determine the morphological parameters of the living single cell based on the characteristic parts. Method 100 can determine the morphological parameters of the living single cell by analyzing the morphology of the characteristic parts. For example, according to one embodiment of the present invention, method 100 for analyzing and determining the morphological parameters of the living single cell may include: performing morphological analysis on the characteristic parts of the living single cell obtained by segmentation to obtain geometric parameters of the characteristic parts; performing clarity measurement on the living single cell image to further screen out single cell images with clear morphology; and determining the morphological parameters of the living single cell based on the geometric parameters and the clarity measurement results.

[0053] Combination of the above Figure 1 In general, the method for detecting the morphology of living cells based on a deep neural network according to the present invention is described exemplarily. It will be understood by those skilled in the art that the above description is illustrative and not restrictive, and those skilled in the art can make adjustments as needed. For example, the characteristic parts of living single cells can be adjusted and set according to the type of living single cells. For example, in another embodiment, the method 100 can further improve the accuracy and efficiency of the detection of the morphology of living cells by, for example, optimizing the target detection model or the cell segmentation model, and screening the images of living single cells. The following will be combined with Figure 4 The specific embodiment of the method for detecting the morphology of living cells based on a deep neural network according to the present invention is exemplarily described.

[0054] Figure 4 Detailed flow chart of method 200 for detecting morphology of living cells based on deep neural network according to an embodiment of the present invention. Figure 1 A specific form of the method 100 shown, so the above Figure 1 The description of method 100 also applies to method 200.

[0055] like Figure 4 As shown in FIG, at step 201, method 200 can directly or indirectly obtain an image to be detected containing living cells from, for example, a microscope or a camera. Then, at step 204, method 200 can use a target detection model based on a deep neural network to identify and locate the living cells in the image to be detected, so as to extract a living single cell image. In order to further improve the accuracy of identification and positioning, the embodiment of the present invention also provides a preferred construction method of the target detection model, such as Figure 4 As further shown in FIG, before step 204, method 200 may further include steps 202 and 203, which will be described in detail below.

[0056] At step 202, method 200 may acquire a living cell image of a large sample and perform a first annotation on individual cells in the living cell image. The living cell image of a large sample may include a certain number of living cell images, and a larger number of living cell images is more conducive to improving the detection accuracy of the target detection model. In one embodiment, method 200 may perform a first annotation on individual cells in the living cell image, including annotating characteristic parts of the individual cells. In another embodiment, method 200 may perform a first annotation on individual cells in the living cell image, for example, by manual annotation or machine annotation. In yet another embodiment, method 200 may achieve a first annotation on individual cells in the living cell image by using an annotation model.

[0057] Next, at step 203, method 200 may train a first deep neural network model using the living cell image with the first annotation to obtain the target detection model. In one embodiment, the first deep neural network model may be constructed based on at least one of the models such as Yolo, Faster R-CNN, etc. By training the first deep neural network model using the living cell image with the first annotation, the parameters and weights of the first deep neural network model may be continuously optimized. Method 200 trains the first deep neural network model using the living cell image with the first annotation to obtain a trained first deep neural network model, and the trained first deep neural network model may be referred to as a target detection model.

[0058] According to another embodiment of the present invention, at step 203, when training the first deep neural network model, method 200 may further include performing image data enhancement processing on the living cell image, wherein the image data enhancement processing may include at least one of spatial transformation processing, scaling processing, and image brightness adjustment processing. Spatial transformation processing may include image processing methods such as scaling, rotation, and perspective transformation. Scaling processing may include image processing methods such as uniform scaling or non-uniform scaling. Image brightness adjustment processing may include image processing methods such as brightening or darkening the entire image or a portion of the image.

[0059] It should be noted that the execution of step 202 or step 203 in method 200 and step 201 can be performed simultaneously in any order. In another embodiment, when method 200 executes step 204, step 202 and step 203 can also be executed simultaneously to continuously optimize the parameters of the target detection model, thereby continuously adjusting and improving the accuracy of recognition and positioning.

[0060] Furthermore, after executing step 204, method 200 may proceed to step 207. In step 207, method 200 may segment the living single cell image using a cell segmentation model based on a deep neural network to obtain the characteristic parts. In order to further improve the accuracy of characteristic part segmentation, the embodiment of the present invention also provides a preferred construction method of the cell segmentation model, for example Figure 4 As further shown in FIG, before step 207, method 200 may further include steps 205 and 206, which will be described in detail below.

[0061] like Figure 4 As shown in FIG, at step 205, the method 200 can perform a second annotation on the characteristic parts of the single cell in the acquired living cell image. The characteristic parts have been combined with the above Figure 1 The detailed description is given and will not be repeated here. In one embodiment, the living cell image obtained in step 205 can adopt the living cell image of the large sample obtained in step 202, and the second annotation can be the annotation of the characteristic parts based on the single cell of the first annotation in step 202. According to such a setting, the number of living cell images obtained and the number of image processing times can be reduced while ensuring the number of training samples, thereby reducing equipment loss and improving training speed. In another embodiment, the living cell image obtained in step 205 can be a living cell image obtained separately from step 202. In another embodiment, the method 200 can implement the second annotation of the characteristic parts of the single cell in the living cell image by, for example, manual annotation or machine annotation. In one embodiment, the method 200 can implement the second annotation of the characteristic parts of the single cell in the living cell image by using an annotation model.

[0062] Next, the process proceeds to step 206, where method 200 can use the living cell image with the second annotation to train a second deep neural network model to obtain the cell segmentation model. In one embodiment, the second deep neural network model can be constructed based on at least one of the models such as U-net, FCN, DeepLab, E-net, etc. By using the living cell image with the second annotation to train the second deep neural network model, the parameters and weights of the second deep neural network model can be continuously optimized. Method 200 uses the living cell image with the second annotation to train the second deep neural network model to obtain a trained second deep neural network model, and the trained second deep neural network model can be referred to as a cell segmentation model.

[0063] According to another embodiment of the present invention, at step 206, when training the second deep neural network model, method 200 may further include performing image data enhancement processing on the living cell image, wherein the image data enhancement processing may include at least one of spatial transformation processing, scaling processing, and image brightness adjustment processing on the image.

[0064] It should be noted that the execution of step 205 or step 206 and step 204 in method 200 can be performed simultaneously in any order. In another embodiment, when method 200 executes step 207, step 205 and step 206 can also be executed simultaneously to continuously optimize and update the parameters of the cell segmentation model, thereby continuously adjusting and improving the accuracy of the segmentation of the characteristic parts.

[0065] According to one embodiment of the present invention, the output part of the cell segmentation model can adopt a single-branch multi-class segmentation structure or a multi-branch single-class segmentation structure. Figure 5a and Figure 5b An exemplary description is given.

[0066] Figure 5a Schematic diagram showing a single-branch multi-class segmentation structure according to an embodiment of the present invention. Figure 5a As shown in , the live single cell image is input into the cell segmentation model and the softmax function is used for classification at the output to obtain the segmentation results of multiple characteristic parts. In one embodiment, the live single cell can be a live single sperm. Figure 5a Category 1 to category n shown in can represent the relevant information of characteristic parts such as the head, vacuole, and midpiece of a living single sperm, respectively.

[0067] Figure 5b Schematic diagram showing a multi-branch single-class segmentation structure according to an embodiment of the present invention. Figure 5b As shown in , a living single cell image can be input into a cell segmentation model, and multiple sigmoid functions are used for classification at the output to obtain segmentation results of multiple characteristic parts. In one embodiment, the living single cell can be a living single sperm. Figure 5b Category 1 to category n shown in can represent the relevant information of characteristic parts such as the head, vacuole, and midpiece of a living single sperm, respectively. Figure 5a and Figure 5b The output structure shown in can be selected as needed.

[0068] Return below Figure 4 According to the following description, those skilled in the art will understand that steps 208, 209, and 210 in method 200 may be Figure 1A specific implementation of step 106 is shown in FIG. Figure 1 The description of step 106 is also applicable to step 208, step 209 and step 210.

[0069] like Figure 4 As shown in , at step 208, method 200 may perform a morphological analysis on the characteristic parts of the living single cell obtained by segmentation to obtain geometric parameters of the characteristic parts. For example, in one embodiment, the geometric parameters may include at least one of length, width, area, ellipticity, number, and position. The geometric parameters may be determined by the morphological characteristics of the characteristic parts. In another embodiment, the living single cell may be a living single sperm, and its characteristic parts may include, for example, the head and vacuoles of the single sperm. Method 200 performs a morphological analysis on the head and vacuoles of the single sperm to obtain geometric parameters of the head, such as head length, head width, head area, and ellipticity, and geometric parameters of the vacuoles, such as vacuoles area, number of vacuoles, and vacuoles position.

[0070] like Figure 4 As further shown in FIG, at step 209, method 200 may perform clarity measurement on the live single-cell images so as to screen out single-cell images with clear morphology. Method 200 may perform clarity measurement on the live single-cell images extracted at step 204 so as to screen out one or more single-cell images with clear morphology. Clarity measurement can effectively exclude live single-cell images collected when live single cells are flipping or floating up and down (i.e., out of focus). Since such images have low clarity or the posture and morphology of live single cells in the images are not conducive to detection and analysis, excluding such images and screening out single-cell images with clear morphology can not only reduce the amount of data in subsequent image analysis, but also effectively improve the accuracy of the morphological parameters of the live single cells finally determined.

[0071] According to one embodiment of the present invention, at step 209, method 200 may include evaluating the clarity of the live single-cell image using one or more focus evaluation operators. Focus evaluation operators may include, for example, at least one of gray value variance (GLVA), gray value variance normalized (GLVN), and absolute center-to-center distance (ACMO). For ease of understanding, the following exemplary descriptions of these focus evaluation operators are provided.

[0072] The calculation formula of the gray value variance (GLVA) of the image mentioned above is as follows:

[0073]

[0074]

[0075] in, Represents the average value of the grayscale image I, the size of image I is m×n, I i，j Represents the grayscale value of the image at pixel (i, j). The smaller the GLVA value, the better the clarity of image I.

[0076] The calculation formula of the normalized image gray value variance (GLVN) mentioned above is as follows:

[0077]

[0078] in, Represents the average value of the grayscale image I, the size of image I is m×n, I i，j Represents the grayscale value of the image when the pixel is (i, j). The smaller the value of GLVN, the better the clarity of image I.

[0079] Furthermore, the calculation formula of the absolute center distance (ACMO) is as follows:

[0080]

[0081] Among them, ACMO is a measurement based on the grayscale histogram H, μ represents the average value of the grayscale histogram H, L represents the number of grayscale levels of the grayscale histogram H, and P k Indicates the frequency of the kth grayscale. The smaller the ACMO value, the better the image clarity.

[0082] Then, return to Figure 4 Continuing to describe, at step 210, method 200 can determine the morphological parameters of the living single cell based on the geometric parameters and the clarity measurement results. In one embodiment, method 200 can determine the morphological parameters of the living single cell based on the geometric parameters of the characteristic parts of the single cell image with clear morphology screened out in step 209. According to another embodiment of the present invention, at step 210, method 200 may include: performing a first sorting of the living single cell images based on the size of the geometric parameters; performing a second sorting of the living single cell images based on the size of the clarity; and screening out one or more images that are at the top of both the first sorting and the second sorting according to the sorting results, and taking the average value of the geometric parameters of the one or more images as the morphological parameters of the living single cell. In another embodiment, method 200 can determine the number of groups of the first sorting according to the type of geometric parameters. For ease of understanding, the following description will be made with reference to specific examples.

[0083] In one specific embodiment, taking a living single cell as a living single sperm as an example, the characteristic part can be the sperm head. The geometric parameters of the head can be set as head area, head length, and head width. Method 200 can perform three groups of first sorting on the living single cell images based on the size of the head area, head length, and head width: a first group of first sorting based on head area from large to small, a second group of first sorting based on head length from large to small, and a third group of first sorting based on head width from large to small. Method 200 can also perform a second sorting based on the clarity of the living single cell images. For example, method 200 can perform a second sorting based on the focus evaluation operator values ​​of the living single cell images from small to large. Then, based on the three groups of first sorting and one group of second sorting (referred to as four groups of sorting), method 200 can filter out one or more images that are ranked at the top of all four groups of sorting. For example, in one embodiment, method 200 can filter out images that appear in the top ten of all four groups of sorting. Next, method 200 can use the average value of the geometric parameters of the filtered one or more images as the morphological parameters of the living single cell.

[0084] Combination of the above Figure 4 The method 200 for detecting the morphology of living cells based on a deep neural network according to an embodiment of the present invention is described. It should be understood by those skilled in the art that the above description is exemplary and not restrictive. For example, step 209 may not be limited to the position shown in the figure, and it may also be adjusted to be performed before step 208 as needed, which may be beneficial to reduce the number of images during morphological analysis. For another example, the segmentation in step 207 may not be limited to segmenting living single cell images. In one embodiment, it may also be possible to segment single cell images within the range of the focal plane. The following will be combined with Figure 6 and Figure 7 An exemplary description is given.

[0085] Figure 6 This is another flow chart showing a method for detecting living cell morphology based on a deep neural network according to an embodiment of the present invention. Figure 6 The method shown is Figure 1 Another specific embodiment of the method shown. Figure 6 As shown in FIG, at step 102, method 100 can use a target detection model based on a deep neural network to identify and locate the collected image containing living cells to be detected, so as to extract a living single cell image. Step 102 has been previously described in conjunction with Figure 1 It has been explained in detail and will not be repeated here.

[0086] Next, at step 103, method 100 can perform focus plane imaging classification on the live single-cell images to select single-cell images within the focal plane. Because live cells may often deviate from the focal plane during activity, and defocused images are blurry and cannot be used for morphological detection, at step 103, method 100 can classify the degree of deviation from the focal plane in the live single-cell images to select single-cell images with clear imaging within the focal plane. In one embodiment, the focal plane (or focal plane) can be, for example, the focal plane of a microscope when acquiring the image to be detected.

[0087] The focus plane range described above may be a plane range centered on the focus plane that can be clearly imaged. For example, in another embodiment, the focus plane range may be a plane range from 1 micron above the focus plane to 1 micron below the focus plane. According to another embodiment of the present invention, in step 103, method 100 may use a focus plane classification model to perform focus plane imaging classification on live single-cell images. Method 100 according to the present invention performs focus plane imaging classification on live single-cell images before using a cell segmentation model to segment the live single-cell images, which can exclude most of the live single-cell images with blurred imaging, thereby effectively reducing the image processing amount and improving the processing speed of subsequent steps. Compared to the implementation method of excluding out-of-focus images through clarity measurement, screening out single-cell images within the focus plane range through the focus plane imaging classification method will be more accurate and intuitive.

[0088] Then, the process proceeds to step 1041, where method 100 can use a cell segmentation model based on a deep neural network to segment the single cell image within the focus plane to obtain the characteristic parts of the living single cell. Since the single cell image within the focus plane is relatively clear, the method 100 can segment the single cell image within the focus plane not only to reduce the amount of image processing, but also to improve the accuracy and efficiency of the segmentation results. From the above description, it can be understood that step 1041 can be Figure 1 FIG1 is a specific embodiment of step 104 shown in FIG1 , so the description of step 104 and its embodiment in the foregoing text is also applicable to step 1041.

[0089] like Figure 6 As further shown in FIG, at step 106, the method 100 can analyze and determine the morphological parameters of the living single cell based on the characteristic parts. Figure 1 A detailed description has been given and will not be repeated here.

[0090] Combination of the above Figure 6Another method for detecting the morphology of living cells based on a deep neural network according to an embodiment of the present invention is described exemplarily. It will be understood by those skilled in the art that the above description is exemplary and not restrictive. For example, the focus plane range can be adjusted and selected as needed, for example, it can be selected based on factors such as the type, size, and imaging effect of the living cells. In another embodiment, method 100 can also implement steps such as focal plane imaging classification of living single cell images based on a deep neural network model. Figure 7 An exemplary description is given.

[0091] Figure 7 This is another detailed flow chart showing a method for detecting the morphology of living cells based on a deep neural network according to an embodiment of the present invention. Through the following description, those skilled in the art will understand that: Figure 7 The method 200 shown in FIG. 2 may be Figure 6 A specific implementation of the method 100 shown in FIG. 1 is also based on Figure 4 A preferred embodiment of the method 200 is shown, so the above Figure 6 The method 100 shown and combined Figure 4 The description of the illustrated method 200 applies equally to the following description.

[0092] like Figure 7 As shown in FIG, step 201, step 202, step 203 and step 204 are combined with the above Figure 4 The same or similar as described above will not be repeated here. After executing step 204, method 200 may proceed to step 213. In step 213, method 200 may use the focal plane classification model to perform focal plane imaging classification on the living single cell image to screen out single cell images within the focus plane range. In one embodiment, the focal plane classification model may adopt at least one of the classification models such as Resnet and Densenet. In another embodiment, before using the focal plane classification model, method 200 may accelerate the focal plane classification model by using, for example, network structure acceleration, model inference acceleration and / or model pruning acceleration. Network structure acceleration, model inference acceleration and model pruning acceleration have been combined in the previous text. Figure 1 Furthermore, in order to improve the accuracy of focal plane imaging classification, the embodiment of the present invention also provides a preferred construction method of the focal plane classification model, such as Figure 7 As further shown in FIG, before step 213, method 200 may further include steps 211 and 212, which will be described in detail below.

[0093] At step 211, method 200 can classify the sample images of the collected cell samples at different focal planes and use them as a focus plane image sample data set. The cell samples described here may include cells with relatively fixed positions. For example, in one embodiment, the cell samples may include frozen cells, that is, the cell samples can be kept in a fixed position without being inactivated by freezing. Method 200 can obtain sample images at different focal planes by moving the cell samples to different focal positions, and can classify and label the sample images at different focal planes according to the physical position of the focal plane when the sample images were collected. The focus plane image sample data set may include one or more sample images and their classification labels and other information.

[0094] Next, at step 212, method 200 may train a third deep neural network model using the focus plane image sample dataset to obtain a focus plane classification model. In one embodiment, the third deep neural network model may be constructed based on at least one of models such as ResNet and DenseNet. By training the third deep neural network model using the focus plane image sample dataset, the parameters and weights of the third deep neural network model may be continuously optimized. Method 200 trains the third deep neural network model using the focus plane image sample dataset to obtain a trained third deep neural network model, which may be referred to as a focus plane classification model.

[0095] According to one embodiment of the present invention, in step 212, when training the third deep neural network model, method 200 may further include performing image data enhancement processing on the focus plane image sample dataset, wherein the image data enhancement processing may include at least one of spatial transformation processing, scaling processing, and image brightness adjustment processing of the image.

[0096] It should be noted that the execution of step 211 or step 212 and step 204 in method 200 can be performed simultaneously, regardless of the order. In another embodiment, when method 200 executes step 213, steps 211 and 212 can also be performed simultaneously to continuously optimize the parameters of the focus plane classification model, thereby continuously adjusting and improving the accuracy of focus plane classification.

[0097] Furthermore, after executing step 213, method 200 may proceed to step 2071. At step 2071, method 200 may use the cell segmentation model to segment the single cell image within the focus plane to obtain the characteristic parts of the living single cell within the focus plane. It is understandable that in step 2071, method 200 only needs to segment the characteristic parts of the single cell image within the focus plane, which can reduce the amount of image data processing and improve segmentation efficiency and accuracy. In one embodiment, the cell segmentation model in step 2071 can be obtained by executing steps 205 and 206 of method 200, wherein steps 205 and 206 have been combined in the above text. Figure 4 A detailed description has been given and will not be repeated here.

[0098] Then, the process can proceed to step 2081, where method 200 can perform morphological analysis on the characteristic parts of the living single cell obtained by segmentation and located within the focus plane to obtain geometric parameters of the characteristic parts. The morphological analysis method can refer to the above description of Figure 4 208 in the description.

[0099] like Figure 7 As further shown in FIG, at step 2091, method 200 may perform a clarity measurement on the single-cell images within the focus plane range, so as to further screen out single-cell images with clear morphology. Method 200 may perform a clarity measurement on the single-cell images within the focus plane range screened out at step 213, so as to screen out one or more single-cell images with clear posture and morphology. In some scenarios, the clarity measurement here can be understood as being used to screen out frontal images of living single cells, so as to exclude living single-cell images (such as side images, etc.) obtained when the living single cells are active and flipped. Since the posture and morphology of living single cells affect the effect of detection and analysis, excluding images with poor posture and morphology can not only further reduce the amount of data in subsequent image analysis, but also effectively improve the accuracy of the morphological parameters of the living single cells finally determined. According to one embodiment of the present invention, at step 2091, method 200 performing a clarity measurement on the single-cell images within the focus plane range may include: using one or more focus evaluation operators to evaluate the clarity of the single-cell images.

[0100] Furthermore, at step 210, method 200 may determine the morphological parameters of the living single cell based on the geometric parameters obtained in step 2081 and the clarity measurement results obtained in step 2091. The method for determining the morphological parameters may refer to Figure 4 The relevant description of step 210 is not repeated here.

[0101] Through the above description of the technical solutions and multiple embodiments of the method for detecting the morphology of living cells of the present invention, it can be understood by those skilled in the art that the method of the present invention can realize non-destructive and accurate detection of the morphology of living cells by performing operations such as identification, positioning and feature segmentation on the image to be detected containing living cells, thereby reducing the subjective error of manual detection and assisting in or partially replacing the clinical diagnosis and evaluation of doctors. Taking sperm morphology detection as an example, compared with the existing technology based on inactivated sperm morphology detection, the method of the present invention can maintain the physiological function of sperm and the integrity of DNA genetic material, and does not require the production of stained smears, etc., so it can eliminate the influence of interfering factors such as smear staining and dehydration on the test results, and has the characteristics of high accuracy and stability, simple process, and short time. More importantly, the living sperm screened by the method of the present invention can be used in clinical practice (such as test tube babies, etc.). In the above-mentioned embodiments of the present invention, implementation schemes such as focal plane imaging classification and clarity measurement are also provided, which can accurately screen out images with clear imaging and suitable morphology, so as to further improve the accuracy and reliability of the test results, and reduce the image processing amount to increase the detection rate, etc.

[0102] Furthermore, in the above description, the present invention also provides an embodiment based on deep learning using target detection models, cell segmentation models, and focal plane classification models. The principle of step-by-step feature abstraction and autonomous learning is closer to the working mode of the human brain, so it can extract feature information that cannot be captured by traditional methods, thereby improving the accuracy of living cell morphology detection. In some embodiments, the target detection model, cell segmentation model, etc. obtained by training with a large sample of living cell images can significantly enhance the generalization ability and robustness of living cell recognition, positioning and segmentation, and can significantly reduce the impact of interference factors such as shooting environment, brightness, impurities on morphological analysis, and have good adaptability and scalability. Through training and continuous iterative updates, the target detection model, cell segmentation model, and focal plane classification model of the present invention can meet the needs of detecting different characteristic parts of living cells (such as sperm head, midpiece or tail, etc.), different focal plane classification methods, different imaging methods (such as differential interference contrast, bright field, dark field, phase contrast, etc.) and living cell morphology analysis under different magnification conditions.

[0103] In a second aspect of the present invention, a device for performing morphological detection of living cells based on a deep neural network is provided, which may include a positioning module, which may be configured to use a target detection model based on a deep neural network to identify and locate a collected image to be detected containing living cells, so as to extract a living single-cell image; a segmentation module, which may be configured to use a cell segmentation model based on a deep neural network to segment the living single-cell image, so as to obtain characteristic parts of the living single cell; and a morphological analysis module, which may be configured to analyze and determine the morphological parameters of the living single cell based on the segmentation result.

[0104] According to one embodiment of the present invention, the apparatus for detecting the morphology of living cells according to the present invention may further include: a focal plane classification module, which may be configured to perform focal plane imaging classification on the living single-cell images to screen out single-cell images within the focal plane range; and the segmentation module may also be configured to segment the single-cell images within the focal plane range.

[0105] In a third aspect of the present invention, a device for detecting the morphology of living cells based on a deep neural network is provided, which may include at least one processor; a memory which may store program instructions, and when the program instructions are executed by the at least one processor, the device executes the method according to any one of the first aspects of the present invention. Figure 8 An exemplary description is given.

[0106] Figure 8 This is a schematic diagram of a device for detecting the morphology of living cells based on a deep neural network according to an embodiment of the present invention. The device 800 can be used to identify and locate images containing living cells to be detected, segment feature parts, and determine morphological parameters, so as to achieve the above-mentioned combination. Figure 1-Figure 7 The living cell morphology detection scheme of the present invention.

[0107] like Figure 8 As shown in FIG, the device 800 may include a CPU 801, which may be a general-purpose CPU, a dedicated CPU, or other execution unit for information processing and program execution. Furthermore, the device 800 may also include a mass storage 802 and a read-only memory ROM 803. The mass storage 802 may be configured to store various types of data, including various programs required for, for example, target detection models and cell segmentation models. The ROM 803 may be configured to store data required for initializing various functional modules in the living cell morphology detection apparatus of the device 800, basic input / output drivers for the system, and booting the operating system.

[0108] Furthermore, the device 800 may also include other hardware or components, such as a graphics processing unit ("GPU") 804 and a field programmable gate array ("FPGA") 805. It will be appreciated that although various hardware or components are shown in the device 800, this is merely exemplary and not restrictive, and those skilled in the art may add or remove corresponding hardware as needed.

[0109] The device 800 of the present invention may further include a communication interface 806, so that it can be connected to a local area network / wireless local area network (LAN / WLAN) through the communication interface 806, and then connected to, for example, a control terminal or to the Internet ("Internet") through the LAN / WLAN. Alternatively or additionally, the device 800 of the present invention may also be directly connected to the Internet or a cellular network based on wireless communication technology through the communication interface 806, such as wireless communication technology based on the third generation ("3G"), fourth generation ("4G") or fifth generation ("5G") generation. In some application scenarios, the device 800 of the present invention may also access servers and possible databases of an external network as needed to obtain various known information, data and modules, etc., and may remotely store various detected data.

[0110] The CPU 801, mass storage 802, read-only memory ("ROM") 803, GPU 804, FPGA 805, and communication interface 806 of the device 800 of the present invention can be interconnected via a bus 807 and can interact with peripheral devices via the bus. In one embodiment, the CPU 801 can control other hardware components in the device 800 and its peripheral devices via the bus 807.

[0111] During operation, the processor CPU 801 or the graphics processor GPU 804 of the device 800 of the present invention can receive data through the bus 807 and retrieve the computer program instructions or codes stored in the memory 802 (for example, codes related to the detection of living cell morphology based on a deep neural network) to detect the received image to be detected. Specifically, the CPU 801 or GPU 804 can execute a target detection model based on a deep neural network to identify and locate the image to be detected to obtain a living single cell image, etc. At the same time, the CPU 801 or GPU 804 of the device 800 can also execute a cell segmentation model to segment the characteristic parts of the living single cell image. Then, the processor CPU 801 or GPU 804 can analyze and determine the morphological parameters of the living single cell based on the characteristic parts. After the CPU 801 or GPU 804 determines the morphological parameter results of the living single cell by executing the detection program, the results can be uploaded to a network, such as a remote database or an external control terminal, etc., through, for example, a communication interface 806.

[0112] It should also be understood that any module, unit, component, server, computer, terminal, or device that executes instructions of the examples of the present invention may include or otherwise access computer-readable media, such as storage media, computer storage media, or data storage devices (removable and / or non-removable) such as magnetic disks, optical disks, or tapes. Computer storage media may include volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable instructions, data structures, program modules, or other data.

[0113] In a fourth aspect of the present invention, a computer-readable storage medium is provided, which may store a program for performing living cell morphology detection based on a deep neural network. When the program is run by a processor, the method according to any one of the first aspects of the present invention is executed.

[0114] The computer-readable storage medium may be any suitable magnetic storage medium or magneto-optical storage medium, such as resistive random access memory (RRAM), dynamic random access memory (DRAM), static random access memory (SRAM), enhanced dynamic random access memory (EDRAM), high-bandwidth memory (HBM), hybrid memory cube (HMC), etc., or any other medium that can be used to store the required information and can be accessed by the application, module, or both. Any such computer storage medium may be part of the device or accessible or connectable to the device. Any application or module described in the present invention may be implemented using computer-readable / executable instructions that may be stored or otherwise retained by such a computer-readable medium.

[0115] In a fifth aspect of the present invention, a system 900 for detecting the morphology of living cells based on a deep neural network is provided. Figure 9 As shown in , it may include: an image acquisition unit 901, which can be used to acquire an image to be detected containing living cells; a control terminal 902, which can be connected to the image acquisition unit 901 and used to receive the image to be detected sent by the image acquisition unit 901; and the device 800 as described in the third aspect of the present invention, which can be connected to the control terminal 902, used to receive the image to be detected sent by the control terminal 902 to detect the image to be detected, and send the detection result to the control terminal 902.

[0116] According to one embodiment of the present invention, the image acquisition unit 901 may include at least one of an optical microscopic imaging device (e.g., a microscope), a camera, a light source device, and the like. The control terminal 902 may be connected to the image acquisition unit 901 via a wired or wireless connection. In another embodiment, the control terminal 902 may include, for example, one or more of a desktop computer, a laptop computer, a tablet computer, a smart phone, and the like. The device 800 and the control terminal 902 may be connected via a wired or wireless connection and exchange information. The device 800 may send control information, such as acquiring an image to be detected, to the control terminal 902 and may send detection results to the control terminal 902. The control terminal 902 may send information about the image to be detected and status information to the device 800 in real time. According to one embodiment of the present invention, the device 800 may include an inference engine. In a specific embodiment, before using the target detection model, cell segmentation model or focal plane classification model, one or more models of the target detection model, focal plane classification model and cell segmentation model can be accelerated by using, for example, network structure acceleration, model inference acceleration and / or model pruning acceleration, and then run on the inference machine to detect the received image to be detected, which is beneficial to improving the inference speed of the target detection model, cell segmentation model or focal plane classification model and the detection speed of the image to be detected.

[0117] In some application scenarios, the image acquisition unit 901, the control terminal 902 and the device 800 can be deployed in an internal network, for example, they can be connected to the same intranet through a router or a switch. According to such a setting, the system of the present invention can be prevented from being publicly accessed, thereby better protecting the information security within the system, especially in terms of information involving personal privacy such as medical images, the system of the present invention has good deployment value and application prospects. Furthermore, in some embodiments, the device 800 can be remotely connected to a server, etc., so as to accept operations such as remote updates. Such a setting can better realize the update and maintenance of the system, and reduce the time and cost of on-site maintenance. In other embodiments, the device 800 can continuously iteratively update the model parameters through local self-learning, so that it can better serve the locally connected control terminal and better adapt to the locally collected image data and detection environment, etc., to ensure the accuracy and reliability of the detection.

[0118] In one specific embodiment, during information exchange between device 800 and control terminal 902, the network packet format can be defined as 32-bit data size + 16-bit data ID + data. The 32-bit data size ensures that the program knows the start and receiving range of the data, the 16-bit data ID ensures that the program performs different processing based on different data types, and the data portion is decoded accordingly based on the data type. The system according to the present invention can achieve real-time requirements through multiple processes and multiple queues, as described below in an exemplary manner.

[0119] Specifically, device 800 can use five process queues to store data at different stages. These include: In the first stage, the network process receives network data streams in real time and stores them in the buffer_queue process queue; in the second stage, a buffer_worker process processes the received buffer_queue data in real time, parses it into network message packets, and passes it to the msg_queue process queue; in the third stage, the msg_worker process processes the msg_queue data in real time, parses control commands and image data, and passes the image data to the img_queue process queue; in the fourth stage, the batch_worker process processes the img_queue data in real time, combining batch_size images into a single data set and passing it to the batch_queue process queue; and in the fifth stage, the tensor_worker process processes the batch_queue data in real time, pre-processes it into tensor data usable by device 800, and then performs inference to obtain the final result. The detection results of device 800 can be transmitted back to the control terminal for display.

[0120] Although the embodiments of the present invention are described above, the contents are only embodiments used to facilitate understanding of the present invention and are not intended to limit the scope and application scenarios of the present invention. Any person skilled in the art of the present invention may make any modifications and changes in the form and details of the implementation without departing from the spirit and scope disclosed by the present invention, but the scope of patent protection of the present invention shall still be based on the scope defined by the attached claims.

Claims

1. A method for detecting cell morphology, comprising: Identifying and locating the collected images containing cells to be detected to extract single-cell images; Performing focal plane imaging classification on the single-cell images using a focal plane classification model to screen out single-cell images within a focal plane range, wherein the focal plane imaging classification is performed to classify the degree of deviation from the focal plane when imaging living single cells in the single-cell images; Segmenting the single cell image within the focus plane to obtain characteristic parts of the single cell; as well as Performing morphological analysis on the characteristic part, and determining the morphological parameters of the single cell according to the geometric parameters of the characteristic part; The identifying and locating of the image to be detected includes: in response to the image to be detected containing a plurality of living cells, tracking the living cells in the images to be detected of different frames collected.

2. The method according to claim 1, wherein identifying and locating the image to be detected comprises identifying at least one of the following to locate the image: Identify entire single cells; or Identify characteristic parts of single cells.

3. The method according to claim 1, wherein identifying and locating the image to be detected comprises Use the target detection model to identify and locate the image to be detected.

4. The method according to claim 3, wherein before using the target detection model to identify and locate the image to be detected, the method further comprises: Acquiring a cell image of a large sample, and performing a first annotation on individual cells in the cell image; as well as A first deep neural network model is trained using the cell image with the first annotation to obtain the target detection model.

5. The method according to any one of claims 1 to 4, wherein segmenting the single cell image comprises: A cell segmentation model is used to segment the single cell image.

6. The method according to claim 5, wherein before segmenting the single cell image using a cell segmentation model, the method further comprises: Performing a second annotation on characteristic parts of a single cell in the acquired cell image; as well as A second deep neural network model is trained using the cell image with the second annotation to obtain the cell segmentation model.

7. The method according to claim 4, wherein when training the first deep neural network model, the method includes performing image data augmentation processing on the cell image, wherein The image data enhancement processing includes at least one of a spatial transformation processing, a scaling processing and an image brightness adjustment processing on the image.

8. The method according to claim 6, wherein when training the second deep neural network model, the method includes performing image data augmentation processing on the cell image, wherein The image data enhancement processing includes at least one of a spatial transformation processing, a scaling processing and an image brightness adjustment processing on the image.

9. The method according to claim 5, wherein the output part of the cell segmentation model adopts a single-branch multi-class segmentation structure or a multi-branch single-class segmentation structure.

10. The method of claim 1, wherein the cells comprise living cells.

11. The method of claim 10, wherein the living cell comprises a living sperm, and the characteristic portion comprises at least one of a sperm head, a vacuole, a midpiece, and a tail.

12. The method according to claim 1, wherein performing focal plane imaging classification on the single cell images to filter out single cell images within a focal plane range comprises: Classify the sample images of the collected cell samples at different focal planes and use them as a focal plane image sample dataset; Using the focus plane image sample dataset to train a third deep neural network model to obtain a focus plane classification model; as well as The single cell images are subjected to focal plane imaging classification using the focal plane classification model to screen out single cell images within the focal plane range.

13. The method according to claim 12, wherein when training the third deep neural network model, the method includes performing image data enhancement processing on the focus plane image sample dataset, wherein The image data enhancement processing includes at least one of a spatial transformation processing, a scaling processing and an image brightness adjustment processing on the image.

14. The method of claim 1 , wherein determining the morphological parameters of the single cell comprises: performing morphological analysis on the characteristic parts of the single cell obtained by segmentation to obtain geometric parameters of the characteristic parts; Performing a clarity measurement on the single cell image to further screen out images with clear morphology; as well as The morphological parameters of the single cell are determined according to the geometric parameters and the clarity measurement results.

15. The method according to claim 14, wherein performing a sharpness measurement on the single cell image comprises: One or more focus evaluation operators are used to evaluate the clarity of the single cell image. 16 . The method according to claim 15 , wherein the focus evaluation operator comprises at least one of image grayscale value variance, normalized image grayscale value variance, and absolute center distance.

17. The method according to claim 15, wherein determining the morphological parameters of the single cell according to the geometric parameters and the clarity measurement result comprises: performing a first sorting on the single cell images based on the size of the geometric parameter; performing a second sorting on the single cell images based on the clarity; as well as According to the sorting results, one or more images that are at the top of both the first sorting and the second sorting are screened out, and the average value of the geometric parameters of the one or more images is used as the morphological parameter of the single cell.

18. The method of claim 1, wherein the geometric parameter comprises at least one of length, width, area, ellipticity, number, and position.

19. The method according to claim 1, wherein the image to be detected comprises at least one of a differential interference contrast image, a phase contrast image, a bright field image, and a dark field image.

20. A device for cell morphology detection, comprising: a positioning module configured to identify and locate the collected image containing cells to be detected to extract a single cell image; a focus plane classification module configured to perform focus plane imaging classification on the single-cell images using a focus plane classification model to filter out single-cell images within a focal plane range, wherein the focus plane imaging classification is performed to classify the degree of deviation from the focus plane when imaging living single cells in the single-cell images; a segmentation module configured to segment the single cell image within the focus plane to obtain characteristic parts of the single cell; as well as a morphological analysis module configured to perform morphological analysis on the characteristic part and determine the morphological parameters of the single cell based on the geometric parameters of the characteristic part; The positioning module is further configured to track the living cells in the images to be detected in different frames in response to the image to be detected containing multiple living cells.

21. A device for detecting cell morphology, comprising: at least one processor; A memory storing program instructions, which, when executed by the at least one processor, causes the device to perform the method according to any one of claims 1 to 19.

22. A computer-readable storage medium storing a program for cell morphology detection, wherein when the program is executed by a processor, the method according to any one of claims 1 to 19 is executed.

23. A system for detecting cell morphology, comprising: An image acquisition unit, which is used to acquire an image containing cells to be detected; a control terminal connected to the image acquisition unit and configured to receive the image to be detected sent by the image acquisition unit; as well as The device as described in claim 21 is connected to the control terminal, and is used to receive the image to be detected sent by the control terminal to detect the image to be detected, and send the detection result to the control terminal.

24. The system of claim 23, wherein the device comprises an inference engine.

Images