Cell quantification method and device, computer equipment and storage medium

Abstract

The invention relates to a cell quantification method and device, computer equipment and a storage medium. The method comprises the following steps: extracting cell contour features of an endoscope cell image corresponding to a region of interest through a pre-constructed deep learning model, and separating a cell contour region corresponding to each cell in the endoscope cell image according to the cell contour features; classifying the cell contour region corresponding to each cell through the deep learning model to obtain a cell category of each cell; determining an overall cell quantification result corresponding to the endoscope cell image through the deep learning model according to the cell contour region of each cell; and performing depth feature extraction on the cell contour region of which the cell category is the abnormal cell through the deep learning model, and determining an abnormal cell quantification result according to the depth feature and the cell contour feature of the abnormal cell. By adopting the method, the abnormal condition of the cells can be accurately and efficiently judged.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a cell quantification method, device, computer equipment and storage medium. Background Art

[0002] An endoscope is a medical device that is mainly used to examine the internal organs and tissues of the human body. With the development of medical technology, endoscopes have gradually become an important tool in the fields of medicine and industry. Endoscopes can help staff observe areas that cannot be directly observed, thereby improving the staff's diagnostic efficiency and accuracy.

[0003] When using an endoscope, doctors need to manually analyze the real-time cell images collected by the endoscope to determine cell abnormalities, which is impossible to accurately and efficiently determine cell abnormalities. Summary of the invention

[0004] Based on this, it is necessary to provide a cell quantification method, device, computer equipment and storage medium that can accurately and efficiently determine cell abnormalities in response to the above technical problems.

[0005] In a first aspect, the present application provides a cell quantification method, comprising:

[0006] Extracting cell contour features of the endoscopic cell image corresponding to the region of interest through a pre-built deep learning model, and separating the cell contour region corresponding to each cell in the endoscopic cell image according to the cell contour features;

[0007] Classifying the cell contour area corresponding to each cell by the deep learning model to obtain the cell category of each cell;

[0008] Determining the overall cell quantification result corresponding to the endoscopic cell image according to the cell contour area of ​​each cell through the deep learning model;

[0009] The deep learning model is used to extract deep features of cell contour areas whose cell categories are abnormal cells, and the quantification results of the abnormal cells are determined according to the deep features and cell contour features of the abnormal cells.

[0010] In one embodiment, determining the overall cell quantification result corresponding to the endoscopic cell image according to the cell contour area of ​​each cell by the deep learning model includes:

[0011] Determining the number of cells in the endoscopic cell image according to the number of the cell outline areas;

[0012] Acquiring an image area of ​​the endoscopic cell image through the deep learning model, and determining a cell density of the endoscopic cell image according to the image area and the number of cells;

[0013] The overall cell quantification result corresponding to the endoscopic cell image is obtained according to the cell number and cell density of the endoscopic cell image.

[0014] In one embodiment, the cell contour area corresponding to each cell is classified by the deep learning model to obtain the cell category of each cell, including:

[0015] Performing denoising on the cell contour area corresponding to each cell through the deep learning model;

[0016] Perform threshold segmentation on the denoised cell contour area;

[0017] Grayscale analysis is performed on the segmented cell contour area to obtain the fluorescence signal intensity of each cell;

[0018] The cell type corresponding to each cell is determined according to the fluorescence signal intensity of each cell.

[0019] In one embodiment, deep feature extraction is performed on the cell contour area of ​​the cell category as abnormal cells by using the deep learning model, and the abnormal cell quantification result is determined according to the deep features and cell contour features of the abnormal cells, including:

[0020] Performing deep feature extraction on the cell contour area of ​​abnormal cells by using the deep learning model;

[0021] The depth features and cell contour features of abnormal cells are fused to obtain fused features;

[0022] The abnormal cells are graded according to the characteristics after fusion to determine the quantification results of the abnormal cells.

[0023] In one of the embodiments, the pre-built deep learning model includes an object detection subnetwork and an enhancement module;

[0024] The method of extracting cell contour features of the endoscopic cell image corresponding to the region of interest through a pre-built deep learning model, and separating the cell contour region corresponding to each cell in the endoscopic cell image according to the cell contour features, comprises:

[0025] Extracting features from the endoscopic cell image corresponding to the region of interest through the target detection subnetwork to obtain an original feature map;

[0026] Performing pixel enhancement on the original feature map by the enhancement module to obtain an enhanced feature map;

[0027] The cell contour features in the enhanced feature map are extracted through the target detection subnetwork.

[0028] In one embodiment, the target detection subnetwork is determined as follows:

[0029] Get the channel factor of each network channel in the lightweight network;

[0030] The network channels whose channel factors are smaller than the preset threshold are regarded as the channels to be pruned;

[0031] The to-be-cropped channels in the lightweight network are cropped, and the residual network in the Mask R-CNN network is replaced with the cropped lightweight network to obtain a target detection subnetwork.

[0032] In one embodiment, the method further includes: acquiring an endoscopic cell image corresponding to the region of interest through an endoscope handle of the endoscope; the endoscope handle includes:

[0033] Lens module;

[0034] An optical fiber pressing plate, including a pressing plate groove body, fixed to the outer wall of the lens module;

[0035] A lighting module including a fiber optic cable and a fiber optic connector;

[0036] Among them, the optical fiber cable is arranged inside the endoscope handle, passes through the pressure plate groove and is connected to one end of the optical fiber connector and extends to the other end; the other end of the optical fiber connector is fixedly connected to the lens module; the optical fiber cable and the optical fiber connector are used to provide incident light to the lens module so that the lens module obtains an endoscopic cell image corresponding to the area of ​​interest.

[0037] In a second aspect, the present application also provides a cell quantification device, comprising:

[0038] A contour extraction module, used to extract the cell contour features of the endoscopic cell image corresponding to the region of interest through a pre-built deep learning model, and separate the cell contour region corresponding to each cell in the endoscopic cell image according to the cell contour features;

[0039] A classification module, used to classify the cell contour area corresponding to each cell through the deep learning model to obtain the cell category of each cell;

[0040] An overall quantification module, used to determine the overall cell quantification result corresponding to the endoscopic cell image according to the cell contour characteristics and cell contour area of ​​each cell through the deep learning model;

[0041] The abnormality quantification module is used to extract deep features of the cell contour area of ​​the cell category as abnormal cells through the deep learning model, and determine the abnormal cell quantification result according to the deep features and cell contour features of the abnormal cells.

[0042] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0043] Extracting cell contour features of the endoscopic cell image corresponding to the region of interest through a pre-built deep learning model, and separating the cell contour region corresponding to each cell in the endoscopic cell image according to the cell contour features;

[0044] Classifying the cell contour area corresponding to each cell by the deep learning model to obtain the cell category of each cell;

[0045] Determining the overall cell quantification result corresponding to the endoscopic cell image according to the cell contour area of ​​each cell through the deep learning model;

[0046] The deep learning model is used to extract deep features of cell contour areas whose cell categories are abnormal cells, and the quantification results of the abnormal cells are determined according to the deep features and cell contour features of the abnormal cells.

[0047] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0048] Extracting cell contour features of the endoscopic cell image corresponding to the region of interest through a pre-built deep learning model, and separating the cell contour region corresponding to each cell in the endoscopic cell image according to the cell contour features;

[0049] Classifying the cell contour area corresponding to each cell by the deep learning model to obtain the cell category of each cell;

[0050] Determining the overall cell quantification result corresponding to the endoscopic cell image according to the cell contour area of ​​each cell through the deep learning model;

[0051] The deep learning model is used to extract deep features of cell contour areas whose cell categories are abnormal cells, and the quantification results of the abnormal cells are determined according to the deep features and cell contour features of the abnormal cells.

[0052] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0053] Extracting cell contour features of the endoscopic cell image corresponding to the region of interest through a pre-built deep learning model, and separating the cell contour region corresponding to each cell in the endoscopic cell image according to the cell contour features;

[0054] Classifying the cell contour area corresponding to each cell by the deep learning model to obtain the cell category of each cell;

[0055] Determining the overall cell quantification result corresponding to the endoscopic cell image according to the cell contour area of ​​each cell through the deep learning model;

[0056] The deep learning model is used to extract deep features of cell contour areas whose cell categories are abnormal cells, and the quantification results of the abnormal cells are determined according to the deep features and cell contour features of the abnormal cells.

[0057] The above-mentioned cell quantification method, device, computer equipment, storage medium and computer program product, due to the high clarity of the endoscopic cell image, can clearly present the cell nucleus and cell outline in the region of interest, and then extract the cell outline features of the endoscopic cell image corresponding to the region of interest through the deep learning model, and can accurately and quickly separate the cell outline area corresponding to each cell in the endoscopic cell image based on the cell outline features. After determining the cell category through the deep learning module, the overall cell quantification result corresponding to the endoscopic cell image can be accurately and efficiently obtained based on the cell outline area of ​​each cell, and deep feature extraction is performed on abnormal cells, and accurate abnormal cell quantification results are obtained by combining the deep features and cell outline features of abnormal cells. In this way, cell abnormalities can be accurately and efficiently determined, effectively assisting clinicians in judging whether there are tumor lesions in the region of interest. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0059] FIG1 is a schematic diagram of a process for cell quantification in one embodiment;

[0060] FIG2 is a schematic diagram of a process for determining the overall cell quantification result corresponding to an endoscopic cell image according to the cell contour area of ​​each cell through a deep learning model in one embodiment;

[0061] FIG3 is a schematic diagram of a process of performing deep feature extraction on a cell contour region of a cell category of abnormal cells through a deep learning model, and determining a quantification result of abnormal cells according to the deep features and cell contour features of the abnormal cells in one embodiment;

[0062] FIG4 is a block diagram of a cell quantification device in one embodiment;

[0063] FIG. 5 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0064] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0065] In one embodiment, as shown in FIG. 1 , a method for detecting abnormal cells is provided. This embodiment is illustrated by applying the method to an endoscope. In this embodiment, the method includes the following steps:

[0066] Step 102: extract the cell contour features of the endoscopic cell image corresponding to the region of interest through a pre-built deep learning model, and separate the cell contour region corresponding to each cell in the endoscopic cell image according to the cell contour features.

[0067] The region of interest may be a lesion site, for example, the region of interest is the brain. The endoscopic cell image refers to a cell image of the region of interest acquired by the endoscope. For example, the endoscope may be a handheld endoscope. The endoscopic cell image is a fluorescent cell image. The fluorescent cell image refers to a cell image generated by irradiating the stained cells with an excitation light source when the cells are stained.

[0068] Specifically, the endoscope includes an endoscope handle and an image processing mechanism. The endoscope is inserted into the inspection cavity of the target object through the endoscope handle to find the region of interest of the target object. The target object refers to the target human body that needs to be biopsied. The region of interest can be a lesion site, for example, the region of interest is the brain. Abnormal cells in the brain are tumor cells, specifically atypical cells of brain media tumors. After the endoscope is positioned at the region of interest, the spraying device sprays the dye to the region of interest through the channel of the endoscope. After the dyeing is completed, the endoscope obtains the endoscopic cell image of the region of interest of the target object. Since the endoscopic cell image collected in real time by the handheld endoscope is a real-time image magnified more than 1000 times, the cell nucleus and cell outline in the region of interest can be clearly presented. The endoscope transmits the acquired endoscopic cell image to the image processing mechanism, and the image processing software is installed in the image processing mechanism. The cell outline area corresponding to each cell in the endoscopic cell image is separated by the image processing software.

[0069] A pre-built deep learning model is stored in the image processing software. The deep learning model is obtained by training a large number of cell images labeled with cell contours. For example, the pre-built deep learning model can be a Mask R-CNN network (Mask Region-based Convolutional Neural Network, regional convolutional neural network) after lightweight processing, and specifically the residual network can be replaced with a Mask R-CNN network of a lightweight network. The image processing software calls the pre-built deep learning model to extract cell contour features from the endoscopic cell image. Cell contour features may include cell boundaries, sizes, morphology, etc. The location area of ​​each cell in the endoscopic cell image, i.e., the cell contour area, is determined based on the extracted cell contour features, and the cell contour area is separated from the endoscopic cell image.

[0070] Furthermore, since the endoscopic cell image obtained by the endoscope may have uneven dyeing, the endoscopic cell image can be subjected to colorimetric processing by image processing software to obtain the endoscopic cell image after colorimetric processing. The colorimetric processing method can be: obtaining a preset cell image, which refers to a cell image with uniform dyeing. The endoscopic cell image and the preset cell image are respectively converted into dye matrices. The dye matrix refers to a matrix subjected to colorimetric normalization processing. The dye matrix corresponding to the endoscopic cell image is aligned with the dye matrix corresponding to the preset cell image, thereby converting the endoscopic cell image into a cell image with uniform dyeing.

[0071] Step 104: classify the cell contour area corresponding to each cell through a deep learning model to obtain the cell category of each cell.

[0072] The deep learning model is used to extract the fluorescence features of the cell contour area corresponding to each cell, and the cell category of each cell is determined according to the intensity of the extracted fluorescence features. Among them, the cell categories include normal cells and abnormal cells. Abnormal cells refer to cell components that are different from normal cells in size, morphology, etc.

[0073] Furthermore, the deep learning model can also obtain the pre-stored cell features of normal cells as standard cell features. Standard cell features include edge features of normal cells. The cell outline features of each cell are compared with the cell outline features of normal cells for similarity. When the similarity is less than a preset threshold, it indicates that the cell is an abnormal cell. When the similarity is greater than or equal to the preset threshold, it indicates that the cell is a normal cell.

[0074] Step 106, determining the overall cell quantification result corresponding to the endoscopic cell image according to the cell contour area of ​​each cell through a deep learning model.

[0075] The deep learning model is used to count the number of cell contour areas, and this number is determined as the number of cells contained in the endoscopic cell image. At the same time, the deep learning model can also obtain the image area of ​​the endoscopic image, so as to determine the cell density of the endoscopic cell image based on the image area and the number of cells. The number of cells and cell density of the endoscopic cell image are then used as the overall cell quantification result. By performing an overall quantitative analysis of the endoscopic cell image, early abnormal changes in the area of ​​interest can be discovered, and it can be determined whether there are signs of lesions in the area of ​​interest.

[0076] Step 108, performing deep feature extraction on the cell contour area of ​​the cell category of abnormal cells through a deep learning model, and determining the abnormal cell quantification result according to the deep features and cell contour features of the abnormal cells.

[0077] Optionally, the cell contour area of ​​the cell category is determined to be an abnormal cell, and the deep feature extraction of the cell contour area of ​​the abnormal cell is performed through a deep learning model. Among them, the deep feature is used to characterize the morphology, structure and other information of the cell. The deep features of the abnormal cells are fused with the cell contour features, and the pathological state of the abnormal cells is graded according to the fused features to obtain the quantification results of the abnormal cells. Among them, the classification of the pathological state can include low-grade tumors, which usually grow slowly, have relatively regular cell morphology, have obvious differentiation characteristics, and have relatively benign biological behaviors, medium-grade tumors, which have cell morphology and biological behaviors between low and high levels, and high-grade tumors, which usually grow rapidly, have irregular cell morphology, and have strong malignant characteristics.

[0078] According to the overall cell quantification results and abnormal cell quantification results corresponding to the endoscopic cell images, the lesion conditions in the area of ​​interest can be accurately understood.

[0079] Furthermore, the key features of multiple scales in the endoscopic cell image can be extracted through the feature graph pyramid network in the deep learning model. Key features refer to features related to cell morphology and structure. Determine the key features of each scale corresponding to the abnormal cells and the grading results corresponding to the cell contour features. For each abnormal cell, the grading results corresponding to the key features of multiple scales corresponding to the abnormal cell are fused with the grading results corresponding to the cell contour features to obtain the abnormal cell quantification results corresponding to the abnormal cell. By fusing the grading results at different scales, a more accurate abnormal cell grading result can be obtained.

[0080] Furthermore, the number of abnormal cells can be obtained from the overall cell quantification result of the endoscopic cell image, and the abnormal cell quantification result can be jointly determined based on the number of abnormal cells, depth features, and cell contour features.

[0081] In the above-mentioned cell quantification method, due to the high clarity of the endoscopic cell image, the cell nucleus and cell outline in the region of interest can be clearly presented, and then the cell outline features of the endoscopic cell image corresponding to the region of interest are extracted through the deep learning model, and the cell outline area corresponding to each cell in the endoscopic cell image can be accurately and quickly separated according to the cell outline features. After the cell category is determined by the deep learning module, the overall cell quantification result corresponding to the endoscopic cell image can be accurately and efficiently obtained according to the cell outline area of ​​each cell, and deep feature extraction is performed on abnormal cells, and accurate abnormal cell quantification results are obtained by combining the deep features and cell outline features of abnormal cells. In this way, cell abnormalities can be accurately and efficiently determined, effectively assisting clinicians in judging whether there are tumor lesions in the region of interest.

[0082] In an exemplary embodiment, as shown in FIG2 , the overall cell quantification result corresponding to the endoscopic cell image is determined according to the cell contour area of ​​each cell by a deep learning model, including:

[0083] Step 202, determining the number of cells in the endoscopic cell image according to the number of cell outline regions.

[0084] Step 204: Acquire the image area of ​​the endoscopic cell image through the deep learning model, and determine the cell density of the endoscopic cell image according to the image area and the number of cells.

[0085] Step 206 , obtaining the overall cell quantification result corresponding to the endoscopic cell image according to the cell quantity and cell density of the endoscopic cell image.

[0086] The number of cell contour areas separated by the deep learning model is counted, and the number is determined to be the number of cells in the endoscopic cell image. The image area of ​​the endoscopic cell image can also be obtained through the deep learning model, and the ratio of the number of cells to the image area of ​​the endoscopic cell image is calculated as the cell density of the endoscopic cell image. The number of cells and cell density corresponding to the endoscopic cell image are compared with the number of cells and density of cells in the normal region of interest. If the comparison is inconsistent, it is determined that there is a lesion in the region of interest.

[0087] In this embodiment, by performing overall quantitative analysis on cells in the endoscopic cell image, it is possible to effectively assist clinicians in determining whether there is a tumor lesion in the target area.

[0088] In an exemplary embodiment, the cell contour area corresponding to each cell is classified through a deep learning model to obtain the cell category of each cell, including: denoising the cell contour area corresponding to each cell through a deep learning model; performing threshold segmentation on the denoised cell contour area; performing grayscale analysis on the segmented cell contour area to obtain the fluorescence signal intensity of each cell; and determining the cell category corresponding to each cell according to the fluorescence signal intensity of each cell.

[0089] Specifically, the denoising process can be an image preprocessing method known to those skilled in the art, which is not limited here. The denoising process is used to remove noise points in the cell outline area. The denoised cell outline area is subjected to threshold segmentation, and the cells in the cell outline area are separated from the background, so that the cell outline area presents an obvious black and white effect. Among them, threshold segmentation can also be called binarization. Afterwards, the grayscale values ​​of the fluorescent points in the segmented cell outline area are calculated to obtain the grayscale values ​​of the fluorescent points, and the average grayscale value of the fluorescent points belonging to the same cell is calculated. According to the relationship between the grayscale value and the fluorescence signal intensity and the average grayscale value of the fluorescent points in each cell, the fluorescence signal intensity of each cell is determined. The fluorescence signal intensity of each cell is compared with the fluorescence signal intensity threshold interval of cells of different cell categories to determine the cell category corresponding to each cell.

[0090] In this embodiment, since the cells in the region of interest are stained, the endoscopic cell image collected by the endoscope is a fluorescence signal image. By denoising the cell contour area corresponding to each cell, the influence of noise points can be avoided, and the denoised cell contour area is threshold segmented. Thereafter, the cell contour area is gray-scale analyzed to obtain the fluorescence signal intensity of the cells. Threshold segmentation can separate the cells in the cell contour area from the background, so that the cell contour area presents an obvious black and white effect, which is beneficial to improve the accuracy of the cell fluorescence signal intensity, thereby accurately identifying the cell type of each cell.

[0091] In an exemplary embodiment, as shown in FIG3 , deep features are extracted from the cell contour region of the cell category of abnormal cells by a deep learning model, and the abnormal cell quantification result is determined according to the deep features and cell contour features of the abnormal cells, including:

[0092] Step 302: Perform deep feature extraction on the cell contour area of ​​the cell category of abnormal cells through a deep learning model.

[0093] Step 304: Fusing the depth features and cell contour features of the abnormal cells to obtain fused features.

[0094] Step 306: classify the abnormal cells according to the fused features to determine the quantification result of the abnormal cells.

[0095] The deep learning model can also extract the deep features corresponding to the cell contour area of ​​abnormal cells. Deep features include texture features and morphological features, which are used to characterize and identify cell structure and morphology. Using dimensionality reduction techniques, such as principal component analysis, to map deep features and cell contour features to a unified space and fuse the mapped features, can reduce feature dimensions, remove redundant information, and improve the efficiency of the deep learning model. Cell contour features characterize the size and morphology of cells, while deep features can characterize both the morphology of cells and their structure. The fused features are used to classify abnormal cells to obtain the quantification results of abnormal cells.

[0096] Furthermore, the depth feature and the cell contour feature may be weightedly fused to obtain the fused feature.

[0097] In this embodiment, the depth features are fused with the cell contour features, and the cells are graded based on the fused features, so that the pathological level of abnormal cells can be identified from multiple features such as cell size, morphology, and structure.

[0098] In an exemplary embodiment, a pre-built deep learning model includes a target detection subnetwork and an enhancement module; the cell contour features of the endoscopic cell image corresponding to the region of interest are extracted through the pre-built deep learning model, and the cell contour regions corresponding to each cell in the endoscopic cell image are separated according to the cell contour features, including: extracting features of the endoscopic cell image corresponding to the region of interest through the target detection subnetwork to obtain an original feature map; enhancing pixels of the original feature map through the enhancement module to obtain an enhanced feature map; extracting cell contour features in the enhanced feature map through the target detection subnetwork.

[0099] Specifically, the pre-built deep learning model includes a target detection subnetwork and an enhancement module. The target detection subnetwork can be a lightweight Mask R-CNN. The enhancement module can be an attention module or an Enhance-Block module. The target detection subnetwork includes a lightweight network, FPN (Feature Pyramid Networks) and RPN (Region Proposal Network).

[0100] The endoscopic cell image is input into the target detection subnetwork in the pre-built deep learning model. The lightweight network and FPN network in the target detection subnetwork are used to extract features from the endoscopic cell image to obtain the original feature map. The original feature map is input into the enhancement module, and the original feature map is enhanced by the enhancement module. The enhanced feature map is then sent to the RPN network to obtain the cell contour features in the enhanced feature map.

[0101] Furthermore, the method of enhancing the pixels of the original feature map by the enhancement module includes: performing global average pooling on each channel of the original feature map to obtain the pooling mean corresponding to each channel, thereby obtaining a first vector. The first vector is mapped to the (0,1) interval using a sigmoid function after passing through two fully connected layers to obtain multiple channel mapping values, each of which corresponds to a channel. The pixel points of each channel are multiplied by the channel mapping value of each channel to obtain an intermediate feature map.

[0102] For each pixel in the intermediate feature map, calculate the mean of all channel mapping values ​​of each pixel to obtain the channel mean corresponding to each pixel. The channel mean of all pixels in the intermediate feature map forms the second vector. After the second vector is expanded and sent to the two-layer full connection, it is mapped to the (0,1) interval using the sigmoid function to obtain multiple pixel mapping values, each of which corresponds to a pixel position. The pixel mapping value of each pixel is multiplied by the corresponding pixel of the intermediate feature map according to the position correspondence to obtain the enhanced feature map.

[0103] In this embodiment, the pixel points of the original feature map are enhanced by the enhancement module to enhance the spatial information, which is beneficial to the regression of the bounding box and thus to the extraction of the cell contour features.

[0104] In an exemplary embodiment, the target detection subnetwork is determined in the following manner: obtaining the channel factor of each network channel in the lightweight network; taking the network channel whose channel factor is less than a preset threshold as the channel to be cropped; cropping the channel to be cropped in the lightweight network, replacing the residual network in the Mask R-CNN network with the lightweight network after the cropping process, and obtaining the target detection subnetwork.

[0105] Among them, the channel factor of each network channel can be a scale factor of each network channel, which is a dynamically changing weight used to indicate the importance of the network channel corresponding to the scale factor to the network output. The lightweight network includes a starting part, a middle part and a final part, each of which includes multiple network channels. Each network channel is used for feature extraction.

[0106] Specifically, the original neural network is obtained. The original neural network is a traditional lightweight network and does not introduce a scaling factor. A scaling factor is introduced for each network channel of the original neural network. For example, the initial value of the introduced scaling factor can be 0.6. A loss function containing the scaling factor of each network channel is obtained, and the loss function is trained by multiple back propagation. When the number of training iterations reaches a preset number or the loss value of the loss function no longer decreases, the network training is stopped to obtain a lightweight network. Since the scaling factor contained in the loss function will also change dynamically during the network training process, when the network training stops, the scaling factor of each network channel in the lightweight network can be obtained.

[0107] Compare the scaling factor with the preset threshold to determine the network channel whose scaling factor is less than the preset threshold. The preset threshold can be a value close to zero. If the scaling factor of a channel is less than the preset threshold, it means that the channel is less important to the network output, the feature map output by the channel can be ignored, and pruning the channel will not affect the network performance. The network channel whose scaling factor is less than the preset threshold is regarded as the channel to be pruned, and pruned, so as to obtain a lightweight network after pruning.

[0108] Then, the pruned lightweight network is used to replace the residual network in the Mask R-CNN network to obtain the target detection subnetwork. By pruning unimportant network channels and reducing the number of network channels, the computational complexity of target detection can be reduced.

[0109] In this embodiment, by further lightweighting the lightweight network, the model complexity can be further reduced, thereby greatly improving the model reasoning speed.

[0110] In an exemplary embodiment, the method also includes: obtaining an endoscopic cell image corresponding to the region of interest through the endoscope handle of the endoscope; the endoscope handle includes: a lens module; an optical fiber pressure plate, including a pressure plate groove, fixed to the outer wall of the lens module; an illumination module, including an optical fiber cable and an optical fiber connector; wherein the optical fiber cable is arranged inside the endoscope handle, passes through the pressure plate groove, is connected to one end of the optical fiber connector and extends to the other end; the other end of the optical fiber connector is fixedly connected to the lens module; the optical fiber cable and the optical fiber connector are used to provide incident light to the lens module, so that the lens module obtains the endoscopic cell image corresponding to the region of interest.

[0111] Specifically, the endoscope can also generate endoscopic cell images through the endoscope handle. The optical fiber cable is arranged inside the handle, and the optical fiber cable enters the handle from the tail of the handle, passes through the pressing plate groove of the optical fiber pressing plate, and is connected to one end of the optical fiber connector to guide the light emitted by the light source into the lens module. The middle end of the optical fiber pressing plate is a pressing plate groove, and both sides of the optical fiber pressing plate are fixedly connected to the lens module.

[0112] The optical fiber cable passes through the interior of the optical fiber connector and comes out from the other end. The interior of the optical fiber connector is set as a multi-corner cavity, which can bend the optical fiber cable 90 degrees at a small angle and multiple times, so that the optical fiber cable can guide the light emitted by the light source into the lens module. The other end of the optical fiber connector is connected to the lens module. After the optical fiber cable is bent 90 degrees in the optical fiber connector, the illumination light emitted by the transmitted light source is also bent 90 degrees and enters the lens module to provide incident light for the endoscope to generate endoscopic cell images.

[0113] In this embodiment, by setting a fiber optic pressure plate, the routing position of the fiber optic cable is limited, which prevents the fiber optic cable from shaking inside the handle, affecting the transmission and incident angle of light, and improving the stability of the lighting function of the endoscope handle. The fiber optic connector uses the multi-corner cavity inside it to bend the optical fiber to 90 degrees without loss, which solves the light loss caused by long-term large-angle bending of the optical fiber cable and avoids damage to the optical fiber caused by the inner wall corner. The multi-corner cavity can effectively alleviate the angle and reduce light loss. By arranging the lens module and the lighting module as a whole, the overall volume of the endoscope handle is reduced, the orderliness of the internal arrangement of the endoscope handle is improved, and the positional relationship of the optical fiber cable inside the endoscope handle is optimized, which improves the convenience of using the endoscope and the stability of its function.

[0114] Furthermore, the endoscope handle also includes a shell, which includes a handle middle shell and a handle rear shell; a fixing part is arranged on the inner wall of the handle middle shell, and the fixing part includes a fixing groove, a first screw hole of the fixing part and a second screw hole of the fixing part; the fixing groove is used to fix the circuit module; the handle rear shell includes a first screw hole of the rear shell, and the first screw hole of the rear shell and the second screw hole of the fixing part are fixedly connected by a first screw, the handle middle shell abuts against the handle rear shell, and the nut of the first screw abuts against the handle rear shell.

[0115] A fixing piece is disposed on the inner wall of the handle middle shell, and the fixing piece is disposed on the inner wall of the second end of the handle middle shell. The fixing piece includes a first fixing piece and a second fixing piece, and the first fixing piece and the second fixing piece are respectively located on both sides of the second end of the handle middle shell. Exemplarily, the first fixing piece includes a fixing groove, a first screw hole of the fixing piece, and a second screw hole of the fixing piece. The two ends of the circuit board in the circuit module are respectively connected to the fixing grooves in the first fixing piece and the second fixing piece, and the circuit module is fixed in the handle middle shell, so that the circuit module is fixed in the middle, and space is left at the upper and lower ends of the handle middle shell.

[0116] The handle rear shell includes a rear shell first screw hole, and the handle middle shell and the handle rear shell are abutted by the rear shell first screw hole. Exemplarily, the second screw hole of the first fixing member is fixedly connected to the first screw hole of the rear shell by a first screw, the handle middle shell abuts against the handle rear shell, the nut of the first screw abuts against the handle rear shell, and the second fixing member fixes the handle middle shell and the handle rear shell in the same way.

[0117] The endoscope handle shell also includes a handle front shell and a handle sheath. The second end of the handle front shell abuts against the first end of the handle middle shell, the first end of the handle front shell is connected to the scope sheath, and a lens module is arranged in the handle front shell; the handle sheath is connected to the handle rear shell, and is used to gather the optical fiber cable, electrical signal cable and ground wire into the endoscope handle.

[0118] In this embodiment, a fixing piece is provided in the handle middle shell to fix the circuit module in the middle, so that there is space between the upper and lower parts of the handle middle shell, which provides reserved space for the optical fiber cable to be routed inside the handle, avoiding direct contact between the optical fiber cable and the circuit module, which affects the transmission of light. The optical fiber cable, the electrical signal cable and the grounding wire are bundled into the handle through the handle sheath, which further limits the routing of the optical fiber cable inside the handle, effectively ensuring the stability of the internal structure of the handle and the transmission quality of light.

[0119] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0120] Based on the same inventive concept, the embodiment of the present application also provides a cell quantification device for implementing the cell quantification method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more cell quantification device embodiments provided below can refer to the limitations on the cell quantification method above, and will not be repeated here.

[0121] In an exemplary embodiment, as shown in FIG4 , a cell quantification device is provided, comprising: a contour extraction module 402, a classification module 404, an overall quantification module 406 and an abnormality quantification module 408, wherein:

[0122] The contour extraction module 402 is used to extract the cell contour features of the endoscopic cell image corresponding to the region of interest through a pre-built deep learning model, and separate the cell contour region corresponding to each cell in the endoscopic cell image according to the cell contour features.

[0123] The classification module 404 is used to classify the cell contour area corresponding to each cell through a deep learning model to obtain the cell category of each cell.

[0124] The overall quantification module 406 is used to determine the overall cell quantification result corresponding to the endoscopic cell image according to the cell contour characteristics and cell contour area of ​​each cell through a deep learning model.

[0125] The abnormality quantification module 408 is used to extract deep features of the cell contour area of ​​the cell category as abnormal cells through a deep learning model, and determine the abnormal cell quantification result according to the deep features and cell contour features of the abnormal cells.

[0126] In an exemplary embodiment, the overall quantification module 406 is also used to determine the number of cells in the endoscopic cell image based on the number of cell contour areas; obtain the image area of ​​the endoscopic cell image through a deep learning model, and determine the cell density of the endoscopic cell image based on the image area and the number of cells; and obtain the overall cell quantification result corresponding to the endoscopic cell image based on the cell number and cell density of the endoscopic cell image.

[0127] In an exemplary embodiment, the classification module 404 is also used to denoise the cell contour area corresponding to each cell through a deep learning model; perform threshold segmentation on the denoised cell contour area; perform grayscale analysis on the segmented cell contour area to obtain the fluorescence signal intensity of each cell; and determine the cell category corresponding to each cell based on the fluorescence signal intensity of each cell.

[0128] In an exemplary embodiment, the abnormality quantification module 408 is also used to extract deep features of the cell contour area of ​​the cell category as abnormal cells through a deep learning model; fuse the deep features and cell contour features of the abnormal cells to obtain fused features; classify the abnormal cells according to the fused features to determine the abnormal cell quantification results.

[0129] In an exemplary embodiment, the pre-built deep learning model includes a target detection subnetwork and an enhancement module; the contour extraction module 402 is also used to extract features of the endoscopic cell image corresponding to the region of interest through the target detection subnetwork to obtain an original feature map; the original feature map is pixel-enhanced through the enhancement module to obtain an enhanced feature map; and the cell contour features in the enhanced feature map are extracted through the target detection subnetwork.

[0130] In an exemplary embodiment, the apparatus further comprises:

[0131] The model building module is used to obtain the channel factor of each network channel in the lightweight network; the network channel whose channel factor is less than the preset threshold is used as the channel to be pruned; the channel to be pruned in the lightweight network is pruned, and the residual network in the Mask R-CNN network is replaced by the pruned lightweight network to obtain the target detection subnetwork.

[0132] In an exemplary embodiment, the apparatus further comprises:

[0133] An image acquisition module is used to acquire an endoscopic cell image corresponding to an area of ​​interest through an endoscope handle of an endoscope; the endoscope handle comprises: a lens module; an optical fiber pressure plate, comprising a pressure plate groove, fixed to the outer wall of the lens module; an illumination module, comprising an optical fiber cable and an optical fiber connector; wherein the optical fiber cable is arranged inside the endoscope handle, passes through the pressure plate groove, is connected to one end of the optical fiber connector and extends to the other end; the other end of the optical fiber connector is fixedly connected to the lens module; the optical fiber cable and the optical fiber connector are used to provide incident light to the lens module, so that the lens module acquires an endoscopic cell image corresponding to the area of ​​interest.

[0134] Each module in the above-mentioned cell quantification device can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0135] In an exemplary embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be shown in FIG5. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. The processor, the memory, and the input / output interface are connected via a system bus, and the communication interface, the display unit, and the input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner may be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a cell quantification method is implemented. The display unit of the computer device is used to form a visually visible picture, which may be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0136] Those skilled in the art will understand that the structure shown in FIG. 5 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0137] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0138] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0139] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0140] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0141] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0142] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0143] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A cell quantification method, characterized in that: The method comprises: Extracting cell contour features of the endoscopic cell image corresponding to the region of interest through a pre-built deep learning model, and separating the cell contour region corresponding to each cell in the endoscopic cell image according to the cell contour features; Classifying the cell contour area corresponding to each cell by the deep learning model to obtain the cell category of each cell; Determining the overall cell quantification result corresponding to the endoscopic cell image according to the cell contour area of ​​each cell through the deep learning model; The deep learning model is used to extract deep features of cell contour areas whose cell categories are abnormal cells, and the quantification results of the abnormal cells are determined according to the deep features and cell contour features of the abnormal cells.

2. The method according to claim 1, characterized in that Determining the overall cell quantification result corresponding to the endoscopic cell image according to the cell contour area of ​​each cell by the deep learning model includes: Determining the number of cells in the endoscopic cell image according to the number of the cell outline areas; Acquiring an image area of ​​the endoscopic cell image through the deep learning model, and determining a cell density of the endoscopic cell image according to the image area and the number of cells; The overall cell quantification result corresponding to the endoscopic cell image is obtained according to the cell number and cell density of the endoscopic cell image.

3. The method according to claim 1, characterized in that The deep learning model is used to classify the cell contour area corresponding to each cell to obtain the cell category of each cell, including: Performing denoising on the cell contour area corresponding to each cell through the deep learning model; Perform threshold segmentation on the denoised cell contour area; Grayscale analysis is performed on the segmented cell contour area to obtain the fluorescence signal intensity of each cell; The cell type corresponding to each cell is determined according to the fluorescence signal intensity of each cell.

4. The method according to claim 1, characterized in that: The deep learning model is used to extract deep features of the cell contour area of ​​the cell category being abnormal cells, and the abnormal cell quantification result is determined according to the deep features and cell contour features of the abnormal cells, including: Performing deep feature extraction on the cell contour area of ​​abnormal cells by using the deep learning model; The depth features and cell contour features of abnormal cells are fused to obtain fused features; The abnormal cells are graded according to the characteristics after fusion to determine the quantification results of the abnormal cells.

5. The method according to claim 1, characterized in that The pre-built deep learning model includes a target detection subnetwork and an enhancement module; The method of extracting cell contour features of the endoscopic cell image corresponding to the region of interest through a pre-built deep learning model, and separating the cell contour region corresponding to each cell in the endoscopic cell image according to the cell contour features, comprises: Extracting features from the endoscopic cell image corresponding to the region of interest through the target detection subnetwork to obtain an original feature map; Performing pixel enhancement on the original feature map by the enhancement module to obtain an enhanced feature map; The cell contour features in the enhanced feature map are extracted through the target detection subnetwork.

6. The method according to claim 5, characterized in that The target detection subnetwork is determined by: Get the channel factor of each network channel in the lightweight network; The network channels whose channel factors are smaller than the preset threshold are regarded as the channels to be pruned; The to-be-cropped channels in the lightweight network are cropped, and the residual network in the Mask R-CNN network is replaced with the cropped lightweight network to obtain a target detection subnetwork.

7. The method according to any one of claims 1 to 6, characterized in that: The method further includes: acquiring an endoscopic cell image corresponding to the region of interest through an endoscope handle of the endoscope; the endoscope handle includes: Lens module; An optical fiber pressing plate, including a pressing plate groove body, fixed to the outer wall of the lens module; A lighting module including a fiber optic cable and a fiber optic connector; Among them, the optical fiber cable is arranged inside the endoscope handle, passes through the pressure plate groove and is connected to one end of the optical fiber connector and extends to the other end; the other end of the optical fiber connector is fixedly connected to the lens module; the optical fiber cable and the optical fiber connector are used to provide incident light to the lens module so that the lens module obtains an endoscopic cell image corresponding to the area of ​​interest.

8. A cell quantification device, characterized in that: The device comprises: A contour extraction module, used to extract the cell contour features of the endoscopic cell image corresponding to the region of interest through a pre-built deep learning model, and separate the cell contour region corresponding to each cell in the endoscopic cell image according to the cell contour features; A classification module, used to classify the cell contour area corresponding to each cell through the deep learning model to obtain the cell category of each cell; An overall quantification module, used to determine the overall cell quantification result corresponding to the endoscopic cell image according to the cell contour characteristics and cell contour area of ​​each cell through the deep learning model; The abnormality quantification module is used to extract deep features of the cell contour area of ​​the cell category as abnormal cells through the deep learning model, and determine the abnormal cell quantification result according to the deep features and cell contour features of the abnormal cells.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images