Cell imaging system with cell recognition and counting functions for semi-automatic microscope

Abstract

The invention relates to the technical field of medical diagnosis, in particular to a cell imaging system with cell recognition and counting functions for a semi-automatic microscope. The system comprises an image recognition module which is adaptively connected with a camera and automatically captures the clearest image in cell images continuously shot by the camera through an algorithm; the real-time module is connected with the image recognition module and is used for marking, classifying and counting cells in the cell image; the storage module is used for storing the cells selected by the real-time module to form a cell classification map, so that the classified cells can be conveniently checked; and the classification report module is used for selecting a cell classification chart needing to be displayed, editing and describing cell morphology and combining cell counting to generate a cell morphology report. The method solves the problem that doctors in primary and intermediate hospitals lack professional cell classification ability and influence the diagnosis accuracy; and the workload of doctors in large and medium-sized hospitals is heavy.

Description

Technical Field

[0001] The invention relates to the technical field of medical diagnosis, in particular to a cell imaging system for a semi-automatic microscope with cell identification and counting functions. Background Art

[0002] In the traditional medical diagnosis process, cell classification and counting usually need to be done manually by doctors with professional skills through a microscope. This process is time-consuming and easily affected by subjective judgment. For grassroots and intermediate hospitals, the lack of professional cell classification capabilities limits the accuracy of diagnosis. At the same time, in medium and large hospitals, laboratory doctors need to operate the cell counter manually when identifying cells, which makes the work process cumbersome and inefficient. Summary of the invention

[0003] In order to solve the problems in the related art, the present invention provides a semi-automatic microscope cell imaging system with cell recognition and counting functions. This method solves the problem that doctors in primary and intermediate hospitals lack professional cell classification capabilities, which affects the accuracy of diagnosis; and the problem that doctors in medium and large hospitals have a heavy workload.

[0004] In order to solve the above problems, the following technical solutions are provided:

[0005] The semi-automatic microscope cell imaging system with cell identification and counting functions of the present invention comprises:

[0006] An image recognition module is adapted and connected to the camera, and determines the clarity value of each frame of the image continuously taken by the camera through an algorithm, selects the frame of the image with the largest clarity value, and stores the cell image in the image recognition module, wherein the cell image is a cell image containing stained and marked cells;

[0007] A real-time module, which is connected to the image recognition module and is used to receive the cell image stored in the image recognition module, select the cells marked by dyeing in the cell image, and classify and count the selected cells;

[0008] A saving module, which is connected to the image recognition module and the real-time module, and is used to save the cells selected by the real-time module to form a cell classification map, so as to facilitate viewing of all classified cells;

[0009] The classification and reporting module is connected to the saving module, and the cell classification diagram to be displayed is selected, and the cell morphology is edited and described, and the cell morphology report can be generated in combination with the cell counting.

[0010] Through the above scheme, the image recognition module is used to convert the cell smear into a clear cell image, and the real-time module classifies and counts the cells in the cell image, thereby improving the cell classification ability of primary and intermediate hospitals and reducing the dependence on professional doctors, thereby solving the problem that doctors in primary and intermediate hospitals lack professional cell classification ability and affect the accuracy of diagnosis, improving the work efficiency of the laboratory of medium and large hospitals, reducing the workload of doctors, thereby solving the problem of heavy workload of doctors in medium and large hospitals. By identifying, classifying and counting cell images, it helps to improve the accuracy and efficiency of diagnosis.

[0011] The algorithm judgment comprises the following steps:

[0012] S1: Image scaling processing, the image scaling processing is used to scale the input image to maintain the overall features of the image and obtain an image of a smaller size;

[0013] S2: Image similarity judgment, which is used to extract image features and calculate the similarity between two images;

[0014] S3: Grayscale conversion, used to convert the color input image into a grayscale image, use the Laplace operator to process the grayscale image, and calculate the gradient change of the image. The Laplace operator is a second-order derivative filter that can highlight the edge and high-frequency information in the image. The processed Laplace image contains the edge and gradient change information in the image. The formula is:

[0015]

[0016] Where f(x,y) represents the input grayscale image;

[0017] S4: Quantification of clarity. The gradient of the processed Laplacian image is calculated by L2 norm. L2 norm is used to measure the strength of edge and gradient information in the image. A higher L2 norm value indicates that the image contains more high-frequency information, which means that the image is clearer. Conversely, a lower L2 norm value indicates that the image has less edge information and is more blurred. The formula is:

[0018]

[0019] Among them, Laplacian ij is the value of pixel position (i,j) in the Laplacian image.

[0020] The specific process of scaling the input image in S1 is:

[0021] The scaled size is determined by multiplying the width and height of the original image by the scaling factor val, and the formula is:

[0022] W′=W×val, H′=H×val,

[0023] Where W is the width of the original image, H is the height of the original image, W′ is the width after scaling, and H′ is the height after scaling.

[0024] The specific image features extracted from S2 are:

[0025] S2-1: extracting a feature function, wherein the feature function is used to extract a feature vector from an input image for image similarity comparison;

[0026] Extract HOG features: Create a HOG descriptor object. The HOG feature is used to capture the edge information of the image. The HOG feature of the image is calculated through the HOG descriptor and the HOG feature is stored in the feature vector.

[0027] Return value: Returns a feature vector containing HOG features for image similarity calculation;

[0028] The similarity between two images is calculated in S2 as follows:

[0029] S2-2: Calculate a similarity function, where the similarity function is used to calculate the similarity between two image feature vectors;

[0030] Calculate the similarity of feature vectors: Use the correlation in the histogram comparison method to measure the linear relationship between two feature vectors. The closer the value is to 1, the higher the similarity.

[0031] Return value: Returns the similarity value between two feature vectors, usually in the range of [-1,1], 1 means complete similarity, -1 means complete dissimilarity;

[0032] The real-time module can mark primitive cells, abnormal promyelocytes, schistocytes and malarial parasites and give reminders.

[0033] The cell images stored in the image recognition module can be zoomed in and out to allow doctors to view local cell distribution.

[0034] Through the above scheme, doctors can view the conditions of local cells in cell images.

[0035] The classification and reporting module can provide an editing interface for doctors to review and adjust the results of classification and counting of the real-time module.

[0036] Through the above scheme, doctors review and adjust the results of classification and counting of the real-time module, which is conducive to improving the accuracy of the final results, and doctors can give diagnostic opinions based on the results of classification and counting.

[0037] The camera is used to be installed on a microscope to take real-time photos of the cell smear.

[0038] In the above scheme, the camera is set to take real-time photos of the cell smear and transmit the photos to the image recognition module, so that the image recognition module can receive the cell images in real time.

[0039] The real-time module is adapted to be connected with the hospital information system to realize automatic entry and analysis of cell classification and counting.

[0040] The above solution has the following advantages:

[0041] The semi-automatic microscope cell imaging system with cell recognition and counting functions of the present invention is adopted, the image recognition module is used to convert the cell smear into a clear cell image, and the real-time module classifies and counts the cells in the cell image, thereby improving the cell classification ability of primary and intermediate hospitals, reducing the dependence on professional doctors, and improving the accuracy of diagnosis; improving the work efficiency of the laboratory of medium and large hospitals, reducing the workload of doctors, and helping to improve the accuracy and efficiency of diagnosis by recognizing, classifying and counting cell images. DETAILED DESCRIPTION

[0042] The following will be combined with the specific embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] In specific embodiment 1, the semi-automatic microscope cell imaging system with cell recognition and counting functions of the present invention comprises:

[0044] The image recognition module is adapted to be connected with the camera. The camera is used to be installed on the microscope. When the microscope is manually adjusted to observe cells, the camera continuously shoots the cell smear, which is a stained cell smear; the clarity value of each frame of the image continuously shot by the camera is judged by an algorithm. The clarity judgment algorithm is an image clarity evaluation method based on the Laplace operator. The specific process of the algorithm judgment is: after scaling the input image, the gradient change of the image edge is calculated using the second-order derivative of the Laplace operator, and the gradient response is further quantified by the L2 norm to obtain the clarity evaluation index of the image, and the frame of the image with the largest clarity value is captured from the continuous image to obtain the clearest cell image, and the cell image is stored in the image recognition module; the cell image saved by the image recognition module can be scaled for doctors to view the local cell distribution;

[0045] A real-time module, which is connected to the image recognition module and is used to receive the cell image saved by the image recognition module. The cells to be observed are stained and marked by staining, the marked cells are selected by a cell segmentation algorithm, the marked cells are classified by a cell classification algorithm, and the classified cells are counted. The specific process is to use a deep learning target detection algorithm to identify and locate the cell area, output the cell boundary box according to the position and size of the cell, crop a single cell image of each cell according to the detected cell boundary box, and use a deep learning classification model to classify the single cell image and identify its type; the real-time module can also mark primitive cells, abnormal promyelocytes, schistocytes and malarial parasites, and give reminders;

[0046] A saving module, which is connected to the image recognition module and the real-time module, and is used to save the cells selected by the real-time module to form a cell classification map, so as to facilitate viewing of all classified cells;

[0047] The classification and reporting module is connected to the saving module. The cell classification diagram to be displayed is selected, and the cell morphology is edited and described. In combination with the cell counting, a cell morphology report can be generated. The classification and reporting module can provide an editing interface for doctors to review and adjust the results of the real-time module classification and counting.

[0048] The algorithm judgment includes the following steps: S1 image scaling processing, S2 image similarity judgment, 3 grayscale conversion, S4 clarity quantification, the specific process is:

[0049] The image scaling processing step in S1 specifically includes: scaling the input image by the scaling factor val to maintain the overall features of the image and obtain an image of a smaller size, which can reduce the computational complexity.

[0050] The target size of the scaling is determined by multiplying the width and height of the original image by the scaling factor val, and the formula is:

[0051] W′=W×val, H′=H×val,

[0052] Where W is the width of the original image, H is the height of the original image, W′ is the width after scaling, H′ is the height after scaling, and val is the scaling factor;

[0053] The steps of image similarity judgment in S2 specifically include: extracting image features and calculating the similarity of two images, which is usually used for image retrieval, image classification or similarity comparison tasks;

[0054] S2-1: Extract feature function, which is used to extract feature vectors from the input image for image similarity comparison;

[0055] Extract HOG features: Create a HOG descriptor object. The HOG feature is used to capture the edge information of the image. The HOG feature of the image is calculated through the HOG descriptor and the HOG feature is stored in the feature vector.

[0056] Return value: Returns a feature vector containing HOG features for image similarity calculation;

[0057] S2-2: Calculate the similarity function of the two images, where the similarity function is used to calculate the similarity between the feature vectors of the two images;

[0058] Calculate the similarity of feature vectors: Use the correlation in the histogram comparison method to measure the linear relationship between two feature vectors. The closer the value is to 1, the higher the similarity.

[0059] Return value: Returns the similarity value between two feature vectors, usually in the range of [-1,1], 1 means complete similarity, -1 means complete dissimilarity;

[0060] The grayscale conversion steps in S3 specifically include: if the input image is a color image, the image is first converted to a grayscale image to simplify subsequent calculations. The conversion formula uses the function in OpenCV to convert the channel to a single-channel grayscale image. The grayscale image is processed using the Laplace operator to calculate the gradient change of the image. The Laplace operator is a second-order derivative filter that can highlight the edge and high-frequency information in the image. The processed Laplace image contains the edge and texture change information in the image. The formula is:

[0061]

[0062] Where f(x,y) represents the input grayscale image;

[0063] The steps of clarity quantification in S4 specifically include: calculating the L2 norm of the gradient response of the Laplacian image, that is, the square root of the sum of the squares of all pixel values. The L2 norm can be used to measure the strength of edge and gradient information in the image. A higher L2 norm value indicates that the image contains more high-frequency information, indicating that the image is clearer. Conversely, a lower L2 norm value indicates that the image has less edge information and the image is more blurred. The formula is:

[0064]

[0065] Among them, Laplacian ij is the value of pixel position (i,j) in the Laplacian image.

[0066] The classification and reporting module is adapted to be connected with the hospital information system and is used to automatically import the results of cell classification and counting into the hospital information system.

[0067] In this embodiment, the cell morphology of the blood smear is checked. First, the blood is pushed and stained to form a blood smear. The microscope is manually adjusted to observe the cells in the blood smear. The image recognition module stores the cell picture with the highest definition. The real-time detection function in the real-time module is turned on. At this time, the cell image in the field of view will be marked. The microscope is manually operated to switch the field of view to obtain a new cell image. The real-time module automatically detects the cell type in the cell image and selects and marks it, and classifies and counts the marked cells. It can also mark primitive cells, abnormal promyelocytes, fragmented red blood cells and malarial parasites, and give reminders; turn on the detection function in the preservation module, and turn off the real-time detection function at the same time. The cell images converted by the image recognition module are stored in the preservation module. The preservation module is also used to save the cells selected by the real-time module to form a cell classification map, so that the cell image and all the cell classification maps that have been classified can be viewed, wherein the cell image can be zoomed in and out to facilitate viewing the local cell distribution; enter the cell classification map, select the map to be displayed, enter the relevant information of the sample and the patient, edit the cell morphology description, and a complete cell morphology report can be generated.

[0068] In the specific embodiment 2, the difference between this embodiment and embodiment 1 is that this embodiment can also be used to examine the cell morphology of body fluid smears, the cell morphology of bone marrow smears, and the cell morphology of smears similar to blood smears.

[0069] Obviously, the above embodiments are merely examples for clear explanation and are not limitations on the implementation methods. For ordinary technicians in the relevant field, other different forms of changes or modifications can be made on the basis of the above description. It is not necessary and impossible to list all the implementation methods here, and the obvious changes or modifications derived therefrom are still within the protection scope of the invention.

Claims

1. A semi-automatic microscope cell imaging system with cell recognition and counting functions, characterized in that: include: An image recognition module is adapted and connected to the camera, and determines the clarity value of each frame of the image continuously taken by the camera through an algorithm, selects the frame of the image with the largest clarity value, and stores the cell image in the image recognition module, wherein the cell image is a cell image containing stained and marked cells; A real-time module, which is connected to the image recognition module and is used to receive the cell image stored in the image recognition module, select the cells marked by dyeing in the cell image, and classify and count the selected cells; A saving module, which is connected to the image recognition module and the real-time module, and is used to save the cells selected by the real-time module to form a cell classification map, so as to facilitate viewing of all classified cells; The classification and reporting module is connected to the saving module, and the cell classification diagram to be displayed is selected, and the cell morphology is edited and described, and the cell morphology report can be generated in combination with the cell counting.

2. The semi-automatic microscope cell imaging system with cell recognition and counting functions as claimed in claim 1, characterized in that: The algorithm judgment comprises the following steps: S1: Image scaling processing, the image scaling processing is used to scale the input image to maintain the overall features of the image and obtain an image of a smaller size; S2: Image similarity judgment, which is used to extract image features and calculate the similarity between two images; S3: Grayscale conversion, used to convert the color input image into a grayscale image, use the Laplace operator to process the grayscale image, and calculate the gradient change of the image. The Laplace operator is a second-order derivative filter that can highlight the edge and high-frequency information in the image. The processed Laplace image contains the edge and gradient change information in the image. The formula is: Where f(x,y) represents the input grayscale image; S4: Quantification of clarity. The gradient of the processed Laplacian image is calculated by L2 norm. L2 norm is used to measure the strength of edge and gradient information in the image. A higher L2 norm value indicates that the image contains more high-frequency information, which means that the image is clearer. Conversely, a lower L2 norm value indicates that the image has less edge information and is more blurred. The formula is: Among them, Laplacian ij is the value of pixel position (i,j) in the Laplacian image.

3. The semi-automatic microscope cell imaging system with cell recognition and counting functions as claimed in claim 2, characterized in that: The specific process of scaling the input image in S1 is: W′=W×val, H′=H×val, Among them, W is the width of the original image, H is the height of the original image, W′ is the width after scaling, H′ is the height after scaling, and val is the scaling factor.

4. The semi-automatic cell imaging system for a microscope with cell recognition and counting functions as claimed in claim 2, characterized in that: The specific image features extracted from S2 are: S2-1: extracting a feature function, wherein the feature function is used to extract a feature vector from an input image for image similarity comparison; Extract HOG features: Create a HOG descriptor object. The HOG feature is used to capture the edge information of the image. The HOG feature of the image is calculated through the HOG descriptor and the HOG feature is stored in the feature vector. Return value: Returns a feature vector containing HOG features for image similarity calculation; The similarity between two images is calculated in S2 as follows: S2-2: Calculate a similarity function, where the similarity function is used to calculate the similarity between two image feature vectors; Calculate the similarity of feature vectors: Use the correlation in the histogram comparison method to measure the linear relationship between two feature vectors. The closer the value is to 1, the higher the similarity. Return value: Returns the similarity value between two feature vectors, usually in the range of [-1,1], where 1 indicates complete similarity and -1 indicates complete dissimilarity.

5. The semi-automatic microscope cell imaging system with cell recognition and counting functions as claimed in claim 1, characterized in that: The real-time module can mark primitive cells, abnormal promyelocytes, schistocytes and malarial parasites and give reminders.

6. The semi-automatic microscope cell imaging system with cell recognition and counting functions as claimed in claim 1, characterized in that: The cell images stored in the image recognition module can be zoomed in and out to allow doctors to view local cell distribution.

7. The semi-automatic cell imaging system for a microscope with cell identification and counting functions as claimed in claim 1, characterized in that: The classification and reporting module can provide an editing interface for doctors to review and adjust the results of real-time module classification and counting.

8. The semi-automatic cell imaging system for a microscope with cell identification and counting functions as claimed in claim 1, characterized in that: The camera is used to be installed on a microscope, and the microscope is manually adjusted to observe the cell smear, and the camera continuously shoots the cell smear.

9. The semi-automatic cell imaging system for a microscope with cell recognition and counting functions as claimed in claim 1, characterized in that: The classification and reporting module is adapted to be connected with the hospital information system and is used to import the results of cell classification and counting into the hospital information system.

Images