Blood cell morphological analysis system based on artificial intelligence

Abstract

The invention discloses a blood cell morphological analysis system based on artificial intelligence. The blood cell morphological analysis system comprises a preprocessing unit, a detection unit, a statistical unit and a report generation unit. According to the invention, the preprocessing unit cuts the digital image through a plurality of parallel cutting paths and carries out standardization processing by adopting a multi-thread parallel processing mode, and the detection unit detects the cell category by adopting the multi-thread parallel processing mode, so that the microscopic examination speed is greatly improved; the whole flow of microscopic examination, statistics, cell morphology analysis and analysis report issuing of the whole blood cell digital image can be automatically completed by combining the statistical unit and the report generation unit; and meanwhile, the report generation unit adopts a visual language large model, so that the cell morphology analysis efficiency and accuracy can be greatly improved.

Description

Technical Field

[0001] The present invention belongs to the field of cell morphology analysis, and in particular relates to a blood cell morphology analysis system based on artificial intelligence. Background Art

[0002] The morphological examination of peripheral blood and other blood cells is a key means of screening for blood diseases, which mainly includes observing the morphology and quantity of peripheral blood and other blood cells. Microscopic examination is the "gold standard" of observation, and a diagnosis report is issued based on the morphology and quantity. In the prior art, for a whole blood smear, the four stages of microscopic examination, statistical examination results, cell morphological analysis, and issuance of analysis reports are usually completed manually, which is time-consuming and labor-intensive, and prone to missed detections. Although some processes such as microscopic examination and statistics in the above four stages have been automatically completed by the system, morphological analysis and issuance of analysis reports are still completed manually, and the automation of the entire process has not been achieved; and the microscopic examination and statistical processes use a single-threaded approach, which is less efficient. Summary of the Invention

[0003] In view of the above-mentioned deficiencies in the prior art, the technical problem to be solved by the present invention is to provide a blood cell morphology analysis system based on artificial intelligence.

[0004] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0005] A blood cell morphology analysis system based on artificial intelligence, comprising

[0006] A preprocessing unit is used to perform parallel cutting of the input blood cell digital image using multiple parallel cutting paths to obtain multiple preprocessed images, and to perform standardization processing on the preprocessed images obtained by cutting each parallel cutting path using a multi-threaded method to obtain a standardized image;

[0007] A detection unit, configured to perform cell location and identification on standardized images based on a cell detection model using a multi-threaded parallel processing method;

[0008] A statistical unit, used to count the number, proportion and reference value of each type of cells based on the recognition results; and

[0009] The report generation unit is used to generate a cell morphology analysis report based on the statistical results based on the pre-trained visual language model.

[0010] Furthermore, the pre-processing unit includes

[0011] A graph cutting algorithm module is used to determine multiple parallel graph cutting paths using a parallel cutting algorithm;

[0012] An image cutting module, configured to use multiple image cutting threads to simultaneously cut the blood cell digital image in parallel to obtain multiple pre-processed images; each of the image cutting threads corresponds to a parallel cutting path; and

[0013] The standardization module is used to use multiple standardization processing threads to simultaneously perform standardization processing on the pre-processed images obtained by each cutting thread to obtain a standardized image.

[0014] Furthermore, the method for determining the parallel cutting path using the parallel cutting algorithm includes the following steps:

[0015] S110, determining a cutting resolution of the blood cell digital image;

[0016] S120, dividing the blood cell digital image into a plurality of blocks to be cut in a row and column manner according to the resolution of the blood cell digital image;

[0017] S130, sequentially connecting the cut pieces in series along a predetermined route to form a cut queue, wherein each element of the cut queue includes position information of a cut piece;

[0018] S140 , dividing the cutting queue into a corresponding number of cutting sub-queues according to the number of image cutting threads, and the cutting path of each cutting sub-queue is a parallel image cutting path.

[0019] Furthermore, each element in the cutting queue includes the coordinates of the upper left corner of the corresponding cut piece in the blood cell digital image; the method for calculating the coordinates of the upper left corner of the cut piece in the blood cell digital image includes the following steps:

[0020] S131. Calculate the number of horizontal and vertical cuts of the digital image of blood cells. The calculation formula for the number of cuts is as follows:

[0021]

[0022]

[0023] Where w×h represents the resolution of the blood cell digital image; w1×h1 represents the cutting resolution; n x Indicates the number of transverse cuts; n y Indicates the number of longitudinal cuts;

[0024] S132, calculate the coordinates (x i,j ,y i,j )=(i×w1,j×h1); where i represents the horizontal index of the slice, i∈[0,n x-1]; j represents the vertical index of the slice, j∈[0,n y -1].

[0025] Furthermore, the detection unit pre-trains the CNN deep neural network by collecting data to obtain a cell detection model, and detects the standardized image based on the pre-trained CNN deep neural network.

[0026] Furthermore, the report generating unit includes

[0027] Knowledge base module, used to store knowledge vector data;

[0028] The knowledge query module is used to convert the statistical results output by the statistical unit into vectors, and obtain the corresponding text knowledge by querying the knowledge base module by calculating the similarity of the vectors;

[0029] The prompt word module is used to provide inference prompt words of input information to the morphological analysis model;

[0030] A morphological analysis module, configured to use the textual knowledge queried by the knowledge query module and the inference prompt words provided by the prompt word module as input information for a morphological analysis model, so that the morphological analysis model outputs the task content required by the inference prompt words; the morphological analysis model is obtained by pre-training a large visual language model; and

[0031] The report generation module is used to fill the output content of the morphological analysis module into the report template to generate a morphological analysis report.

[0032] Furthermore, the method for generating a knowledge base module includes the following steps:

[0033] S210. Collect expert consensus, literature, and authoritative books on the diagnosis of blood cell morphology diseases in advance and organize them into electronic documents;

[0034] S220, pre-trained visual language embedding model;

[0035] S230. Extract knowledge from the electronic document through the trained visual language embedding model and convert it into vector data for storage to obtain a knowledge base module.

[0036] Furthermore, in the step S230, the knowledge extracted from the electronic document is an image-text pair, where the image in the image-text pair is a blood cell image, and the text is a text description of the blood cells in the image.

[0037] Furthermore, the formula for calculating the similarity of vectors in the knowledge query module is:

[0038]

[0039] Among them, e i Represents the normalized vector of statistical results; e j Represents a knowledge vector in the vector database; sim cos (e i ,e j ) means e i and e j The cosine similarity between .

[0040] Furthermore, the method for determining the inference prompt word of the prompt word module includes the following steps:

[0041] S310, setting the role of the visual language model;

[0042] S320: Determine the prompt content of the character based on the character of the visual language large model and the output content of the statistical unit and the knowledge query module, and generate inference keywords of the known information based on the prompt content;

[0043] S330: Set a task for the visual language model, and generate inference keywords for the work that needs to be completed by the visual language model based on the results required to be achieved in the task.

[0044] The present invention uses a parallel image-cutting algorithm to slice digital images, and employs multi-threaded processing and a CNN deep neural network to detect cell types, significantly improving the speed of microscopic examination. A pre-built expert knowledge base and large visual language model provide graphical and textual descriptions of the statistical results of the entire blood smear, allowing for the automatic generation of highly accurate blood cell morphological analysis reports based on pre-defined report templates. This allows for the automated completion of the entire process of microscopic examination, statistics, cell morphological analysis, and analysis report generation for a digital image of blood cells. The morphological analysis report can be used to aid in disease diagnosis, significantly reducing misdiagnosis and missed diagnoses of cells and diseases, and significantly improving diagnostic speed and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0046] Figure 1 This is a structural block diagram of an embodiment of the artificial intelligence-based blood cell morphology analysis system of the present invention.

[0047] Figure 2 Flowchart of the working process of this embodiment. DETAILED DESCRIPTION

[0048] The following describes the implementation of the present invention through specific examples. The illustrations provided in the following embodiments are only used to schematically illustrate the basic concept of the present invention. The following embodiments and features in the embodiments may be combined with each other unless there is any conflict.

[0049] See also Figure 1 , Figure 1 This is a structural block diagram of an embodiment of an artificial intelligence-based blood cell morphology analysis system of the present invention. The artificial intelligence-based blood cell morphology analysis system of this embodiment includes a preprocessing unit, a detection unit, a statistics unit, and a report generation unit.

[0050] The preprocessing unit is configured to perform parallel segmentation of an input digital blood cell image (e.g., a digital image of peripheral blood cells, bone marrow blood cells, or the like) using multiple parallel segmentation paths to obtain multiple preprocessed images, and to perform standardization on the preprocessed images obtained by each parallel segmentation path using a multithreaded approach to obtain standardized images. In this embodiment, the preprocessing unit may include a segmentation algorithm module, an image segmentation module, and a standardization module.

[0051] The graph cutting algorithm module is used to determine multiple parallel graph cutting paths using a parallel cutting algorithm. The method of determining parallel graph cutting paths using a parallel cutting algorithm may include the following steps:

[0052] S110 , determining the cutting resolution of the blood cell digital image.

[0053] S120 , dividing the blood cell digital image into a plurality of blocks to be cut according to the resolution of the blood cell digital image in a row and column cutting manner.

[0054] S130: Serially connect the slices along a predetermined route to form a slice queue, where each element of the slice queue includes the position information of a slice. In this embodiment, each element of the slice queue includes the coordinates of the upper left corner of the corresponding slice in the digital image of blood cells. The method for calculating the coordinates of the upper left corner of the slice in the digital image includes the following steps:

[0055] S131. Calculate the number of horizontal and vertical cuts of the blood cell digital image. Assume that the cut resolution of the blood cell digital image is w1×h1 and the resolution of the blood cell digital image is w×h. The calculation formula for the cut number is as follows:

[0056]

[0057] Among them, n x Indicates the number of transverse cuts; n y Indicates the number of longitudinal cuts; the number of cuts in step S120 is n x ×ny .

[0058] S132, calculate the coordinates (x i,j ,y i,j )=(i×w1,j×h1); where i represents the horizontal index of the slice, i∈[0,n x -1]; j represents the vertical index of the slice, j∈[0,n y -1].

[0059] According to the coordinates of the upper left corner of the cut block in the blood cell digital image and the cutting resolution of the blood cell digital image, the coordinates of the other three corner points of the cut block in the digital image can be calculated. According to the coordinates of the four corner points, the cut block can be cut out from the blood cell digital image to obtain a preprocessed image.

[0060] S140 , dividing the cutting queue into a corresponding number of cutting sub-queues according to the number of image cutting threads, and the cutting path of each cutting sub-queue is a parallel image cutting path.

[0061] The above method can form multiple parallel cutting paths, so that image cutting and cell detection can be performed in parallel along multiple paths, and the efficiency of microscopy can be accelerated through multi-threaded processing.

[0062] The image segmentation module is used to simultaneously segment the digital blood cell image using multiple image segmentation threads, generating multiple pre-processed images. Regions with insufficient resolution are padded with zero pixels. Digital blood cell images are typically obtained by scanning a blood smear at 100x magnification and then stitching the scanned images together.

[0063] The standardization module is used to use multiple standardization processing threads to simultaneously perform standardization processing (such as resizing and normalization operations) on the pre-processed images obtained by each cutting thread to obtain a standardized image.

[0064] The detection unit is used to adopt a multi-threaded parallel processing method, using multiple detection threads to respectively locate and identify cells in the standardized image based on the cell detection model. The parallel image cutting path, image cutting thread, standardization processing thread and detection thread can correspond one to one.

[0065] The detection unit can pre-train a CNN deep neural network using collected data to obtain a cell detection model, and then detect the standardized image based on the pre-trained CNN deep neural network. The input of the cell detection model is a standardized image, and the output is the position of each identified cell in the pre-processed image and the cell category. The position can include the coordinates of the upper left corner and lower right corner of the cell, and the cell category can be one of C1, C2, C3, ..., CN. Using a CNN deep neural network to detect cell position and category has high detection efficiency.

[0066] The statistical unit is used to count the number, proportion and reference value of each type of cells according to the recognition results, and output them as statistics to the report generation unit. The processing process of the statistical unit is a mature existing technology and will not be described in detail here.

[0067] The report generation unit is used to generate a cell morphology analysis report based on the statistical results based on the pre-trained visual language model. The report generation unit may include a knowledge base module, a knowledge query module, a prompt word module, a morphology analysis module and a report generation module.

[0068] The knowledge base module is used to store the knowledge vector data required for generating the report. The method for generating the knowledge base module may include the following steps:

[0069] S210. Collect expert consensus, literature, and authoritative books on the diagnosis of blood cell morphology diseases in advance and organize them into electronic documents as the basis of the expert knowledge base.

[0070] S220: Pre-training a visual language embedding model. The visual language embedding model is a multimodal embedding model that can convert between different modalities, thereby being able to learn information from multiple input modes and output information in multiple output modes.

[0071] S230: Extract knowledge from the electronic document using the trained visual language embedding model and convert it into vector data for storage, thereby obtaining a knowledge base module. In this step, the knowledge extracted from the electronic document is generally an image-text pair, where the image in the image-text pair is a blood cell image and the text is a text description of the blood cells in the image. In the visual language embedding model, the cell image and cell text description are expressed as follows:

[0072] E k =f k (x k )∈R d

[0073] Among them, x k represents the input of the kth modality (e.g., cell image), f k() represents the encoder of the kth mode, R d represents a d-dimensional vector in the shared space; E k Represents input x k A d-dimensional embedding vector in the shared space.

[0074] The knowledge query module is used to convert the statistical results output by the statistical unit into vectors, and to obtain the corresponding textual knowledge by querying the knowledge base module by calculating the similarity of the vectors. During the query process, different modalities can have different weights.

[0075] In this embodiment, cosine similarity is used as the similarity index of the vectors. The threshold of the cosine similarity can be set to θ, all knowledge vectors in the knowledge base module are retrieved, and the contents of the first three knowledge vectors with cosine similarity greater than θ are returned as the prompt content of the visual language large model.

[0076] The formula for calculating the similarity of vectors by the knowledge query module is:

[0077]

[0078] Among them, e i Represents the normalized vector of statistical results; e j Represents a knowledge vector in the vector database; sim cos (e i ,e j ) means e i and e j The cosine similarity between .

[0079] The prompt word module is used to provide the inference prompt words of the input information to the morphological analysis model (i.e., the visual language model). The method for determining the inference prompt words of the prompt word module may include the following steps:

[0080] S310: Set the role of the visual language big model. For example, the visual language big model may be assumed to be a peripheral blood morphology test expert, and inference keywords for the visual language big model may be extracted from the perspective of the peripheral blood morphology test expert.

[0081] S320: Based on the role of the visual language model, the output of the statistical unit and the knowledge query module are combined to determine the role prompt content, and based on the prompt content, inference keywords are generated based on the known information (primarily information related to the input data of the visual language model). For example, inference keywords related to the input data of the visual language model may include: detected cell type and number, proportion, reference value, related images, and queried knowledge.

[0082] S330: Set a task for the visual language model and generate inference keywords for the work that the visual language model needs to complete (mainly related to the output data of the visual language model) based on the results required by the task. For example, the task of the visual language model can be to generate a peripheral blood morphology test report, including the following content:

[0083] (1) The proportion of each type of cells and a textual description of the degree of abnormality of abnormal indicators.

[0084] (2) Text description and related images of the morphology of various types of cells (mainly focusing on cells with abnormal indicators).

[0085] (3) Text description of the disease auxiliary diagnosis results.

[0086] Thus, the above content is used to generate inference keywords related to the input data of the visual language large model.

[0087] The morphological analysis module includes a morphological analysis model obtained by pre-training a large visual language model. This model uses the textual knowledge queried by the knowledge query module and the inference prompts provided by the prompt word module as input information for the morphological analysis model, causing the morphological analysis model to output the task content required by the inference prompts, i.e., the content required for the morphological analysis report. The large visual language model can be trained using a large number of blood cell morphology analysis reports or diagnostic reports related to blood cell morphology analysis as samples to ensure the accuracy of the output results of the trained morphological analysis model.

[0088] The report generation module is used to fill the content output by the morphological analysis module into the report template to generate a morphological analysis report (which can be an auxiliary diagnosis report). The morphological analysis report can be used as a reference for auxiliary diagnosis of the disease. The report template is pre-defined according to the format of the morphological analysis report. Multiple report templates can be pre-defined to accommodate the report template formats of different testing units. At this time, a report template can be specified in the report generation module as a template for generating the morphological analysis report according to needs.

[0089] The working principle of this embodiment is as follows:

[0090] See also Figure 1 and Figure 2 The image slicing algorithm module first determines parallel slicing paths. The image slicing module then simultaneously slices the digital blood cell image obtained by microscope scanning along multiple parallel slicing paths. The pre-processed images formed by each parallel slicing path undergo multi-threaded parallel processing. This multi-threaded parallel processing includes standardization by the standardization module and cell location and identification in the standardized image by the detection unit. Because multi-threaded parallel processing is performed throughout the slicing, standardization, and detection processes, detection efficiency is significantly improved.

[0091] Afterwards, the number, proportion and reference value of each type of cell are counted by the statistical unit and output to the report generation unit. In the report generation unit, the statistical results are first converted into vectors through the knowledge query module, and the corresponding text knowledge is obtained as the query result by querying the knowledge base module based on the similarity. The query results and the inference prompt words are then input into the morphological analysis model of the morphological analysis module. The morphological analysis model outputs the content required for the morphological analysis report based on the prompt content and task settings of the prompt word module. Finally, the report generation module fills the above content into the pre-defined report template to generate a morphological analysis report for auxiliary diagnosis.

[0092] In this embodiment, a blood cell morphology analysis system based on artificial intelligence analysis is used. For a digital image of blood cells on a whole blood smear, the entire process of microscopic examination, statistics, cell morphology analysis, and analysis report issuance can be automatically completed. This embodiment uses a parallel cutting algorithm to cut the digital image, and adopts a multi-threaded processing method and a CNN deep neural network to detect cell types, which greatly improves the speed of microscopic examination. Through a pre-built expert knowledge base and a large visual language model, the statistical results of the whole blood smear are described in pictures and texts. According to a pre-defined report template, a highly accurate morphological analysis report of peripheral blood cells can be automatically issued, thereby assisting in the diagnosis of diseases. It can also greatly reduce the misdiagnosis and missed diagnosis of cells and diseases, and significantly improve the speed and accuracy of diagnosis.

[0093] The above embodiments merely represent preferred embodiments of the present invention. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent. It should be noted that a person skilled in the art would be able to make various modifications and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the patent for this invention shall be determined by the appended claims.

Claims

1. A blood cell morphology analysis system based on artificial intelligence, characterized in that: include A preprocessing unit is used to perform parallel cutting of the input blood cell digital image using multiple parallel cutting paths to obtain multiple preprocessed images, and to perform standardization processing on the preprocessed images obtained by cutting each parallel cutting path using a multi-threaded method to obtain a standardized image; A detection unit, configured to locate and identify cells in standardized images based on a cell detection model using a multi-threaded parallel processing method; A statistical unit is used to count the number, proportion and reference value of various types of cells based on the recognition results; as well as The report generation unit is used to generate a cell morphology analysis report based on the statistical results based on the pre-trained visual language model.

2. The artificial intelligence-based blood cell morphology analysis system according to claim 1, characterized in that: The pre-processing unit includes A graph cutting algorithm module is used to determine multiple parallel graph cutting paths using a parallel cutting algorithm; An image cutting module is used to use multiple image cutting threads to cut the blood cell digital image in parallel to obtain multiple pre-processed images; each of the image cutting threads corresponds to a parallel cutting path; as well as The standardization module is used to use multiple standardization processing threads to simultaneously perform standardization processing on the pre-processed images obtained by each cutting thread to obtain a standardized image.

3. The artificial intelligence-based blood cell morphology analysis system according to claim 2, characterized in that: The method for determining a parallel cutting path using a parallel cutting algorithm includes the following steps: S110, determining a cutting resolution of the blood cell digital image; S120, dividing the blood cell digital image into a plurality of blocks to be cut in a row and column manner according to the resolution of the blood cell digital image; S130, sequentially connecting the cut pieces in series along a predetermined route to form a cut queue, wherein each element of the cut queue includes position information of a cut piece; S140 , dividing the cutting queue into a corresponding number of cutting sub-queues according to the number of image cutting threads, and the cutting path of each cutting sub-queue is a parallel image cutting path.

4. The artificial intelligence-based blood cell morphology analysis system according to claim 3, characterized in that: Each element in the cutting queue includes the coordinates of the upper left corner of the corresponding cut piece in the blood cell digital image; the method for calculating the coordinates of the upper left corner of the cut piece in the blood cell digital image includes the following steps: S131. Calculate the number of horizontal and vertical cuts of the digital image of blood cells. The calculation formula for the number of cuts is as follows: Where w×h represents the resolution of the blood cell digital image; w1×h1 represents the cutting resolution; n x Indicates the number of transverse cuts; n y Indicates the number of longitudinal cuts; S132, calculate the coordinates (x i,j ,y i,j )=(i×w1,j×h1); where i represents the horizontal index of the slice, i∈[0,n x -1]; j represents the vertical index of the slice, j∈[0,n y -1].

5. The artificial intelligence-based blood cell morphology analysis system according to claim 1, characterized in that: The detection unit pre-trains the CNN deep neural network by collecting data to obtain a cell detection model, and detects the standardized image based on the pre-trained CNN deep neural network.

6. The artificial intelligence-based blood cell morphology analysis system according to any one of claims 1 to 5, characterized in that: The report generating unit includes Knowledge base module, used to store knowledge vector data; The knowledge query module is used to convert the statistical results output by the statistical unit into vectors, and obtain the corresponding text knowledge by querying the knowledge base module by calculating the similarity of the vectors; The prompt word module is used to provide inference prompt words of input information to the morphological analysis model; The morphological analysis module is used to use the text knowledge queried by the knowledge query module and the inference prompt words provided by the prompt word module as input information of the morphological analysis model, so that the morphological analysis model outputs the task content required by the inference prompt words; The morphological analysis model is obtained by pre-training a large visual language model; as well as The report generation module is used to fill the output content of the morphological analysis module into the report template to generate a morphological analysis report.

7. The artificial intelligence-based blood cell morphology analysis system according to claim 6, characterized in that: The method for generating a knowledge base module includes the following steps: S210. Collect expert consensus, literature, and authoritative books on the diagnosis of blood cell morphology diseases in advance and organize them into electronic documents; S220, pre-trained visual language embedding model; S230. Extract knowledge from the electronic document through the trained visual language embedding model and convert it into vector data for storage to obtain a knowledge base module.

8. The artificial intelligence-based blood cell morphology analysis system according to claim 7, characterized in that: In the step S230 , the knowledge extracted from the electronic document is an image-text pair, where the image in the image-text pair is a blood cell image, and the text is a text description of the blood cells in the image.

9. The artificial intelligence-based blood cell morphology analysis system according to claim 6, characterized in that: The formula for calculating the similarity of vectors by the knowledge query module is: Among them, e i Represents the normalized vector of statistical results; e j Represents a knowledge vector in the vector database; sim cos (e i ,e j ) means e i and e j The cosine similarity between .

10. The artificial intelligence-based blood cell morphology analysis system according to claim 6, characterized in that: The method for determining the inference prompt word of the prompt word module includes the following steps: S310, setting the role of the visual language model; S320: Determine the prompt content of the character based on the character of the visual language large model and the output content of the statistical unit and the knowledge query module, and generate inference keywords of the known information based on the prompt content; S330: Set a task for the visual language model, and generate inference keywords for the work that needs to be completed by the visual language model based on the results required to be achieved in the task.

Images