A deep learning-based blood plasmodium rapid detection and auxiliary diagnosis system

Abstract

The application relates to the field of medical examination, in particular to a blood plasmodium rapid detection and auxiliary diagnosis system based on deep learning. The system comprises a region positioning module, an image preprocessing module, a detector first-stage training module, a plasmodium first-stage detection module, an image enhancement module, a detector second-stage training module, a plasmodium second-stage detection module, a classifier training module, a plasmodium analysis module and a result display module. Through the above scheme, the application solves the problems of long detection time and human errors caused by fatigue and cognitive overload.

Description

TECHNICAL FIELD

[0001] The application relates to the field of medical examination, in particular to a blood plasmodium rapid detection and auxiliary diagnosis system based on deep learning. BACKGROUND

[0002] Malaria is a mosquito-borne infectious disease caused by plasmodium, which is widely distributed, rapidly transmitted and has a long incubation period. It is directly related to human health, economic development and social stability. Early detection of malaria is the key to reducing mortality and controlling the spread of malaria. The traditional microscopic blood smear detection method requires high professional skills, and the whole detection process takes a long time, and human errors caused by fatigue and cognitive overload are prone to occur. Therefore, how to improve the efficiency and accuracy of malaria examination has important value for malaria-prone areas.

[0003] By introducing a deep learning algorithm, automatic identification and classification of plasmodium infected cells can be realized, which can facilitate malaria diagnosis in resource-poor environments, greatly reduce detection costs and improve diagnosis coverage. The application of this intelligent auxiliary diagnosis system not only accelerates the early diagnosis of malaria and reduces its spread, but also conducts efficacy evaluation and drug resistance detection, which has important public health significance and provides an innovative idea for promoting technological progress in the medical field. SUMMARY

[0004] The application provides a blood plasmodium rapid detection and auxiliary diagnosis system based on deep learning to solve the problems in the prior art.

[0005] To solve the above problems, the following technical solutions are provided:

[0006] The blood plasmodium rapid detection and auxiliary diagnosis system based on deep learning comprises:

[0007] A region positioning module is configured to obtain a low-resolution blood smear image, locate a target region, and obtain a spatial position and a high-resolution image of the target region.

[0008] An image preprocessing module is configured to receive the spatial position and the high-resolution image, preprocess the high-resolution image, and obtain original image data, original label data, image data of a normal sample, and image data to be detected.

[0009] A position acquisition module is configured to receive the image data to be detected, detect and store image data of plasmodium infected red blood cells and position information of the plasmodium infected red blood cells.

[0010] The category acquisition module is configured to receive image data of malaria-infected red blood cells, implement classification of the malaria-infected red blood cells, and obtain category information of the malaria-infected red blood cells.

[0011] The result display module is configured to receive the high-resolution image, the position information of the malaria-infected red blood cells, and the category information of the malaria-infected red blood cells, locate and classify the malaria-infected red blood cells on the high-resolution image.

[0012] By using the above scheme, the position acquisition module is configured to acquire the position information, the category acquisition module is configured to acquire the category information, and the result display module is configured to superimpose the position information and the category information on the original high-resolution image. After locating and classifying the malaria-infected red blood cells on the high-resolution image, a doctor can quickly search for the position and quantity of the malaria-infected red blood cells in a full field of view, and determine the category of the malaria after checking the classification result, so as to formulate a suitable treatment plan.

[0013] The position acquisition module includes a detector first-stage training module, a malaria first-stage detection module, an image enhancement module, a detector second-stage training module, and a malaria second-stage detection module. The detector first-stage training module is configured to receive and integrate original image data and original label data to form a first training data set, and train a first weight file. The malaria first-stage detection module is configured to receive image data of normal samples and the first weight file, detect false positive single cells in the image data of the normal samples, and store image data and position information of the false positive single cells. The image enhancement module is configured to receive original image data, original label data, and image data of false positive single cells, and perform data enhancement on the first training data set to obtain an enhanced training data set. The detector second-stage training module is configured to receive the enhanced training data set and the first weight file, and train a second weight file. The malaria second-stage detection module is configured to receive to-be-detected image data and the second weight file, and detect and store image data of malaria-infected red blood cells and position information of the malaria-infected red blood cells.

[0014] The category acquisition module includes a classifier training module and a malaria analysis module. The classifier training module is configured to receive the enhanced training data set, extract single cell data, implement classification training, and obtain a classification model weight file. The malaria analysis module is configured to receive image data of malaria-infected red blood cells and the classification model weight file, implement classification of the malaria-infected red blood cells, and obtain category information of the malaria-infected red blood cells.

[0015] By adopting the above scheme, the regional positioning module automatically identifies the blood smear area suitable for observation in the malaria analysis task, avoiding the time-consuming process of manually finding the target area in traditional microscope detection, and combining the scanning device to obtain high-resolution images, which greatly shortens the detection time; the image enhancement module generates diversified training data by fusing true positive and false positive single cell data, combining random rotation, contrast adjustment and other transformations, which alleviates the sample imbalance problem and can adapt to blood smear images with different staining conditions and cell morphologies; the system can automatically complete image preprocessing, target detection and classification tasks, reducing the dependence on professional microscopic detection personnel, especially suitable for malaria high-incidence areas with insufficient medical resources, which helps to improve the diagnosis and treatment coverage of primary medical institutions; the malaria analysis module can not only identify infected cells, but also distinguish developmental stages (ring body, trophozoite, etc.) and malaria parasite categories (Plasmodium falciparum, Plasmodium vivax, etc.), providing more detailed pathological information for clinicians; the result display module directly superimposes the position information of malaria-infected red blood cells and the category information of malaria-infected red blood cells on the high-resolution image, which is convenient for doctors to quickly verify and interpret, improves the diagnosis efficiency, and provides traceable visual data for teaching and scientific research, thereby solving the problems of long detection time and human error caused by fatigue and cognitive overload, and the system can significantly improve the efficiency, accuracy and applicability of malaria detection.

[0016] The regional positioning module comprises:

[0017] S1.1: a first image acquisition unit, configured to acquire blood smear images taken by a camera device, and preview images to form a first data set;

[0018] S1.2: a dual-channel normalization unit, configured to eliminate staining differences between the images in the first data set by a normalization method;

[0019] S1.3: a perception segmentation unit, configured to realize density perception segmentation by adaptive grid density estimation;

[0020] S1.4: a push slide track analysis unit, configured to determine the blood smear push slide direction according to the density field estimation obtained by the perception segmentation unit;

[0021] S1.5: a single cell layer positioning unit, configured to determine that the single cell layer is located at the rear 1 / 3 to 1 / 2 of the push slide path according to the density field heat map and the push slide track;

[0022] S1.6: a spatial position extraction unit, configured to determine that the to-be-scanned area is located in a 1cm*1cm area centered on the blood boundary in the single cell layer distribution range according to the blood boundary and the single cell layer distribution, and output the spatial coordinates of the four corner points of the area as the position information of the analysis area, and feed back the position information to the scanning device, and the scanning device scans the high-resolution image.

[0023] The image preprocessing module comprises:

[0024] S2.1: a second image acquisition unit configured to acquire and crop a high-resolution image according to position information of the receiving space position extraction unit;

[0025] S2.2: a staining standardization unit configured to implement staining standardization on the high-resolution image cropped by the second image acquisition unit through a Reinhard color migration algorithm;

[0026] S2.3: a first storage unit configured to save original image data and original label data of each high-resolution image after the image preprocessing module, image data marked as normal samples, and to-be-detected image data.

[0027] The detector first-stage training module comprises:

[0028] S3.1-1: a first data acquisition unit configured to receive the original image data and the original label data in the first storage unit, and construct a first training data set;

[0029] S3.1-2: a loss optimization unit configured to receive the first training data set, train a first weight file by using an adaptive time point estimation algorithm and applying online difficult case mining in a detection head;

[0030] S3.1-3: a detector storage unit configured to save the first weight file trained by the loss optimization unit;

[0031] The first-stage Plasmodium detection module comprises:

[0032] S3.2-1: a second data acquisition unit configured to acquire the image data marked as normal samples in the first storage unit and the first weight file in the detector storage unit;

[0033] S3.2-2: a target detection unit configured to receive and load the first weight file into a detector architecture of Faster-RCNN, receive the image data in the second data acquisition unit, and implement detection of easily confused false positive single cells in normal image data by using the Faster-RCNN with the loaded weight file;

[0034] S3.2-3: a second storage unit configured to store image data and position information of the false positive single cells.

[0035] The image enhancement module comprises:

[0036] S3.3-1: a third data acquisition unit, configured to receive the false positive single-cell image data in the second storage unit and the original image data and the original label data in the first storage unit, obtain and save the true positive single-cell image data;

[0037] S3.3-2: an image transformation unit, configured to perform random rotation and contrast adjustment on all the single-cell image data in the third data acquisition unit, and set a random factor range to control the intensity of the contrast adjustment;

[0038] S3.3-3: an image selection unit, configured to randomly select the false positive single-cell image data and the false negative single-cell image data output by the image transformation unit, select the image data as supplementary image data at each time, and record the category information corresponding to the image data, the category information being true positive or false positive;

[0039] S3.3-4: an image fusion unit, configured to overlay the supplementary image data on the original image data for training obtained by the third data acquisition unit, while ensuring that the overlay region does not block the region annotated by the original label data, to generate new fusion data;

[0040] S3.3-5: a label generation unit, configured to generate a new label file for the fusion data obtained by overlaying the true positive single-cell image data in the supplementary image data on the original image, when the supplementary image data contains the true positive single-cell image data;

[0041] S3.3-6: a label alignment unit, configured to rename the fusion data and the corresponding label, to ensure that the data and the label name are consistent, to obtain the enhanced image data for training and the corresponding label data.

[0042] The detector second-stage training module comprises:

[0043] S3.4-1: a fourth data acquisition unit, configured to receive the enhanced image data and the label data in the label alignment unit to construct a detector second training data set, and receive the first weight file in the detector storage unit;

[0044] S3.4-2: a loss optimization unit, configured to receive the detector second training data set and the detector first weight file, load the first weight file parameters into the detector architecture of Faster-RCNN, and train to obtain a second weight file;

[0045] S3.4-3: a detector storage unit, configured to save the second weight file trained by the loss optimization unit;

[0046] The Plasmodium second-stage detection module comprises:

[0047] S3.5-1: a fifth data acquisition unit, configured to acquire the image data to be detected in the first storage unit and the second weight file of the detector storage unit;

[0048] S3.5-2: a target detection unit, configured to receive and load the second weight file parameter into the detector architecture of Faster-RCNN, receive the image data to be detected in the fifth data acquisition unit, input the image data to be detected into the detector architecture of Faster-RCNN loaded with the weight file, and realize detection of the malaria-infected cells in the image data to be detected;

[0049] S3.5-3: a third storage unit, configured to store the image data of the malaria-infected red blood cells and the position information of the malaria-infected red blood cells detected in the image data to be detected.

[0050] The classifier training module comprises:

[0051] S4.1-1: a sixth data acquisition unit, configured to acquire all positive single-cell data and negative single-cell data obtained by the image transformation unit in the image enhancement module, and mark the positive single-cell data by an expert, and the label type is: developmental stage-malaria category;

[0052] S4.1-2: a size resetting unit, configured to adjust the size of the single-cell data of the sixth data acquisition unit to a fixed size;

[0053] S4.1-3: a fourth storage unit, configured to receive the images adjusted by the size resetting unit, and sequentially perform developmental stage classification and malaria category classification to obtain four developmental stage classification labels and five malaria-infected red blood cell classification labels in the trophozoite stage and output the labels;

[0054] S4.1-4: a first gradient updating unit, configured to receive the four developmental stage classification labels of the fourth storage unit, and train the classifier of the ResNet50 architecture by an adaptive time estimation method to obtain a third weight file after the training;

[0055] S4.1-5: a second gradient updating unit, configured to receive the five malaria-infected red blood cell classification labels in the trophozoite stage of the fourth storage unit, and train the classifier of the ResNet50 architecture by the adaptive time estimation method to obtain a fourth weight file after the training;

[0056] S4.1-6: a classifier storage unit, configured to store the third weight file and the fourth weight file trained by the first gradient updating unit and the second gradient updating unit, respectively, and the classification model weight file is the third weight file and the fourth weight file.

[0057] The malaria analysis module comprises:

[0058] S4.2-1: a seventh data acquisition unit, configured to acquire the Plasmodium infected cell image data in the third storage unit and the classification model weight file in the classifier storage unit;

[0059] S4.2-2: a size resetting unit, configured to adjust the single cell data size of the seventh data acquisition unit to a fixed size;

[0060] S4.2-3: a first classification unit, configured to load the third weight file in the seventh data acquisition unit into ResNet50 to obtain a first classifier architecture, and receive the Plasmodium infected cell image obtained by the size resetting unit as the input of the first classifier, and output the developmental stages of different input cell images: ring, trophozoite, schizont and gametocyte, so as to realize developmental stage classification and output the first-level classification label;

[0061] S4.2-4: a fifth storage unit, configured to store the first-level classification label output by the first classification unit and the Plasmodium infected cell image obtained by the size resetting unit;

[0062] S4.2-5: a second classification unit, configured to load the fourth weight file in the seventh data acquisition unit into ResNet50 to obtain a second classifier architecture, and receive the trophozoite developmental stage image data in the fifth storage unit as the input of the second classifier architecture, and output the Plasmodium species in different input cell images: P. falciparum, P. vivax, P. malariae, P. ovale and P. knowlesi, so as to realize Plasmodium species analysis;

[0063] S4.2-6: a sixth storage unit, configured to assign the category information of all Plasmodium infected red blood cells in the fourth storage unit according to the classification results output by the first classification unit and the second classification unit, and store in the developmental stage-Plasmodium species format.

[0064] With the above scheme, the following advantages are achieved:

[0065] The rapid blood malaria parasite detection and auxiliary diagnosis system based on deep learning can automatically identify the blood smear area suitable for observation in the malaria analysis task through the region positioning module, avoid the time-consuming process of manually finding the target area in the traditional microscope detection, combine the scanning equipment to obtain a high-resolution image, and greatly shorten the detection time; the image enhancement module generates diversified training data by fusing true positive and false positive single cell data, combining random rotation, contrast adjustment and other transformations, alleviates the sample imbalance problem, and can adapt to blood smear images with different staining conditions and cell morphologies; the system can automatically complete image preprocessing, target detection and classification tasks, reduce the dependence on professional microscopic detection personnel, is especially suitable for malaria high-incidence areas with insufficient medical resources, and is helpful to improve the diagnosis and treatment coverage of primary medical institutions; the malaria parasite analysis module can not only identify infected cells, but also distinguish development stages (ring body, trophozoite, etc.) and malaria parasite categories (falciparum malaria parasite, vivax malaria parasite, etc.), so as to provide more detailed pathological information for clinicians; the result display module directly superimposes the position information of the malaria parasite infected red blood cells and the category information of the malaria parasite infected red blood cells on the high-resolution image, which is convenient for doctors to quickly verify and interpret, improves the diagnosis efficiency, and provides traceable visual data for teaching and scientific research, so as to solve the problems of long detection time and human error caused by fatigue and cognitive overload, and the system can significantly improve the efficiency, accuracy and applicability of malaria detection. BRIEF DESCRIPTION OF DRAWINGS

[0066] In order to make the content of the present application more easily understood, the present application will be further described in detail below according to specific embodiments of the present application and in conjunction with the drawings, in which:

[0067] Fig. 1 It is a framework diagram of a rapid blood malaria parasite detection and auxiliary diagnosis system based on deep learning;

[0068] Fig. 2 It is a flowchart of a rapid blood malaria parasite detection and auxiliary diagnosis system based on deep learning. DETAILED DESCRIPTION

[0069] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings of the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0070] In specific embodiments, as shown in Figs. 1-2 The rapid blood malaria parasite detection and auxiliary diagnosis system based on deep learning comprises:

[0071] The region positioning module is connected with the camera device and the scanning device, and is configured to acquire a low-resolution blood smear image captured by the camera device, locate a region suitable for observation in the blood smear image, acquire a spatial position of the region, and feed back the spatial position to the scanning device, so that the scanning device scans a high-resolution image.

[0072] The region positioning module comprises the following units:

[0073] The first image acquisition unit is configured to acquire the blood smear image captured by the camera device, and preview the image to form a first data set; the double-channel normalization unit is configured to eliminate staining differences between the images in the first data set by a normalization method; the perception segmentation unit is configured to realize density perception segmentation by adaptive grid density estimation; the slide pushing track analysis unit is configured to determine a blood smear slide pushing direction according to a density field estimation obtained by the perception segmentation unit; the single cell layer positioning unit is configured to determine that a single cell layer is located at a rear 1 / 3 to 1 / 2 of a slide pushing path according to a density field heat map and the slide pushing track; and the spatial position extraction unit is configured to determine that a scanning region is located in a 1cm*1cm region centered on a blood boundary in a single cell layer distribution range, and output spatial coordinates of four corner points of the region as position information of an analysis region, so as to feed back the position information to the scanning device, and the scanning device scans a high-resolution image.

[0074] The image preprocessing module is configured to receive the spatial position and the high-resolution image obtained by the region positioning module, and pre-process the high-resolution image to obtain original image data and original label data, image data of a normal sample which has been marked, and image data to be detected.

[0075] The image preprocessing module comprises the following units:

[0076] The second image acquisition unit is configured to acquire and crop the high-resolution image according to the position information received by the spatial position extraction unit; the staining standardization unit is configured to realize staining standardization on the high-resolution image cropped by the second image acquisition unit by using a Reinhard color migration algorithm; and the first storage unit is configured to save the original image data and the original label data, the image data of the normal sample which has been marked, and the image data to be detected after each high-resolution image is pre-processed by the image preprocessing module.

[0077] The position acquisition module comprises a detector first stage training module, a malaria first stage detection module, an image enhancement module, a detector second stage training module and a malaria second stage detection module; the detector first stage training module is configured to receive and integrate original image data and original label data to form a first training data set, and to train a first weight file; the malaria first stage detection module is configured to receive image data marked as normal samples and the first weight file, to detect false positive single cells in the image data of the normal samples, and to store image data and position information of the false positive single cells; the image enhancement module is configured to receive original image data, original label data and image data of false positive single cells, to perform data enhancement on the first training data set to obtain an enhanced training data set; the detector second stage training module is configured to receive the enhanced training data set and the first weight file, and to train a second weight file; and the malaria second stage detection module is configured to receive image data to be detected and the second weight file, to detect and store image data of malaria-infected red blood cells and position information of the malaria-infected red blood cells.

[0078] The detector first stage training module comprises the following units:

[0079] The first data acquisition unit is configured to receive original image data and original label data in the first storage unit, and to construct a first training data set; the loss optimization unit is configured to receive the first training data set, to train a first weight file by using an adaptive time estimation algorithm and applying online difficult case mining in a detection head; and the detector storage unit is configured to save the first weight file trained by the loss optimization unit.

[0080] The malaria first stage detection module comprises the following units:

[0081] The second data acquisition unit is configured to acquire image data marked as normal samples in the first storage unit and the first weight file in the detector storage unit; the target detection unit is configured to receive and load the first weight file into a Faster-RCNN detector architecture, to receive image data in the second data acquisition unit, to load the image data into the Faster-RCNN detector architecture with the loaded weight file, and to detect false positive single cells in the normal image data; and the second storage unit is configured to store image data and position information of the false positive single cells.

[0082] The image enhancement module comprises the following units:

[0083] The third data acquisition unit receives false-positive single-cell image data from the second storage unit and raw image data and raw label data from the first storage unit, and obtains and saves true-positive single-cell image data. The image transformation unit performs random rotation and contrast adjustment on all single-cell image data from the third data acquisition unit, and sets a random factor range to control the intensity of contrast adjustment. The image selection unit randomly selects from the false-positive and false-negative single-cell image data output by the image transformation unit, using each selected image as supplementary image data for the next step, and records the corresponding category information of the image data. The category information is either true positive or false positive; the image fusion unit is used to overlay supplementary image data onto the original image data for training obtained by the third data acquisition unit, while ensuring that the overlaid area does not obscure the area annotated by the original label data, generating new fused data; the label generation unit, when the supplementary image data contains true positive single-cell image data, generates a new label file for the fused data obtained by overlaying the true positive single-cell image data in the supplementary data onto the original image; the label alignment unit is used to rename the fused data and its corresponding labels, ensuring that the data and label names are consistent, and obtaining the enhanced image data and corresponding label data for training.

[0084] The second-stage training module for the detector includes the following units:

[0085] The fourth data acquisition unit is used to receive the augmented image data and label data used for training in the label alignment unit to construct the second training dataset of the detector, and to receive the first weight file in the detector storage unit; the loss optimization unit is used to receive the second training dataset of the detector and the first weight file of the detector, load the parameters of the first weight file into the detector architecture of Faster-RCNN, and train to obtain the second weight file; the detector storage unit is used to save the second weight file trained by the loss optimization unit.

[0086] The second-stage detection module for malaria parasites includes the following units:

[0087] The fifth data acquisition unit is used to acquire the image data to be detected in the first storage unit and the second weight file in the detector storage unit; the target detection unit is used to receive and load the second weight file parameters into the detector architecture of Faster-RCNN, receive the image data to be detected in the fifth data acquisition unit, and input it into the detector architecture of Faster-RCNN with the weight file loaded, so as to realize the detection of Plasmodium-infected cells in the image data to be detected; the third storage unit is used to store the image data of Plasmodium-infected red blood cells detected in the image data to be detected and the location information of Plasmodium-infected red blood cells.

[0088] The classifier training module receives the augmented training dataset, extracts single-cell data, performs classification training, and obtains the classification model weight file.

[0089] The classifier training module includes the following units:

[0090] The sixth data acquisition unit acquires all positive and negative single-cell data obtained from the image transformation unit in the image enhancement module. Experts then label the positive single-cell data with the tag type: developmental stage - Plasmodium category. The size reset unit adjusts the single-cell data size in the sixth data acquisition unit to a fixed size. The fourth storage unit receives the image adjusted by the size reset unit and sequentially performs developmental stage classification and Plasmodium category classification, obtaining four developmental stage classification labels and five classification labels for Plasmodium-infected erythrocytes in the trophozoite stage, which are then output. The first gradient update unit receives data from the fourth storage unit. The ResNet50 classifier is trained using an adaptive time estimation method, which generates a third weight file. The second gradient update unit receives five classification labels for trophozoite-infected red blood cells from the fourth storage unit and trains the ResNet50 classifier using the adaptive time estimation method, generating a fourth weight file. The classifier storage unit stores the third and fourth weight files generated by the first and second gradient update units, respectively. The classification model weight files are the third and fourth weight files.

[0091] The Plasmodium analysis module is used to receive image data and classification model weight files of Plasmodium-infected red blood cells, classify the Plasmodium-infected red blood cells, and obtain the category information of Plasmodium-infected red blood cells.

[0092] The malaria parasite analysis module includes the following units:

[0093] The seventh data acquisition unit is used to acquire the Plasmodium-infected cell image data from the third storage unit and the classification model weight file from the classifier storage unit; the size reset unit is used to adjust the size of the single-cell data from the seventh data acquisition unit to a fixed size; the first classification unit receives the third weight file from the seventh data acquisition unit, loads it into ResNet50 to obtain the first classifier architecture, and receives the Plasmodium-infected cell images obtained from the size reset unit as input to the first classifier. The first classifier outputs the developmental stages of different input cell images: ring body, trophozoite, schizont, and gametophyte, realizing developmental stage classification and outputting the first-level classification label; the fifth storage unit is used to store the first-level classification label output by the first classification unit. The system consists of: a class labeling and resizing unit for obtaining images of Plasmodium-infected cells; a second classification unit for receiving the fourth weight file from the seventh data acquisition unit and loading it into ResNet50 to obtain the second classifier architecture; and a fifth storage unit for receiving trophozoite developmental stage image data as input to the second classifier architecture. The second classifier architecture outputs the Plasmodium categories infecting different input cell images: Plasmodium falciparum, Plasmodium vivax, Plasmodium malariae, Plasmodium ovale, and Plasmodium norotri, thus realizing Plasmodium category analysis; and a sixth storage unit for assigning category information to all Plasmodium-infected erythrocytes in the fourth storage unit based on the classification results output by the first and second classification units, and storing the information in the developmental stage-Plasmodium category format.

[0094] The results display module is used to receive the location information and category information of malaria-infected red blood cells, and to locate and classify malaria-infected red blood cells on high-resolution images.

[0095] The operation process of the deep learning-based rapid detection and auxiliary diagnosis system for malaria parasites in blood is as follows:

[0096] S1.1: Used to acquire images of blood smears taken by a camera device, and to preview the images to form the first dataset;

[0097] S1.2: Eliminate coloring differences between images in the first dataset using a normalization method. The specific steps are as follows;

[0098] The image is decomposed using the HSV + LAB hybrid color space, and its color intensity, brightness, and color cast are optimized separately. The color saturation (S) is extracted through the HSV space, and the saturation is adjusted based on global statistical features (mean and variance) to make the color consistent. The brightness (L) and color cast (a / b channels) are extracted through the LAB space, and brightness adjustment and color cast correction are achieved by histogram matching and mean shift. Finally, the normalized S, L, and a / b channels are merged and converted back to the RGB image control to obtain the output image.

[0099] S1.3: Density-aware segmentation is achieved through adaptive mesh density estimation. The specific steps are as follows;

[0100] Let the input image be The image is processed using morphological TopHat transform to highlight local contrast variations: And determine the boundary information of blood on the smear, and set the initial size of the dynamic sliding window to [value missing]. Calculate the density distribution under each window: This generates a three-dimensional density matrix and uses color mapping to construct a density field heatmap. Gaussian filtering is used for smoothing and Gaussian difference is used to extract abrupt change regions in the density map.

[0101] S1.4: Determine the direction of blood smear pushing based on the density field estimation obtained from the sensing segmentation unit;

[0102] S1.5: Based on the density field thermogram and the slide trajectory, the single cell layer is located at the last 1 / 3 to 1 / 2 of the slide path;

[0103] S1.6: Based on the blood boundary and single-cell layer distribution, the area to be scanned is determined to be a 1cm×1cm region centered on the blood boundary of the single-cell layer distribution range. The spatial coordinates of the four corner points of this region are output as the location information of the analysis area. The location information is fed back to the scanning device, and the scanning device scans a high-resolution image.

[0104] S2.1: Based on the location information of the receiving spatial location extraction unit, acquire and crop a high-resolution image;

[0105] S2.2: Based on the characteristic that the channels in the Lab color space are not correlated with each other, the Reinhard color migration algorithm is used to achieve color normalization of the high-resolution image after cropping from the second image acquisition unit;

[0106] First, convert the source image and the high-resolution image from the RGB color space to the Lab color space. In the Lab color space, calculate the mean and standard deviation of the three color channels of the source image and the colored image respectively. Subtract the mean from the data of the source image to obtain the normalized data. Scale the normalized data proportionally, with the scaling factor being the ratio of the standard deviations of the two images. Add the mean of the target image to the scaled data to obtain the final data.

[0107] S2.3: Save the original image data and original label data of each high-resolution image after the image preprocessing module, the image data already marked as normal samples, and the image data to be detected.

[0108] The training process for the location acquisition module is as follows:

[0109] S3.1-1: Receive the original image data and original label data from the first storage unit to construct the first training dataset;

[0110] S3.1-2: Receive the first training dataset, and train the first weight file by using the adaptive time estimation algorithm (Adam algorithm) and applying online hard example mining (OHEM) in the detector head. This enables the selection of samples that contribute significantly to the loss during gradient update for backpropagation, thereby improving the learning ability of the Faster-RCNN detector architecture for difficult samples.

[0111] The OHEM mechanism is introduced in the loss calculation stage of the detector head: the loss of the Faster-RCNN detector head is composed of the classification loss. and bounding box regression loss Composition, for each candidate box Calculate its total loss : ;

[0112] in: The sum of classification loss and regression loss. For the first The predicted class probability distribution of each candidate box (RoI). For real category labels, and These are the predicted bounding box offset and the actual offset, respectively. The weighting coefficient for the regression loss is usually set to 1; according to Sort the candidate boxes in descending order and select the K samples with the highest loss values, where K is determined by the sampling ratio. Decide, Typically set to 0.3-0.7; only the loss of the first K hard samples is retained, and the weighted average is recalculated to construct the OHEM loss function: When using the Adam algorithm to backpropagate and update the model gradient parameters, only the gradients of the Top-K samples are updated, while the gradients of the remaining samples are set to zero.

[0113] The Adam algorithm is specifically calculated as follows: The learning rate is dynamically adjusted by calculating the first and second momentum of the gradient. The first momentum is the exponentially weighted average of the gradients, and the second momentum is the exponentially weighted average of the squared gradients. Assuming the current time step is... Calculate the gradient of the loss function with respect to the current parameters: Then update the first-order momentum of the gradient: ,in, The first momentum is the exponential decay rate, taken as 0.9; update the second momentum: ,in, This is the exponential decay rate of the second momentum, taken as 0.9; finally, the parameters are updated by combining the first and second momentum: ;

[0114] A label regularization method is introduced into the classification loss function of the model, and noise is added to the ont-hot encoded label vector to improve the poor generalization ability of the model.

[0115] Use updated label vectors To replace traditional ONT-HOT encoded label vectors : Where K is the number of categories, It is a hyperparameter with a value of 0.1;

[0116] S3.1-3: Save the first weight file of the detector after training by the loss optimization unit.

[0117] S3.2-1: Obtain the image data labeled as normal samples and the first weight file of the detector storage unit;

[0118] S3.2-2: A detector architecture for receiving and loading the first weight file parameters into Faster-RCNN, receiving image data from the second data acquisition unit, and loading the Faster-RCNN detector data into the weight file to achieve the detection of easily confused false positive single cells in normal image data.

[0119] S3.2-3: Store image data and location information of false positive single cells.

[0120] S3.3-1: Receive false positive single-cell image data, raw image data, and raw label data; obtain and save true positive single-cell image data.

[0121] S3.3-2: Image transformation unit, used to randomly rotate and adjust the contrast of all single-cell image data in the third data acquisition unit, and set the range of random factors to control the intensity of contrast adjustment;

[0122] S3.3-3: Randomly select false positive single-cell image data and false negative single-cell image data output by the image transformation unit. Each selected image data is used as supplementary image data for the next step, and record the category information corresponding to the image data, which is either true positive or false positive.

[0123] S3.3-4: Overlay the supplementary image data onto the original image data acquired by the third data acquisition unit for training, while ensuring that the overlaid area does not obscure the area annotated by the original label data, and generate new fused data;

[0124] S3.3-5: When the supplementary image data contains true positive single-cell image data, generate a new label file for the fused data obtained by overlaying the true positive single-cell image data in the supplementary data onto the original image;

[0125] S3.3-6: Rename the fused data and its corresponding labels to ensure that the data and label names are consistent, so as to obtain the enhanced image data and corresponding label data for training.

[0126] S3.4-1: Receive the augmented image data and label data used for training in the label alignment unit to construct the second training dataset for the detector, and receive the first weight file in the detector storage unit;

[0127] S3.4-2: Receive the second training dataset of the detector and the first weight file of the detector, load the parameters of the first weight file into the detector architecture of Faster-RCNN, input the second training dataset of the detector and retrain the detector according to the optimization method in step S3.2 to obtain the second weight file;

[0128] S3.4-3: Save the second weight file after training by the loss optimization unit;

[0129] S3.5-1: Obtain the image data to be detected from the first storage unit and the second weight file from the detector storage unit;

[0130] S3.5-2: Receive and load the second weight file parameters into the Faster-RCNN detector architecture, receive the image data to be detected, input it into the Faster-RCNN detector architecture with the weight file already loaded, and realize the detection of Plasmodium-infected cells in the image data to be detected;

[0131] S3.5-3: Image data of erythrocytes infected with Plasmodium and location information of erythrocytes infected with Plasmodium detected in the image data to be detected.

[0132] The training process for the category acquisition module is as follows:

[0133] S4.1-1: Obtain all positive and negative single-cell data obtained by the image transformation unit in the image enhancement module, and have experts label the positive single-cell data with the tag type: developmental stage - Plasmodium category;

[0134] S4.1-2: Adjust the single-cell data size of the sixth data acquisition unit to a fixed size of 224 pixels × 224 pixels, and add white fill to maintain the original image aspect ratio. Specific details are as follows:

[0135] Read each image one by one and save its original width and height; if the original image is landscape and the width is greater than the height, adjust the width and calculate the new height proportionally; otherwise, adjust the height and calculate the new width proportionally; calculate the number of pixels to fill in the four directions so that the final image size becomes the specified size; save the adjusted image according to the source image format;

[0136] S4.1-3: Used to receive the image after the size reset unit is adjusted, and to perform developmental stage classification and malaria parasite classification in sequence, to obtain four developmental stage classification labels and five trophozoite-infected red blood cell labels and output them;

[0137] S4.1-4: Receive the four developmental stage classification labels from the fourth storage unit. Since the data for the annular body is much larger than that for the other three classes, to prevent the model from overfitting to the majority class, the sampling strategy, classifier design, and loss function are optimized. The ResNet50 architecture classifier is then trained using the Adam optimizer. After training, the third weight file is obtained. The steps are as follows:

[0138] During the first stage of training, stages 1-3 of ResNet50 are frozen, and only stage 4 and the classifier are trained. Progressive balanced sampling is used, with each batch sampled according to the square root of the class frequency to mitigate class bias in the initial stage. The loss function adopts adaptive marginal loss. , for each category Set dynamic margins ,in, The original score for the true category. For category-adaptive marginal terms, The score for the non-true category. The minimum number of samples in the smallest class. Control the intensity of marginal growth; unfreeze the entire network during the second stage of training and add a penalty term to the loss function to constrain the model's complexity: ,in To be consistent with the sample A set of positive samples of the same type For the entire batch of samples, For temperature parameters; during the third stage of training, the feature extractor is frozen and only the classifier is trained, the sampling method is switched to balanced sampling, and four classes of samples are evenly extracted in each batch, forcing the model to pay equal attention to all classes;

[0139] S4.1-5: Receive the classification labels of five types of malaria parasite-infected red blood cells in the trophozoite stage from the fourth storage unit, train the ResNet50 architecture classifier through the Adam optimizer, and obtain the fourth weight file after training is completed;

[0140] S4.1-6: Store the third and fourth weight files after the first and second gradient update units have been trained, respectively. The weight files for the classification model are the third and fourth weight files.

[0141] S4.2-1: Obtain the image data of malaria parasite-infected cells in the third storage unit and the classification model weight file in the classifier storage unit;

[0142] S4.2-2: Adjust the size of the single-cell data in the seventh data acquisition unit to a fixed size of 224 pixels × 224 pixels, and add white fill to maintain the original image aspect ratio. The specific method is the same as step S4.1-2.

[0143] S4.2-3: The third weight file in the seventh data acquisition unit is loaded into ResNet50 to obtain the first classifier architecture. The image of Plasmodium-infected cells obtained by the size reset unit is used as the input of the first classifier. The first classifier outputs the developmental stage of different input cell images: ring body, trophozoite, schizont, and gametophyte, realizing the classification of developmental stages and outputting the first-level classification label.

[0144] S4.2-4: The fifth storage unit is used to store the first-level classification label output by the first classification unit and the image of Plasmodium-infected cells obtained by the size reset unit;

[0145] S4.2-5: The fourth weight file in the seventh data acquisition unit is loaded into ResNet50 to obtain the second classifier architecture. The trophozoite development stage image data in the fifth storage unit is used as the input of the second classifier architecture. The second classifier architecture outputs the malaria parasite categories in different input cell images: Plasmodium falciparum, Plasmodium vivax, Plasmodium malariae, Plasmodium ovale, and Plasmodium norotri, thus realizing the malaria parasite category analysis.

[0146] S4.2-6: Assign category information for all Plasmodium-infected red blood cells to the fourth storage unit based on the classification results output by the first and second classification units, and store them in folders in the format of developmental stage-Plasmodium category.

[0147] S5: Receives high-resolution images, location information of Plasmodium-infected red blood cells, and category information of Plasmodium-infected red blood cells, and locates and classifies Plasmodium-infected red blood cells on the high-resolution images.

[0148] By following the steps above, the initial system operation can be completed. To run the system again, follow these steps:

[0149] A1.1~A2.2: Same as S1.1~S2.2 above, and the image data to be detected for each high-resolution image can be obtained;

[0150] A3: Obtain the image data to be detected and the second weight file. Load the parameters of the second weight file and the image data to be detected into the detector architecture of Faster-RCNN to realize the detection of Plasmodium-infected cells in the image data to be detected; store the image data of Plasmodium-infected red blood cells detected in the image data to be detected and the location information of Plasmodium-infected red blood cells;

[0151] A4: Obtain image data and classification model weight file of erythrocytes infected with Plasmodium; then, as in S4.2-2 to S4.2-6 above, obtain the category information of erythrocytes infected with Plasmodium;

[0152] A5: Receives high-resolution images, location information of Plasmodium-infected red blood cells, and category information of Plasmodium-infected red blood cells, and locates and classifies Plasmodium-infected red blood cells on the high-resolution images.

[0153] Obviously, the above embodiments are merely examples for clear illustration and are not intended to limit the implementation. For those skilled in the art, other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all implementation methods here, and the obvious variations or modifications derived therefrom are still within the protection scope of this invention.

Claims

1. A rapid detection and auxiliary diagnostic system for bloodborne malaria parasites based on deep learning, characterized in that, include: The region localization module is used to locate the target region in low-resolution blood smear images and obtain the spatial location and high-resolution image of the target region. An image preprocessing module is used to receive the spatial location and high-resolution image, preprocess the high-resolution image, and obtain training image data and image data to be detected for training. The location acquisition module is used to receive the training image data for training to obtain an enhanced training dataset. After training, the location acquisition module receives the image data to be detected, detects, obtains, and stores the image data of Plasmodium infecting red blood cells and the location information of Plasmodium infecting red blood cells. The category acquisition module is used to receive the enhanced training dataset for training. After training, the category acquisition module receives image data of malaria parasite-infected red blood cells to classify malaria parasite-infected red blood cells and obtain the category information of malaria parasite-infected red blood cells. The results display module is used to receive high-resolution images, location information of Plasmodium-infected red blood cells, and category information of Plasmodium-infected red blood cells, and to locate and classify Plasmodium-infected red blood cells on the high-resolution images; The area positioning module includes: S1.1: First image acquisition unit, used to acquire blood smear images captured by camera equipment and preview the images to form a first dataset; S1.2: Dual-channel normalization unit, used to eliminate coloring differences between images in the first dataset through normalization methods; S1.3: Perceptual segmentation unit, used to achieve density-aware segmentation through adaptive grid density estimation; S1.4: Pushing trajectory analysis unit, used to determine the pushing direction of blood smear based on the density field estimation obtained from the sensing segmentation unit; S1.5: Single-cell layer positioning unit, used to determine the location of the single-cell layer in the latter 1 / 3 to 1 / 2 of the slide pushing path based on the density field thermogram and the slide pushing trajectory; S1.6: Spatial position extraction unit, used to determine the area to be scanned as a 1cm×1cm region centered on the blood boundary of the single cell layer distribution range within the single cell layer distribution range based on the blood boundary and single cell layer distribution. It outputs the spatial coordinates of the four corner points of the 1cm×1cm region as the position information of the analysis area and feeds the position information back to the scanning device, which then scans the high-resolution image.

2. The rapid detection and auxiliary diagnosis system for bloodborne malaria parasites based on deep learning as described in claim 1, characterized in that, The training image data contains original image data, original label data, and image data labeled as normal samples. The location acquisition module includes: a detector first-stage training module, a malaria parasite first-stage detection module, an image enhancement module, a detector second-stage training module, and a malaria parasite second-stage detection module. The detector first-stage training module receives and integrates the original image data and original label data to form a first training dataset and trains to obtain a first weight file. The malaria parasite first-stage detection module receives the image data labeled as normal samples and the first weight file, detects easily confused false positive single cells in the normal sample image data, and stores the image data and location information of false positive single cells. The image enhancement module receives the original image data, original label data, and image data of false positive single cells, and performs data enhancement on the first training dataset to obtain an enhanced training dataset. The detector second-stage training module receives the enhanced training dataset and the first weight file and trains to obtain a second weight file. The malaria parasite second-stage detection module receives the image data to be detected and the second weight file, detects, obtains, and stores the image data of malaria parasite-infected red blood cells and the location information of malaria parasite-infected red blood cells.

3. The rapid detection and auxiliary diagnosis system for bloodborne malaria parasites based on deep learning as described in claim 2, characterized in that, The category acquisition module includes a classifier training module and a malaria parasite analysis module. The classifier training module is used to receive the augmented training dataset, extract single-cell data, perform classification training, and obtain a classification model weight file. The malaria parasite analysis module is used to receive image data of malaria parasite-infected red blood cells and the classification model weight file, perform classification of malaria parasite-infected red blood cells, and obtain category information of malaria parasite-infected red blood cells.

4. The rapid detection and auxiliary diagnosis system for bloodborne malaria parasites based on deep learning as described in claim 3, characterized in that, The image preprocessing module includes: S2.1: The second image acquisition unit is used to acquire and crop a high-resolution image based on the position information of the receiving spatial position extraction unit. S2.2: Color normalization unit, which performs color normalization on the high-resolution image after cropping from the second image acquisition unit using the Reinhard color transfer algorithm; S2.3: First storage unit, used to store the original image data and original label data of each high-resolution image after passing through the image preprocessing module, the image data already marked as normal samples, and the image data to be detected.

5. The rapid detection and auxiliary diagnosis system for bloodborne malaria parasites based on deep learning as described in claim 4, characterized in that, The first-stage training module for the detector includes: S3.1-1: First data acquisition unit, used to receive the original image data and original label data in the first storage unit to construct the first training dataset; S3.1-2: Loss optimization unit, used to receive the first training dataset, and train the first weight file by using an adaptive time estimation algorithm and applying online hard example mining in the detection head; S3.1-3: Detector storage unit, used to store the first weight file after training by the loss optimization unit; The first-stage detection module for malaria parasites includes: S3.2-1: Second data acquisition unit, used to acquire image data labeled as normal samples in the first storage unit and the first weight file of the detector storage unit; S3.2-2: Target detection unit, used to receive and load the first weight file parameters into the detector architecture of Faster-RCNN, receive image data from the second data acquisition unit, load the Faster-RCNN detector architecture with the weight file, and realize the detection of easily confused false positive single cells in normal image data. S3.2-3: Second storage unit, used to store image data and location information of false positive single cells.

6. The rapid detection and auxiliary diagnosis system for bloodborne malaria parasites based on deep learning as described in claim 5, characterized in that, The image enhancement module includes: S3.3-1: The third data acquisition unit is used to receive false positive single-cell image data from the second storage unit and the original image data and original label data from the first storage unit, and to obtain and save true positive single-cell image data. S3.3-2: Image transformation unit, used to randomly rotate and adjust the contrast of all single-cell image data in the third data acquisition unit, and set the range of random factors to control the intensity of contrast adjustment; S3.3-3: Image selection unit, used to randomly select false positive single-cell image data and false negative single-cell image data output by the image transformation unit. Each selected image data is used as supplementary image data for the next step, and the category information corresponding to the image data is recorded. The category information is true positive or false positive. S3.3-4: Image fusion unit, used to overlay supplementary image data onto the original image data acquired by the third data acquisition unit for training, while ensuring that the overlaid area does not obscure the area annotated by the original label data, and generate new fused data; S3.3-5: Tag generation unit, when the supplementary image data contains true positive single cell image data, generates a new tag file by overlaying the true positive single cell image data in the supplementary image data onto the original image and obtaining the fused data; S3.3-6: Label alignment unit, used to merge data and rename their corresponding labels, ensuring that the data and label names are consistent, and obtaining augmented image data and corresponding label data for training.

7. The rapid detection and auxiliary diagnosis system for bloodborne malaria parasites based on deep learning as described in claim 6, characterized in that, The second-stage training module for the detector includes: S3.4-1: The fourth data acquisition unit is used to receive the augmented image data and label data used for training in the label alignment unit to construct the second training dataset for the detector, and to receive the first weight file in the detector storage unit; S3.4-2: Loss optimization unit, used to receive the second training dataset of the detector and the first weight file of the detector, load the parameters of the first weight file into the detector architecture of Faster-RCNN, and train to obtain the second weight file; S3.4-3: Detector storage unit, used to store the second weight file trained by the loss optimization unit; The second-stage malaria parasite detection module includes: S3.5-1: The fifth data acquisition unit is used to acquire the image data to be detected in the first storage unit and the second weight file in the detector storage unit; S3.5-2: Target detection unit, used to receive and load the second weight file parameters into the detector architecture of Faster-RCNN, receive the image data to be detected from the fifth data acquisition unit, and input it into the detector architecture of Faster-RCNN with the weight file loaded, so as to realize the detection of Plasmodium-infected cells in the image data to be detected; S3.5-3: The third storage unit is used to store the image data of malaria parasite-infected red blood cells detected in the image data to be detected and the location information of the malaria parasite-infected red blood cells.

8. The rapid detection and auxiliary diagnosis system for bloodborne malaria parasites based on deep learning as described in claim 7, characterized in that, The classifier training module includes: S4.1-1: The sixth data acquisition unit is used to acquire all positive and negative single-cell data obtained by the image transformation unit in the image enhancement module, and the positive single-cell data are labeled by experts. The label type is: developmental stage - malaria parasite category. S4.1-2: Size Reset Unit, used to adjust the size of single-cell data in the sixth data acquisition unit to a fixed size; S4.1-3: The fourth storage unit is used to receive the image after it has been adjusted by the size reset unit, and to perform developmental stage classification and malaria parasite classification in sequence, so as to obtain four developmental stage classification labels and five trophozoite-infected red blood cell classification labels and output them. S4.1-4: The first gradient update unit is used to receive the four developmental stage classification labels from the fourth storage unit and train the ResNet50 architecture classifier using an adaptive time estimation method. After training, the third weight file is obtained. S4.1-5: The second gradient update unit is used to receive the classification labels of five types of malaria parasite-infected red blood cells in the trophozoite stage from the fourth storage unit. The ResNet50 classifier is trained using an adaptive time estimation method, and the fourth weight file is obtained after training. S4.1-6: Classifier storage unit, used to store the third and fourth weight files after the first and second gradient update units have been trained, respectively. The classification model weight files are the third and fourth weight files.

9. The rapid detection and auxiliary diagnosis system for bloodborne malaria parasites based on deep learning as described in claim 8, characterized in that, The malaria parasite analysis module includes: S4.2-1: The seventh data acquisition unit is used to acquire the image data of malaria parasite-infected cells in the third storage unit and the classification model weight file in the classifier storage unit; S4.2-2: Size Reset Unit, used to adjust the size of single-cell data in the seventh data acquisition unit to a fixed size; S4.2-3: The first classification unit receives the third weight file from the seventh data acquisition unit and loads it into ResNet50 to obtain the first classifier architecture. It receives the image of Plasmodium-infected cells obtained by the size reset unit as the input of the first classifier. The first classifier outputs the developmental stages of different input cell images: ring body, trophozoite, schizont, and gametophyte, realizing the classification of developmental stages and outputting the first-level classification label. S4.2-4: The fifth storage unit is used to store the first-level classification label output by the first classification unit and the image of Plasmodium-infected cells obtained by the size reset unit; S4.2-5: The second classification unit receives the fourth weight file from the seventh data acquisition unit and loads it into ResNet50 to obtain the second classifier architecture. It receives trophozoite development stage image data from the fifth storage unit as input to the second classifier architecture. The second classifier architecture outputs the malaria parasite categories infecting different input cell images: Plasmodium falciparum, Plasmodium vivax, Plasmodium malariae, Plasmodium ovale, and Plasmodium norotri, thus realizing malaria parasite category analysis. S4.2-6: The sixth storage unit is used to assign category information to all erythrocytes infected with Plasmodium in the fourth storage unit based on the classification results output by the first and second classification units, and to store it in the developmental stage-Plasmodium category format.

Images