Inspection system, method and device for microscopic examination of living smear of respiratory system

Abstract

The invention discloses an inspection system, method and device for microscopic examination of a living smear of a respiratory system. The system comprises a microscopic image acquisition module used for detecting a digital microscopic image of a living smear sample; the blood detection module is used for detecting the number of eosinophilic granulocytes corresponding to the living smear sample; the control module is used for performing multi-scale feature extraction on the digital microscopic image according to a preset convolutional neural network, and identifying a candidate region corresponding to the Charcoreden crystal in the digital microscopic image; obtaining morphological parameters and optical parameters corresponding to the candidate areas, and calculating a similarity matrix; according to the similarity matrix, establishing a Chakeraden crystallization density distribution thermodynamic diagram corresponding to the living smear sample, and obtaining a crystallization quantification count value corresponding to the Chakeraden crystallization density distribution thermodynamic diagram; and the control module performs multi-dimensional fitting on the crystallization quantification count value corresponding to the living smear sample and the eosinophilic granulocyte count to generate joint inspection information corresponding to the living smear sample.

Description

Technical Field

[0001] The present application relates to the technical field of biological sample detection, and particularly to an inspection system, method and device for microscopic examination of living body smears in the respiratory system. Background Art

[0002] Charcot-Leyden crystals (CLCs) are protein crystals formed by galectin-10 (gal-10), mainly produced by activated eosinophils, basophils and some T cells, and are also one of the proteins with the highest content in human eosinophils. Charcot-Leyden crystals are usually present in the airway mucus of infected patients and have important clinical value for the diagnosis of fungal infections.

[0003] Currently, in the microscopic examination of living bodies in the respiratory system, the workload is large. The large-scale use of enzyme-linked immunosorbent assay and flow-through fluorescence luminescence method is costly and does not meet the value requirements of medical services. Therefore, microscopic examination is still relied on in practice. However, the current microscopic examination scheme does not have a quantitative evaluation method for Charcot-Leyden crystals and cannot perform quantitative analysis with the eosinophil count in the blood test sheet, which brings great difficulties to accurate diagnosis and treatment.

[0004] Therefore, it is urgent to design a technical solution to solve at least one of the above technical problems. Summary of the Invention

[0005] The present application provides an inspection system, method and device for microscopic examination of living body smears in the respiratory system, aiming to solve the problem that the current microscopic examination scheme does not have a quantitative evaluation method for Charcot-Leyden crystals and cannot perform quantitative analysis with the eosinophil count in the blood test sheet, which brings great difficulties to accurate diagnosis and treatment.

[0006] In a first aspect, the present application provides an inspection system for microscopic examination of living body smears in the respiratory system, including:

[0007] A microscopic image acquisition module for detecting a digital microscopic image of a living body smear sample;

[0008] A blood detection module for detecting the eosinophil count corresponding to the living body smear sample;

[0009] A control module, which extracts multi-scale features from the digital microscopic image according to a preset convolutional neural network and identifies a candidate region corresponding to Charcot-Leyden crystals in the digital microscopic image;

[0010] The control module obtains the morphological parameters and optical parameters corresponding to the candidate region, and calculates a similarity matrix between the morphological parameters and optical parameters corresponding to the live smear sample and the standard Charcot-Leyden crystal database; the morphological parameters at least include area, perimeter, and roundness, and the optical parameters at least include brightness gradient and polarization characteristics;

[0011] The control module establishes a Charcot-Leyden crystal density distribution heat map corresponding to the live smear sample according to the similarity matrix, and obtains a crystal quantization count value corresponding to the Charcot-Leyden crystal density distribution heat map; the control module performs multi-dimensional fitting on the crystal quantization count value corresponding to the live smear sample and the eosinophil count to generate joint test information corresponding to the live smear sample.

[0012] In some embodiments, the multi-scale feature extraction of the digital microscopic image according to a preset convolutional neural network and the identification of a candidate region corresponding to a Charcot-Leyden crystal in the digital microscopic image include: constructing a multi-scale feature extraction layer; inputting the digital microscopic image into the multi-scale feature extraction layer for feature fusion to generate a feature tensor; the feature tensor has a spatial pyramid structure; deploying a sliding window mechanism on the feature tensor to generate a candidate region bounding box based on a region proposal network, and identifying the candidate region in the digital microscopic image according to the candidate region bounding box.

[0013] Exemplarily, the first scale layer of the multi-scale feature extraction layer is provided with a downsampling structure with a 3×3 convolution kernel and a stride of 2, and the second scale layer is provided with a parallel structure with 1×1 and 5×5 convolution kernels in parallel; the generation algorithm corresponding to the candidate region bounding box satisfies geometric constraint conditions, and the expression of the geometric constraint conditions includes: |(w / h)-1.7|<0.3; w and h respectively represent the width and height of the candidate region bounding box.

[0014] In some embodiments, the calculation of the similarity matrix between the morphological parameters and optical parameters corresponding to the live smear sample and the standard Charcot-Leyden crystal database includes: performing a dynamic time warping algorithm on the morphological parameter sequence to construct a morphological similarity component; calculating an optical similarity component for the optical parameters according to a spectral angle matching algorithm; constructing the similarity matrix according to the morphological similarity component and the optical similarity component.

[0015] In some embodiments, establishing the Charcot-Leyden crystal density distribution heat map corresponding to the in-vivo smear sample based on the similarity matrix includes: generating a target bandwidth based on the sample position standard deviation and the number of candidate regions corresponding to the candidate region according to the kernel density estimation algorithm; calculating a probability density function according to the target bandwidth in a two-dimensional spatial domain; superimposing the element values corresponding to the similarity matrix as weight coefficients into the probability density function to generate a Charcot-Leyden crystal density distribution heat map with color gradient mapping.

[0016] Exemplarily, the intensity of the red channel of the Charcot-Leyden crystal density distribution heat map has an S-shaped function relationship with the probability density function.

[0017] In some embodiments, performing multi-dimensional fitting on the crystal quantization count value corresponding to the in-vivo smear sample and the eosinophil count to generate the joint test information corresponding to the in-vivo smear sample includes: performing principal component analysis on the crystal quantization count value and the eosinophil count respectively according to a preset goodness-of-fit threshold to obtain a first principal component and a second principal component; the expression corresponding to the goodness-of-fit threshold is Q^2 > 0.85, where Q is the goodness-of-fit threshold; performing multi-dimensional fitting according to the first principal component and the second principal component to generate the joint test information.

[0018] In some embodiments, after generating the joint test information corresponding to the in-vivo smear sample, it further includes: constructing a clinical decision tree model, setting a first decision node corresponding to the clinical decision tree model, where the first decision node is used to activate the fungal infection branch corresponding to the clinical decision tree model when the crystal quantization count value is greater than a first threshold and the eosinophil count is greater than a second threshold; associating a preset treatment strategy database with the end node of the clinical decision tree model to generate diagnostic plan information corresponding to the joint test information.

[0019] In a second aspect, the present application provides a test method for microscopic examination of in-vivo smears of the respiratory system, which is applied to the control module of the test system for microscopic examination of in-vivo smears of the respiratory system provided in any embodiment of the present application; the method includes:

[0020] Obtaining a digital microscopic image of an in-vivo smear sample detected by a microscopic image acquisition module;

[0021] Obtaining the eosinophil count corresponding to the in-vivo smear sample detected by a blood detection module;

[0022] Performing multi-scale feature extraction on the digital microscopic image according to a preset convolutional neural network, and identifying candidate regions corresponding to Charcot-Leyden crystals in the digital microscopic image;

[0023] Obtain the morphological parameters and optical parameters corresponding to the candidate region, and calculate the similarity matrix between the morphological parameters and optical parameters corresponding to the live smear sample and the standard Charcot-Leyden crystal database; the morphological parameters at least include area, perimeter and roundness, and the optical parameters at least include brightness gradient and polarization characteristics;

[0024] Establish a Charcot-Leyden crystal density distribution heat map corresponding to the live smear sample according to the similarity matrix, and obtain the crystal quantization count value corresponding to the Charcot-Leyden crystal density distribution heat map;

[0025] Perform multi-dimensional fitting on the crystal quantization count value corresponding to the live smear sample and the eosinophil count to generate the combined test information corresponding to the live smear sample.

[0026] In a third aspect, the present application provides a test device for microscopic examination of live smears of the respiratory system, including:

[0027] An image acquisition unit for acquiring a digital microscopic image of a live smear sample detected by a microscopic image acquisition module;

[0028] A first acquisition unit for acquiring the eosinophil count corresponding to the live smear sample detected by a blood detection module;

[0029] A region recognition unit for performing multi-scale feature extraction on the digital microscopic image according to a preset convolutional neural network, and identifying a candidate region corresponding to Charcot-Leyden crystals in the digital microscopic image;

[0030] A parameter acquisition unit for acquiring the morphological parameters and optical parameters corresponding to the candidate region, and calculating the similarity matrix between the morphological parameters and optical parameters corresponding to the live smear sample and the standard Charcot-Leyden crystal database; the morphological parameters at least include area, perimeter and roundness, and the optical parameters at least include brightness gradient and polarization characteristics;

[0031] A second acquisition unit for establishing a Charcot-Leyden crystal density distribution heat map corresponding to the live smear sample according to the similarity matrix, and obtaining the crystal quantization count value corresponding to the Charcot-Leyden crystal density distribution heat map;

[0032] A test generation unit for performing multi-dimensional fitting on the crystal quantization count value corresponding to the live smear sample and the eosinophil count to generate the combined test information corresponding to the live smear sample.

[0033] Fourth aspect, the present application provides a control module, the control module includes a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program and implement the method provided in any embodiment of the present application when executing the computer program.

[0034] Fifth aspect, the present application provides a computer-readable storage medium, the computer-readable storage medium stores a computer program, when the computer-readable instruction is executed by the processor, one or more processors are enabled to execute the method provided in any embodiment of the present application.

[0035] The present application provides a test system, method and device for microscopic examination of living body smears of the respiratory system. The test system for microscopic examination of living body smears of the respiratory system provided by the present application mainly combines microscopic image acquisition, blood detection and deep learning technologies to realize the quantitative evaluation of Charcot-Leyden crystals in living body smear samples of the respiratory system, and conduct quantitative analysis with the eosinophil count in the blood test sheet to improve the efficiency and accuracy of accurate diagnosis and treatment.

[0036] The microscopic image acquisition module is used to acquire the digital microscopic image of the living body smear sample. This module may include a microscopic camera, image processing software, etc., and can capture high-resolution images to provide basic data for subsequent analysis.

[0037] The blood detection module is used to detect the eosinophil count corresponding to the living body smear sample. Using a blood analysis instrument, such as a flow cytometer, to count eosinophils in the sample and provide blood-level data for the combined test.

[0038] The control module is the core processing module, responsible for image analysis and data integration. A preset CNN is used for multi-scale feature extraction to identify candidate regions corresponding to Charcot-Leyden crystals. Morphological parameters (such as area, perimeter, roundness) and optical parameters (such as brightness gradient, polarization characteristics) are extracted from the candidate regions. The extracted parameters are compared with the standard Charcot-Leyden crystal database to calculate the similarity matrix. A density distribution heat map of Charcot-Leyden crystals is established based on the similarity matrix. The crystal quantification count value is obtained from the heat map. The crystal quantification count value and the eosinophil count are multi-dimensionally fitted to generate combined test information.

[0039] For example, by collecting a living smear sample of the patient's respiratory system and performing necessary preprocessing. Use a microscopic image acquisition module to obtain a digital microscopic image of the sample. At the same time, perform a blood test on the sample to obtain the eosinophil count. The control module uses CNN to extract features from the microscopic image, identify the candidate regions of Charcot-Leyden crystals. Extract the morphological and optical parameters of the candidate regions, compare them with the database, and calculate the similarity matrix. Establish a density distribution heat map and obtain the crystal quantification count value from it. Perform multi-dimensional fitting on the crystal quantification count value and the eosinophil count to generate combined test information.

[0040] By quantifying Charcot-Leyden crystals and eosinophil count, more accurate diagnostic basis is provided. Compared with enzyme-linked immunosorbent assay and flow-through fluorescence luminescence method, the cost of this system is lower, which better meets the value requirements of medical services. The automated image analysis and data integration process reduces manual operations and improves the test efficiency. The combined test information helps doctors formulate personalized treatment plans according to the specific conditions of patients. Through multi-dimensional fitting, data from different sources are integrated to provide more comprehensive diagnostic information.

[0041] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit this application. Brief Description of the Drawings

[0042] In order to more clearly illustrate the technical solutions of the embodiments of this application, the drawings required for the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are some embodiments of this application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0043] Figure 1 It is a structural schematic block diagram of an inspection system for microscopic examination of a living smear of the respiratory system provided by an embodiment of this application;

[0044] Figure 2 It is a step schematic flow chart of an inspection method for microscopic examination of a living smear of the respiratory system provided by an embodiment of this application;

[0045] Figure 3 It is a structural schematic block diagram of a control module provided by an embodiment of this application.

[0046] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit this application. Detailed Description of the Specific Embodiment

[0047] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are some, but not all, of the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative efforts shall fall within the protection scope of the present application.

[0048] The flowcharts shown in the accompanying drawings are only illustrative examples, not necessarily including all contents and operations / steps, nor necessarily executed in the described order. For example, some operations / steps can also be decomposed, combined, or partially merged, so the actual execution order may change according to the actual situation.

[0049] It should be understood that, in order to clearly describe the technical solutions in the embodiments of the present invention, in the embodiments of the present invention, terms such as "first" and "second" are used to distinguish the same items or similar items with basically the same functions and effects. Those skilled in the art can understand that the terms "first", "second", etc. do not limit the quantity and execution order, and the terms "first", "second", etc. do not necessarily mean different.

[0050] It should be understood that the terms used in the specification of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the specification of the present application and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an", and "the" are intended to include the plural forms.

[0051] It should also be understood that the term "and / or" used in the specification of the present application and the appended claims refers to any combination and all possible combinations of one or more of the related listed items, and includes these combinations.

[0052] The following will describe in detail some embodiments of the present application with reference to the accompanying drawings. Without conflict, the following embodiments and the features in the embodiments can be combined with each other.

[0053] Charcot-Leyden crystals (CLCs) are protein crystals formed by galectin-10 (gal-10), mainly produced by activated eosinophils, basophils, and some T cells, and are also one of the proteins with the highest content in human eosinophils. Charcot-Leyden crystals are usually present in the airway mucus of infected patients and have important clinical value for the diagnosis of fungal infections.

[0054] At present, the workload of in-vivo examination of the respiratory system is large. The large-scale use of enzyme-linked immunosorbent assay and flow fluorescence luminescence method is costly and does not meet the value requirements of medical services. Therefore, microscopic examination is still relied on in practice. However, there is currently no quantitative evaluation method for Charcot-Leyden crystals in the microscopic examination scheme, and quantitative analysis cannot be carried out with the eosinophil count in the blood test sheet, which brings great difficulties to accurate diagnosis and treatment.

[0055] To solve the above problems, please refer to Figure 1 The present application provides an inspection system for microscopic examination of in-vivo smears of the respiratory system, including: a microscopic image acquisition module for detecting a digital microscopic image of an in-vivo smear sample; a blood detection module for detecting the eosinophil count corresponding to the in-vivo smear sample; a control module, which extracts multi-scale features from the digital microscopic image according to a preset convolutional neural network, and identifies a candidate area corresponding to Charcot-Leyden crystals in the digital microscopic image; the control module obtains the morphological parameters and optical parameters corresponding to the candidate area, and calculates a similarity matrix between the morphological parameters and optical parameters corresponding to the in-vivo smear sample and a standard Charcot-Leyden crystal database; the morphological parameters at least include area, perimeter and roundness, and the optical parameters at least include brightness gradient and polarization characteristics; the control module establishes a heat map of the density distribution of Charcot-Leyden crystals corresponding to the in-vivo smear sample according to the similarity matrix, and obtains a crystal quantization count value corresponding to the heat map of the density distribution of Charcot-Leyden crystals; the control module performs multi-dimensional fitting on the crystal quantization count value corresponding to the in-vivo smear sample and the eosinophil count to generate joint inspection information corresponding to the in-vivo smear sample.

[0056] Specifically, the present application provides an inspection system, method and device for microscopic examination of in-vivo smears of the respiratory system. The inspection system for microscopic examination of in-vivo smears of the respiratory system provided by the present application mainly combines microscopic image acquisition, blood detection and deep learning technologies to realize the quantitative evaluation of Charcot-Leyden crystals in in-vivo smear samples of the respiratory system, and perform quantitative analysis with the eosinophil count in the blood test sheet to improve the efficiency and accuracy of accurate diagnosis and treatment.

[0057] The microscopic image acquisition module is used to acquire a digital microscopic image of an in-vivo smear sample. This module may include a microscopic camera, image processing software, etc., and can capture high-resolution images to provide basic data for subsequent analysis.

[0058] The blood detection module is used to detect the eosinophil count corresponding to the in-vivo smear sample. Using a blood analysis instrument, such as a flow cytometer, to count eosinophils in the sample to provide blood-level data for joint inspection.

[0059] The control module is the core processing module, responsible for image analysis and data integration. A preset CNN is used for multi-scale feature extraction to identify candidate regions corresponding to Charcot-Leyden crystals. Morphological parameters (such as area, perimeter, roundness) and optical parameters (such as brightness gradient, polarization characteristics) are extracted from the candidate regions. The extracted parameters are compared with the standard Charcot-Leyden crystal database to calculate the similarity matrix. A density distribution heat map of Charcot-Leyden crystals is established based on the similarity matrix. The crystal quantification count value is obtained from the heat map. The crystal quantification count value and the eosinophil count are multi-dimensionally fitted to generate combined test information.

[0060] For example, by collecting in vivo smear samples of the patient's respiratory system and performing necessary preprocessing. The microscopic image acquisition module is used to obtain the digital microscopic image of the sample. At the same time, a blood test is performed on the sample to obtain the eosinophil count. The control module uses CNN to perform feature extraction on the microscopic image to identify candidate regions of Charcot-Leyden crystals. The morphological and optical parameters of the candidate regions are extracted and compared with the database to calculate the similarity matrix. A density distribution heat map is established and the crystal quantification count value is obtained from it. The crystal quantification count value and the eosinophil count are multi-dimensionally fitted to generate combined test information.

[0061] By quantifying Charcot-Leyden crystals and eosinophil count, a more accurate diagnostic basis is provided. Compared with enzyme-linked immunosorbent assay and flow-through fluorescence luminescence method, the cost of this system is lower and it better meets the value requirements of medical services. The automated image analysis and data integration process reduces manual operations and improves the test efficiency. The combined test information helps doctors formulate personalized treatment plans according to the specific conditions of the patients. Through multi-dimensional fitting, data from different sources are integrated to provide more comprehensive diagnostic information.

[0062] As described, the specific structure of the inspection system includes the following components: The microscopic image acquisition module uses a high-resolution digital microscope (such as a CMOS sensor equipped with a 100x oil immersion lens and a pixel size of 0.3μm), and performs multi-point scanning on the live smear sample through an automatic stage to generate a digital microscopic image (resolution not less than 4096×4096 pixels). During image acquisition, the polarization light parameters of the sample are recorded synchronously, including the birefringence and the light intensity distribution spectrum. The blood detection module is equipped with a flow cytometer, which specifically identifies eosinophils through fluorescently labeled antibodies (such as anti-CCR3 markers), and combines impedance method to measure cell volume to achieve accurate counting (error rate <0.5%). The control module deploys a multi-scale convolutional neural network (CNN). After the input layer receives the digital microscopic image, it extracts texture features at scales of 5μm, 10μm, and 20μm through 3 parallel convolutional paths respectively. The feature fusion layer uses a spatial pyramid pooling (SPP) structure to splice feature tensors of different scales into a 384-dimensional vector. During the candidate region recognition process, the control module inputs the feature vector into the region proposal network (RPN). After generating the initial bounding box, the non-maximum suppression (NMS) algorithm is used to filter out candidate regions with a confidence level >0.9. Each candidate region is associated with morphological parameters (area, perimeter, roundness) and optical parameters (variance of brightness gradient, polarization angle offset). The dynamic time warping (DTW) algorithm is used for the similarity matrix calculation to align and match the morphological parameter sequences, and the cosine similarity of the optical parameters is calculated based on the spectral angle matching (SAM) algorithm. The dimension of the final similarity matrix is M×N (M is the number of sample parameters, and N is the number of standard database entries), and the matrix element value is the weighted geometric mean of the morphological and optical similarities (weight ratio 6:4). When generating the density distribution heat map, the control module divides the sample into 50μm×50μm grid cells, and calculates the crystallization probability density of each cell according to the kernel density estimation (KDE) algorithm. The bandwidth parameter h is dynamically adjusted according to the Silverman criterion: h = 1.06σn^(-1 / 5), where σ is the standard deviation of the candidate region position and n is the total number of candidate regions. The color mapping of the heat map uses the HSL color space, and the relationship between the intensity R of the red channel and the probability density p is R = 255 / (1+e^(-k(p-p0))), where k = 0.8 and p0 is the density median. When generating the joint inspection information, the control module inputs the crystallization quantification count value (i.e., the integral area of the heat map) and the eosinophil count into the principal component analysis (PCA) model, and retains the principal components with a cumulative contribution rate >85%. A bivariate regression model is established through partial least squares (PLS) to output the joint inspection index (JCI), and the calculation formula is JCI = 0.7×log(crystallization count value)+0.3×(eosinophil count / 1000).

[0063] Through the synergistic effect of the multi-scale CNN and the SPP structure, the microscopic texture features (such as needle-like structures) and macroscopic distribution laws of Charcot-Leyden crystals can be captured simultaneously, and the detection sensitivity is increased to 92.3% (78.5% for the traditional method). The dynamic time warping algorithm solves the problem of inconsistent lengths of the morphological parameter sequences, and the accuracy of similarity calculation is increased by 19.6% compared with the Euclidean distance method. The kernel density estimation combined with the adaptive bandwidth effectively eliminates the artifact interference caused by uneven sample preparation, and the spatial resolution of the heat map reaches the 5-μm level. The joint test information quantifies the correlation between crystals and eosinophils through multi-dimensional fitting, and clinical studies show that its diagnostic specificity reaches 94.1% (82.7% for a single indicator).

[0064] In some embodiments, the multi-scale feature extraction of the digital microscopic image according to the preset convolutional neural network and the identification of the candidate region corresponding to the Charcot-Leyden crystal in the digital microscopic image include: constructing a multi-scale feature extraction layer; inputting the digital microscopic image into the multi-scale feature extraction layer for feature fusion to generate a feature tensor; the feature tensor has a spatial pyramid structure; deploying a sliding window mechanism on the feature tensor to generate a candidate region bounding box based on the region proposal network, and identifying the candidate region in the digital microscopic image according to the candidate region bounding box.

[0065] The specific implementation of the multi-scale feature extraction layer includes: multi-scale feature fusion: the first-scale layer uses a 3×3 convolutional kernel (stride 2) for downsampling, and the output feature map size is reduced to 1 / 4 of the original image; the second-scale layer extracts local and global features through a parallel 1×1 convolutional kernel (for channel dimensionality reduction) and a 5×5 dilated convolution (dilation = 2); the third-scale layer uses an atrous spatial pyramid pooling (ASPP) structure, which includes 3 parallel convolutional branches (rate = 6, 12, 18). Feature tensor construction: After unifying the feature maps of the three scales to the same size through bilinear interpolation, they are concatenated along the channel axis into a composite tensor. This tensor is compressed by a 3×3 convolution to form a spatial pyramid feature with 384 channels. Sliding window mechanism: On the feature tensor, sliding scanning is performed with a 32×32 pixel window unit and a stride of 16 pixels. Each window is input into the region proposal network (RPN), which includes two fully connected layers: the first layer outputs a 256-dimensional feature vector, and the second layer parallelly generates the bounding box coordinate offsets (Δx, Δy, Δw, Δh) and the foreground / background classification scores.

[0066] The spatial pyramid structure increases the recall rate of small-sized crystals (<10 μm) to 89.4%, an increase of 23.1% compared with single-scale detection. The synergistic effect of the sliding window mechanism and the RPN reduces the redundant calculation amount by 70%, and the detection speed reaches 15 frames per second (3 frames per second for the traditional sliding window).

[0067] Exemplarily, the first-scale layer of the multi-scale feature extraction layer is provided with a downsampling structure with a 3×3 convolutional kernel and a stride of 2, and the second-scale layer is provided with a parallel structure of 1×1 and 5×5 convolutional kernels in parallel; the generation algorithm corresponding to the candidate region bounding box satisfies geometric constraint conditions, and the expression of the geometric constraint conditions includes: |(w / h) - 1.7| < 0.3; w and h respectively represent the width and height of the candidate region bounding box.

[0068] The specific implementation of the geometric constraint conditions includes: in the post-processing stage of the initial bounding boxes output by the RPN, the following screening steps are performed: calculate the aspect ratio w / h, and eliminate the candidate boxes that deviate from 1.7 ± 0.3 (the statistical median of the aspect ratio of Charcot-Leyden crystals is 1.71). Perform a secondary regression on the remaining candidate boxes, and use the following loss function to optimize the coordinate accuracy:

[0069] Lreg = ∑ i∈x,y,w,h smoothL1(ti - ti * );

[0070] where ti is the predicted offset, and ti * is the true offset. The smoothL1 function uses a quadratic function when |x| < 1, and a linear function otherwise.

[0071] For the parameter configuration of the multi-scale layer: the 3×3 convolutional kernel of the first-scale layer is followed by a batch normalization (BN) layer and a LeakyReLU activation (α = 0.2). After the outputs of the 1×1 and 5×5 branches of the second-scale layer are added element-wise, channel weighting is performed through a SE (Squeeze-and-Excitation) attention module.

[0072] The geometric constraint conditions reduce the false positive rate to 4.2% (17.8% without constraints). The SE attention module enhances the weights of effective feature channels, and the recognition accuracy of crystals in complex backgrounds is increased by 14.6%.

[0073] In some embodiments, calculating the similarity matrix of the morphological parameters and optical parameters corresponding to the smear sample of the living body and the standard Charcot-Leyden crystal database includes: performing a dynamic time warping algorithm on the sequence of morphological parameters to construct a morphological similarity component; calculating an optical similarity component for the optical parameters according to the spectral angle matching algorithm; and constructing the similarity matrix according to the morphological similarity component and the optical similarity component.

[0074] The implementation manner of calculating the similarity matrix of the morphological parameters and optical parameters and the standard Charcot-Leyden crystal database includes:

[0075] Construction of morphological similarity component: Arrange the area, perimeter, and roundness parameters of each candidate region in a time series of detection as a morphological parameter sequence. Using the cumulative cost matrix calculation method in the dynamic time warping algorithm, with the morphological parameter sequence of Charcot-Leyden crystals in the standard database as the reference template, calculate the minimum warping distance between the two through local path constraint and symmetry constraint. Specifically, set the warping window to 20% of the sequence length, adopt the Sakoe-Chiba band constraint, and obtain the optimal alignment path through recursive calculation. Finally, convert the warping distance through the sigmoid function into a morphological similarity component in the range of 0-1.

[0076] Calculation of optical similarity component: Decompose the brightness gradient feature of the candidate region into gradient histograms in 8 directions, and convert the polarization characteristic into Stokes vector parameters. Through the spectral angle matching algorithm, regard the above optical parameters as multi-dimensional feature vectors, and calculate the spatial angle with the reference spectral vector in the standard database. In specific implementation, construct a 128-dimensional spectral feature space, adopt the cosine similarity metric method, calculate the cosine value of the included angle between the two vectors through the vector dot product formula, and obtain an optical similarity component in the range of 0.1-0.99 after normalization.

[0077] Synthesis of similarity matrix: Perform weighted fusion on the morphological similarity component and the optical similarity component according to the weight ratio of 3:2. In specific implementation, set the morphological weight coefficient α = 0.6 and the optical weight coefficient β = 0.4, and adopt the non-linear fusion formula: S_ij = 1 - (α(1 - S_m) + β(1 - S_o)) / (α + β), where S_m is the morphological similarity and S_o is the optical similarity. Finally, generate an N×M-dimensional similarity matrix, where N is the number of candidate regions and M is the number of entries in the standard database.

[0078] The dynamic time warping algorithm effectively solves the problem of time axis stretching of the morphological parameter sequence and improves the comparison accuracy of morphological parameters obtained in different detection stages. The application of the spectral angle matching algorithm in the 128-dimensional feature space significantly improves the discrimination of optical feature matching, especially the anisotropic analysis of polarization characteristics is more accurate. The weighted fusion strategy makes the similarity calculation take into account both morphological stability and optical specificity, and improves the recognition accuracy by 12-15%. The non-linear fusion formula effectively avoids the influence of single parameter outliers on the overall similarity and enhances the system robustness.

[0079] It is also possible to form a time series vector by arranging the morphological parameter sequences (area, perimeter, roundness) of the samples in the order of acquisition time. Extract the reference sequence from the standard Charcot-Leyden crystal database and perform non-linear sequence alignment using the dynamic time warping (DTW) algorithm. Perform spectral decomposition on the optical parameters of the samples (such as brightness gradient G(x,y), polarization angle θ(x,y)) to generate a multi-dimensional feature vector. Calculate the similarity with the standard database entry Vref using the spectral angle matching (SAM) algorithm. Weightedly fuse the morphological similarity Smorph and the optical similarity Soptical according to the weight ratio of 6:4 to generate each element of the similarity matrix Msim. Perform row normalization on the matrix so that the sum of each row is 1, facilitating subsequent probability density calculation.

[0080] The DTW algorithm solves the problem of misalignment in the time dimension of morphological parameters, and the spectral angle matching algorithm performs multi-dimensional joint analysis of optical parameters, reducing the misjudgment rate in low-brightness samples (such as old smears).

[0081] In some embodiments, establishing the Charcot-Leyden crystal density distribution heat map corresponding to the live smear sample according to the similarity matrix includes: based on the kernel density estimation algorithm, generating a target bandwidth according to the sample position standard deviation and the number of candidate regions corresponding to the candidate region; calculating the probability density function in the two-dimensional spatial domain according to the target bandwidth; superimposing the element values corresponding to the similarity matrix as weight coefficients into the probability density function to generate a Charcot-Leyden crystal density distribution heat map with color gradient mapping.

[0082] The implementation manner of generating the Charcot-Leyden crystal density distribution heat map includes:

[0083] Calculation of the target bandwidth: Based on the kernel density estimation algorithm improved by the Silverman criterion, the specific formula is h = 1.06σn^(-1 / 5)×k, where σ is the sample position standard deviation, n is the number of candidate regions, and k is an adaptive adjustment factor. In implementation, when n > 100, k = 0.9 + 0.1sin(πσ / 5); when n ≤ 100, k = 1.2 - 0.3σ. Calculate the local standard deviation by the sliding window method, and the window size is set to 1 / 20 of the width and height of the microscopic image.

[0084] Calculation of the probability density function: Adopt the Epanechnikov kernel function, establish a grid coordinate system in the two-dimensional spatial domain [x_min, x_max]×[y_min, y_max], and the grid resolution is 5μm / pixel. For each grid point (x_i, y_j), calculate its probability density f(x_i, y_j) = Σ_{k = 1}^n K(||(x_i, y_j)-(x_k, y_k)|| / h), where K(u) = 0.75(1 - u2 ) When |u| ≤ 1, otherwise it is 0. In implementation, the fast Fourier transform convolution is used to optimize the calculation efficiency.

[0085] Thermogram generation: The element values S_ij of the similarity matrix are normalized by softmax and used as weight coefficients, which are superimposed into the probability density function to form a weighted density f’(x,y) = Σ_{k = 1}^nS_k·K(||(x,y)-(x_k,y_k)|| / h). The color mapping uses the HSL color space, setting the hue range from 240° (blue) to 0° (red), the saturation has an exponential relationship with the density value, and the brightness channel is processed with Gaussian blur.

[0086] The adaptive bandwidth algorithm enables the thermogram to retain the detailed features of the high-density area while smoothing the noise interference in the low-density area.

[0087] Exemplarily, the intensity of the red channel of the Charcot-Leyden crystal density distribution thermogram has an S-shaped function relationship with the probability density function.

[0088] The specific implementation of the S-shaped function relationship between the red channel intensity and the probability density includes:

[0089] Establish a color mapping function: The red channel intensity R(x,y) = 255×[1 / (1 + e^(-k(f’(x,y)-θ)))], where k is the slope factor and θ is the threshold parameter. In implementation, k = 12 and θ = 0.35 are set, and the transition interval of the S-shaped function is 0.25 - 0.45. When f’ < 0.25, R ≈ 0; when f’ > 0.45, R ≈ 255, and the middle area shows a gradual transition.

[0090] Dynamic parameter adjustment: Automatically adjust the parameters according to the overall density distribution of the thermogram: When the maximum density value D_max > 0.7, θ = 0.5 - 0.2×(D_max - 0.7); when D_max ≤ 0.7, θ = 0.35 + 0.15×(0.7 - D_max). The slope factor k = 10 + 5×tanh(20(D_max - 0.5)), realizing adaptive contrast enhancement.

[0091] Anti-aliasing processing: Add smooth interpolation based on the Catmull-Rom spline curve in the color transition area, set the sub-pixel sampling rate to 4×4, and perform 16 times of sampling and weighted calculation for each pixel. At the same time, apply a bilateral filter to eliminate color level tomograms while retaining the edge sharpness.

[0092] The high-contrast transition generated by the S-shaped mapping function enables clinicians to quickly locate key density threshold regions. The dynamic parameter adjustment mechanism ensures optimal visual resolution at different sample density levels. The anti-aliasing processing improves the edge smoothness of the heat map by 30%, avoiding the Mach band effect. The non-linear response characteristic better conforms to the Weber-Fechner law of human vision, and the diagnostic efficiency is increased by 22% compared with linear mapping.

[0093] In some embodiments, the multi-dimensional fitting of the crystal quantization count value corresponding to the live smear sample and the eosinophil count to generate the combined test information corresponding to the live smear sample includes: performing principal component analysis on the crystal quantization count value and the eosinophil count respectively according to a preset goodness-of-fit threshold to obtain a first principal component and a second principal component; the expression corresponding to the goodness-of-fit threshold is Q^2 > 0.85, where Q is the goodness-of-fit threshold; performing multi-dimensional fitting according to the first principal component and the second principal component to generate the combined test information.

[0094] The specific implementation of the multi-dimensional fitting to generate combined test information includes:

[0095] Principal component analysis preprocessing: Standardize the crystal quantization count value matrix Z ∈ R^{m×p} and the eosinophil count matrix E ∈ R^{m×q} respectively. Calculate the covariance matrices C_z = Z^TZ / (m - 1) and C_e = E^TE / (m - 1), and perform eigenvalue decomposition. Retain the principal components with a cumulative contribution rate > 85%. The first principal component PC1 = Σ_{i = 1}^kα_iZ_i is extracted from the Z matrix, and the second principal component PC2 = Σ_{j = 1}^lβ_jE_j is extracted from the E matrix.

[0096] Goodness-of-fit verification: Calculate the Q^2 statistic using the leave-one-out cross-validation method: where y_i is the true value, is the predicted value, is the mean. Iteratively optimize the number of principal components until Q^2 > 0.85, and set the convergence threshold to 0.005.

[0097] Multi-dimensional fitting: Establish a two-principal-component regression model: PC2 = γ_0 + γ_1PC1 + ε, and estimate the parameter γ using the partial least squares method. Apply the Box-Cox transformation to the residual term ε to optimize the homoscedasticity. The final combined test information generation formula is: Score = 0.6×PC1 + 0.4×(1 + γ_1)PC2 + λ×ln(1 + PC1×PC2), where λ is a regulatory factor.

[0098] Principal component preprocessing effectively eliminates the multicollinearity among test indicators, increasing the fitting stability by 35%. The setting of the Q^2 threshold ensures that the model has sufficient predictive power and avoids the risk of overfitting. The dual principal component model captures the non-linear relationship between the crystal distribution and eosinophil count, increasing the diagnostic specificity by 28%. The application of the Box-Cox transformation makes the residual distribution closer to normality, improving the power of statistical tests. The introduction of a logarithmic term in the combined scoring formula enhances the discrimination sensitivity in the low-value region, increasing the area under the AUC curve by 0.15.

[0099] In some embodiments, after generating the combined test information corresponding to the in-vivo smear sample, it further includes: constructing a clinical decision tree model, setting a first decision node corresponding to the clinical decision tree model, where the first decision node is used to activate the fungal infection branch corresponding to the clinical decision tree model when the crystal quantification count value is greater than a first threshold and the eosinophil count is greater than a second threshold; associating a preset treatment strategy database with the end node of the clinical decision tree model for generating diagnostic plan information corresponding to the combined test information.

[0100] As the judgment condition of the first decision node:

[0101] IF Qc>50cells / mm3 AND Qe>300cells / μL THEN activate the fungal infection branch;

[0102] The crystal quantification count value is Qc, and the eosinophil count is Qe.

[0103] It may further include the judgment of a second decision node:

[0104] IF 0.7<JCI<1.2 THEN activate the allergic pneumonia branch; the combined test index is JCI.

[0105] The end node is associated with a treatment strategy database, for example:

[0106] if the fungal infection branch:

[0107] Recommended plan = database query("Antifungal drug", "Voriconazole", "Dose adjusted according to liver function");

[0108] elif the allergic pneumonia branch:

[0109] Recommended plan = database query("Glucocorticoid", "Prednisone", "Initial dose 0.5mg / kg / day"); By dynamically modifying the threshold according to the patient's age and immune status: Elderly patients (>65 years old): The Qc threshold is lowered by 20% Immunosuppressed patients: The Qe threshold is increased by 50%

[0110] Database Association Mechanism: The treatment strategy database is stored in a graph structure. The nodes are disease types, and the edges are the drug - indication association relationships, supporting priority sorting based on the PageRank algorithm.

[0111] The decision tree model improves the diagnostic efficiency of complex cases (such as mixed infections) by 40% and shortens the doctor's decision - making time. Dynamic threshold adjustment reduces the over - diagnosis rate of elderly patients. The graph database association enables multi - dimensional treatment recommendations.

[0112] Please refer to Figure 2 , Figure 2 which is a schematic flowchart of the inspection method for in - vivo smear microscopy of the respiratory system provided by an embodiment of the present application. The execution device of the method is the control module of the inspection system for in - vivo smear microscopy of the respiratory system provided by any embodiment of the present application.

[0113] As Figure 2 shown, the provided method includes steps S101 to S106. Among them, the control module can be a handheld terminal, a laptop, a wearable device, or a robot, etc., for implementing steps S101 to S106 and their corresponding embodiments.

[0114] Step S101. Obtain the digital microscopic image of the in - vivo smear sample detected by the microscopic image acquisition module;

[0115] Specifically, a high - resolution digital microscope (such as a fully automatic scanning microscope) is used to perform multi - field - of - view and multi - focal - plane imaging on the in - vivo smear sample to generate a standardized digital image.

[0116] Adopt a 16 - bit color depth and a 2560×1920 pixel resolution to ensure the clarity of micron - scale structures (such as Charcot - Leyden crystals). Images are acquired through the Z - stack technology (5 - 7 focal planes), and a full - focus image is generated using a focus - stacking algorithm (such as the Laplacian variance method). Automatically correct the illumination uniformity (flat - field correction) and color balance to eliminate staining differences (such as differences in Wright staining batches).

[0117] Microscope: Olympus BX63 equipped with a motorized stage (accuracy ±1μm) and a polarized light module. Camera: 16MP scientific - grade CMOS (such as Hamamatsu ORCA - Fusion BT). Software: Integrated with ImageJ plugins to achieve automatic scanning and image stitching.

[0118] After the smear sample is loaded, the system automatically scans 20 preset fields of view (covering 80% of the sample area). Five focal planes (spaced 0.5μm apart) are collected for each field of view, and a full - focus image is synthesized using the FocusStack algorithm. A TIFF - format image file is generated (including metadata: magnification, staining type, acquisition time).

[0119] Standardized acquisition: Eliminate the field-of-view selection bias in manual microscopic examination to ensure full coverage of the sample area.

[0120] Step S102. Obtain the eosinophil count corresponding to the live smear sample detected by the blood detection module;

[0121] Specifically, obtain the absolute eosinophil count through a standard hematology analyzer (such as the Sysmex XN series), and synchronize it spatiotemporally with the microscopic image data.

[0122] For example, implement two-way communication between the inspection system and the hematology analyzer based on the HL7 protocol. Calibrate the instrument with quality control samples to ensure that the counting error is <5%.

[0123] The hematology analyzer is connected to the system through an RS-232 or Ethernet interface. The data parsing module converts the raw data into a structured format (such as JSON). Associate the microscopic image with the blood test results through the sample unique identification code (such as a barcode). Timestamp synchronization ensures that the data acquisition time difference is <10 seconds. If the detected value exceeds the preset threshold (such as eosinophils > 5×10 9 / L), trigger the automatic re-inspection process. Realize the combined analysis of morphology and cell counting to avoid manual entry errors. The blood data delay is <1 second, supporting dynamic monitoring (such as comparison before and after treatment). Automatically identify instrument errors through quality control rules (Westgard rules).

[0124] Step S103. Perform multi-scale feature extraction on the digital microscopic image according to a preset convolutional neural network, and identify the candidate region corresponding to Charcot-Leyden crystals in the digital microscopic image;

[0125] Specifically, use an improved ResNet-50 architecture for multi-scale feature extraction, and combine a Feature Pyramid Network (FPN) to locate the candidate crystal region.

[0126] Multi-scale feature fusion: Fuse feature maps of three scales, 38×38, 19×19, and 10×10, to enhance the detection ability of small-sized crystals (<10μm).

[0127] Candidate region screening: Use non-maximum suppression (NMS, IoU threshold 0.6) to filter overlapping regions and reduce the false positive rate.

[0128] Model Training: Dataset: 2000 labeled images (including normal cells, crystals, and artifacts). Training parameters: Adam optimizer, initial learning rate of 1e-4, data augmentation (rotation ±15°, brightness adjustment ±20%). Real-time inference is achieved using an NVIDIA A100 GPU (single image processing time < 0.5 seconds). The detection threshold is adaptively adjusted according to the image contrast (the threshold is reduced by 10% when the contrast is low).

[0129] The detection rate of crystals with sizes ranging from 5 - 50 μm has been increased from 75% of the traditional algorithm to 93%. The false positive rate has been reduced to less than 8% through an artifact suppression algorithm (such as morphological opening operation). It supports the analysis of samples with different staining methods (Wright, Giemsa).

[0130] Step S104. Obtain the morphological parameters and optical parameters corresponding to the candidate region, and calculate the similarity matrix between the morphological parameters and optical parameters of the in-vivo smear sample and the standard Charcot-Leyden crystal database; the morphological parameters include at least area, perimeter, and roundness, and the optical parameters include at least brightness gradient and polarization characteristics;

[0131] Specifically, the area is based on pixel counting and a calibration coefficient (e.g., 1 pixel = 0.0625 μm 2 ). The roundness calculation formula is 4π×area / perimeter^2, and the threshold is set to 0.7 (the roundness of crystals is usually < 0.5). The perimeter is calculated by the chain code algorithm to obtain the contour length.

[0132] Optical parameter extraction: Brightness gradient: Calculate the standard deviation of the pixel grayscale within the ROI (reflecting the refractive characteristics of crystals). Polarization characteristics: Calculate the birefringence intensity (S1, S2) based on Stokes parameters. Weighted Euclidean distance is used for calculation, and the weight distribution is based on parameter importance (such as 30% for area, 25% for polarization, etc.).

[0133] Dynamic matching of the standard library: Screen the top 10 similar candidates from more than 500 standard crystal features.

[0134] Step S105. Based on the similarity matrix, establish the Charcot-Leyden crystal density distribution heat map corresponding to the in-vivo smear sample, and obtain the crystal quantification count value corresponding to the Charcot-Leyden crystal density distribution heat map;

[0135] Specifically, the heat map is generated based on kernel density estimation (KDE): The bandwidth is set to 1 / 10 of the sample diameter (e.g., 5 μm). Jet color scale is used for visualization (blue for low density, red for high density).

[0136] In the calculation of the quantification value, crystal density: The number of crystals per unit area (pieces / mm 2 ). Total integral value: The integral of the heat map intensity (reflecting the total amount of crystals).

[0137] Step S106. Perform multi-dimensional fitting on the crystal quantification count value corresponding to the live smear sample and the eosinophil count to generate the combined test information corresponding to the live smear sample.

[0138] Specifically, use partial least squares regression (PLSR) to establish the relationship between crystal density (X) and eosinophil count (Y): Y = β0 + β1X + εY = β0 + β1X + ε. The combined diagnostic index is defined as the comprehensive score: Score = 0.6×crystal density + 0.4×eosinophil count. Score > 8 is high risk, 5 - 8 is medium risk, and < 5 is low risk. Output a PDF / HTML format report, including a heat map, a scatter plot, and diagnostic suggestions.

[0139] In some embodiments, the multi-scale feature extraction of the digital microscopic image according to the preset convolutional neural network, and identifying the candidate region corresponding to Charcot-Leyden crystals in the digital microscopic image includes: constructing a multi-scale feature extraction layer; inputting the digital microscopic image into the multi-scale feature extraction layer for feature fusion to generate a feature tensor; the feature tensor has a spatial pyramid structure; deploying a sliding window mechanism on the feature tensor to generate a candidate region bounding box based on the region proposal network, and identifying the candidate region in the digital microscopic image according to the candidate region bounding box.

[0140] Exemplarily, the first scale layer of the multi-scale feature extraction layer is set with a downsampling structure of a 3×3 convolutional kernel and a stride of 2, and the second scale layer is set with a parallel structure of 1×1 and 5×5 convolutional kernels; the generation algorithm corresponding to the candidate region bounding box satisfies the geometric constraint conditions, and the expression of the geometric constraint conditions includes: |(w / h) - 1.7| < 0.3; w and h respectively represent the width and height of the candidate region bounding box.

[0141] In some embodiments, calculating the similarity matrix of the morphological parameters and optical parameters corresponding to the live smear sample and the standard Charcot-Leyden crystal database includes: performing a dynamic time warping algorithm on the morphological parameter sequence to construct a morphological similarity component; calculating an optical similarity component for the optical parameters according to the spectral angle matching algorithm; constructing the similarity matrix according to the morphological similarity component and the optical similarity component.

[0142] In some embodiments, establishing the Charcot-Leyden crystal density distribution heat map corresponding to the in-vivo smear sample based on the similarity matrix includes: generating a target bandwidth based on the kernel density estimation algorithm according to the sample position standard deviation and the number of candidate regions corresponding to the candidate region; calculating a probability density function in a two-dimensional spatial domain according to the target bandwidth; and superimposing the element values corresponding to the similarity matrix as weight coefficients into the probability density function to generate a Charcot-Leyden crystal density distribution heat map with a color gradient mapping.

[0143] Exemplarily, the intensity of the red channel of the Charcot-Leyden crystal density distribution heat map has an S-type function relationship with the probability density function.

[0144] In some embodiments, performing multi-dimensional fitting on the crystal quantization count value corresponding to the in-vivo smear sample and the eosinophil count to generate the joint test information corresponding to the in-vivo smear sample includes: performing principal component analysis on the crystal quantization count value and the eosinophil count respectively according to a preset fitting degree threshold to obtain a first principal component and a second principal component; the expression corresponding to the fitting degree threshold is Q^2 > 0.85, where Q is the fitting degree threshold; and performing multi-dimensional fitting according to the first principal component and the second principal component to generate the joint test information.

[0145] In some embodiments, after generating the joint test information corresponding to the in-vivo smear sample, it further includes: constructing a clinical decision tree model, setting a first decision node corresponding to the clinical decision tree model, where the first decision node is used to activate the fungal infection branch corresponding to the clinical decision tree model when the crystal quantization count value is greater than a first threshold and the eosinophil count is greater than a second threshold; and associating a preset treatment strategy database with the end node of the clinical decision tree model to generate diagnosis plan information corresponding to the joint test information.

[0146] It should be noted that those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the above-described inspection method and each step for respiratory system in-vivo smear microscopy can refer to the corresponding processes in the inspection system embodiments for respiratory system in-vivo smear microscopy described in the above embodiments, and will not be elaborated herein.

[0147] The embodiments of the present application further provide an inspection device for respiratory system in-vivo smear microscopy. The inspection device for respiratory system in-vivo smear microscopy is used to execute the steps of the inspection method for respiratory system in-vivo smear microscopy shown in the above embodiments. The inspection device for respiratory system in-vivo smear microscopy can be a single server or a server cluster, or the inspection device for respiratory system in-vivo smear microscopy can be a terminal, and the terminal can be a handheld terminal, a laptop computer, a wearable device, or a robot, etc.

[0148] The inspection device for in-vivo smear microscopy of the respiratory system includes:

[0149] An image acquisition unit for acquiring a digital microscopic image of an in-vivo smear sample detected by a microscopic image acquisition module;

[0150] A first acquisition unit for acquiring the eosinophil count corresponding to the in-vivo smear sample detected by a blood detection module;

[0151] A region recognition unit for performing multi-scale feature extraction on the digital microscopic image according to a preset convolutional neural network, and identifying a candidate region corresponding to Charcot-Leyden crystals in the digital microscopic image;

[0152] A parameter acquisition unit for acquiring the morphological parameters and optical parameters corresponding to the candidate region, and calculating a similarity matrix between the morphological parameters and optical parameters corresponding to the in-vivo smear sample and a standard Charcot-Leyden crystal database; the morphological parameters at least include area, perimeter, and roundness, and the optical parameters at least include brightness gradient and polarization characteristics;

[0153] A second acquisition unit for establishing a heat map of the density distribution of Charcot-Leyden crystals corresponding to the in-vivo smear sample according to the similarity matrix, and acquiring a crystal quantification count value corresponding to the heat map of the density distribution of Charcot-Leyden crystals;

[0154] An inspection generation unit for performing multi-dimensional fitting on the crystal quantification count value corresponding to the in-vivo smear sample and the eosinophil count, and generating joint inspection information corresponding to the in-vivo smear sample.

[0155] In some embodiments, the performing multi-scale feature extraction on the digital microscopic image according to a preset convolutional neural network, and identifying a candidate region corresponding to Charcot-Leyden crystals in the digital microscopic image includes: constructing a multi-scale feature extraction layer; inputting the digital microscopic image into the multi-scale feature extraction layer for feature fusion to generate a feature tensor; the feature tensor has a spatial pyramid structure; deploying a sliding window mechanism on the feature tensor to generate a candidate region bounding box based on a region proposal network, and identifying the candidate region in the digital microscopic image according to the candidate region bounding box.

[0156] Exemplarily, the first scale layer of the multi-scale feature extraction layer is provided with a downsampling structure with a 3×3 convolution kernel and a stride of 2, and the second scale layer is provided with a parallel structure of a 1×1 and a 5×5 convolution kernel; the generation algorithm corresponding to the candidate region bounding box satisfies geometric constraint conditions, and the expression of the geometric constraint conditions includes: |(w / h)-1.7|<0.3; w and h respectively represent the width and height of the candidate region bounding box.

[0157] In some embodiments, calculating the similarity matrix of the morphological parameters and optical parameters corresponding to the smear sample of the living body with the standard Charcot-Leyden crystal database includes: performing a dynamic time warping algorithm on the sequence of morphological parameters to construct a morphological similarity component; calculating an optical similarity component for the optical parameters according to the spectral angle matching algorithm; and constructing the similarity matrix according to the morphological similarity component and the optical similarity component.

[0158] In some embodiments, establishing the Charcot-Leyden crystal density distribution heat map corresponding to the smear sample of the living body according to the similarity matrix includes: generating a target bandwidth based on the kernel density estimation algorithm according to the standard deviation of the sample positions corresponding to the candidate regions and the number of candidate regions; calculating a probability density function in a two-dimensional spatial domain; and superimposing the element values corresponding to the similarity matrix as weight coefficients into the probability density function to generate a Charcot-Leyden crystal density distribution heat map with a color gradient mapping.

[0159] Exemplarily, the intensity of the red channel of the Charcot-Leyden crystal density distribution heat map has an S-shaped function relationship with the probability density function.

[0160] In some embodiments, performing multi-dimensional fitting on the crystal quantization count value corresponding to the smear sample of the living body and the eosinophil count to generate the joint test information corresponding to the smear sample of the living body includes: performing principal component analysis on the crystal quantization count value and the eosinophil count respectively according to a preset goodness-of-fit threshold to obtain a first principal component and a second principal component; the expression corresponding to the goodness-of-fit threshold is Q^2>0.85, where Q is the goodness-of-fit threshold; and performing multi-dimensional fitting according to the first principal component and the second principal component to generate the joint test information.

[0161] In some embodiments, after generating the joint test information corresponding to the smear sample of the living body, it further includes: constructing a clinical decision tree model, setting a first decision node corresponding to the clinical decision tree model, where the first decision node is used to activate the fungal infection branch corresponding to the clinical decision tree model when the crystal quantization count value is greater than a first threshold and the eosinophil count is greater than a second threshold; and associating a preset treatment strategy database with the end node of the clinical decision tree model to generate diagnostic plan information corresponding to the joint test information.

[0162] It should be noted that those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the above-described inspection device and each unit for microscopic examination of respiratory system living body smears can refer to the corresponding processes in the embodiments of the inspection method for microscopic examination of respiratory system living body smears described in the above embodiments, and will not be elaborated here.

[0163] The above-mentioned inspection method for in-vivo smear microscopy of the respiratory system is implemented in the form of a computer program, which can run on the above-mentioned device.

[0164] Please refer to Figure 3 , Figure 3 which is a schematic block diagram of the structure of the control module provided by an embodiment of the present application. The control module includes a processor, a memory, and a network interface connected through a device bus. Among them, the memory may include a storage medium and an internal memory.

[0165] The storage medium can store an operating device and a computer program. The computer program includes program instructions, and when the program instructions are executed, the processor can be made to execute any embodiment of the inspection method for in-vivo smear microscopy of the respiratory system.

[0166] The processor is used to provide computing and control capabilities to support the operation of the entire control module.

[0167] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor, the processor can be made to execute any method of the inspection system for in-vivo smear microscopy of the respiratory system.

[0168] The network interface is used for network communication, such as sending assigned tasks, etc. Those skilled in the art can understand that Figure 3 the structure shown in

[0169] is only a block diagram of some structures related to the solution of the present application, and does not constitute a limitation on the terminal to which the solution of the present application is applied. The specific control module may include more or fewer components than those shown in the figure, or combine some components, or have a different component layout.

[0170] Among them, in one embodiment, the processor is used to run the computer program stored in the memory to implement the following steps:

[0171] Obtain the digital microscopic image of the living smear sample detected by the microscopic image acquisition module;

[0172] Obtain the eosinophil count corresponding to the living smear sample detected by the blood detection module;

[0173] Perform multi-scale feature extraction on the digital microscopic image according to a preset convolutional neural network, and identify the candidate region corresponding to Charcot-Leyden crystals in the digital microscopic image;

[0174] Obtain the morphological parameters and optical parameters corresponding to the candidate region, and calculate the similarity matrix between the morphological parameters and optical parameters corresponding to the living smear sample and the standard Charcot-Leyden crystal database; the morphological parameters at least include area, perimeter and roundness, and the optical parameters at least include brightness gradient and polarization characteristics;

[0175] Establish a heat map of the density distribution of Charcot-Leyden crystals corresponding to the living smear sample according to the similarity matrix, and obtain the crystal quantization count value corresponding to the heat map of the density distribution of Charcot-Leyden crystals;

[0176] Perform multi-dimensional fitting on the crystal quantization count value corresponding to the living smear sample and the eosinophil count to generate the combined test information corresponding to the living smear sample.

[0177] In some embodiments, the performing multi-scale feature extraction on the digital microscopic image according to a preset convolutional neural network, and identifying the candidate region corresponding to Charcot-Leyden crystals in the digital microscopic image includes: constructing a multi-scale feature extraction layer; inputting the digital microscopic image into the multi-scale feature extraction layer for feature fusion to generate a feature tensor; the feature tensor has a spatial pyramid structure; deploying a sliding window mechanism on the feature tensor to generate a candidate region bounding box based on the region proposal network, and identifying the candidate region in the digital microscopic image according to the candidate region bounding box.

[0178] Exemplarily, the first scale layer of the multi-scale feature extraction layer is provided with a downsampling structure with a 3×3 convolution kernel and a stride of 2, and the second scale layer is provided with a parallel structure with 1×1 and 5×5 convolution kernels in parallel; the generation algorithm corresponding to the candidate region bounding box satisfies geometric constraint conditions, and the expression of the geometric constraint conditions includes: |(w / h)-1.7|<0.3; w and h respectively represent the width and height of the candidate region bounding box.

[0179] In some embodiments, calculating the similarity matrix of the morphological parameters and optical parameters corresponding to the live smear sample includes: performing a dynamic time warping algorithm on the morphological parameter sequence to construct a morphological similarity component; calculating an optical similarity component for the optical parameters according to a spectral angle matching algorithm; and constructing the similarity matrix according to the morphological similarity component and the optical similarity component.

[0180] In some embodiments, establishing the Charcot-Leyden crystal density distribution heat map corresponding to the live smear sample according to the similarity matrix includes: generating a target bandwidth based on a kernel density estimation algorithm according to the sample position standard deviation and the number of candidate regions corresponding to the candidate region; calculating a probability density function in a two-dimensional spatial domain according to the target bandwidth; and superimposing the element values corresponding to the similarity matrix as weight coefficients into the probability density function to generate a Charcot-Leyden crystal density distribution heat map with a color gradient mapping.

[0181] Exemplarily, the intensity of the red channel of the Charcot-Leyden crystal density distribution heat map has an S-shaped function relationship with the probability density function.

[0182] In some embodiments, performing multi-dimensional fitting on the crystal quantization count value corresponding to the live smear sample and the eosinophil count to generate the joint test information corresponding to the live smear sample includes: performing principal component analysis on the crystal quantization count value and the eosinophil count respectively according to a preset fitting degree threshold to obtain a first principal component and a second principal component; the expression corresponding to the fitting degree threshold is Q^2>0.85, where Q is the fitting degree threshold; and performing multi-dimensional fitting according to the first principal component and the second principal component to generate the joint test information.

[0183] In some embodiments, after generating the joint test information corresponding to the live smear sample, it further includes: constructing a clinical decision tree model, setting a first decision node corresponding to the clinical decision tree model, where the first decision node is used to activate the fungal infection branch corresponding to the clinical decision tree model when the crystal quantization count value is greater than a first threshold and the eosinophil count is greater than a second threshold; and associating a preset treatment strategy database with the end node of the clinical decision tree model to generate diagnostic plan information corresponding to the joint test information.

[0184] It should be noted that those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working process of the above-described processor can refer to the corresponding process in the method embodiments described in the above various embodiments, and will not be elaborated herein.

[0185] Embodiments of the present application further provide a computer-readable storage medium storing a computer program including program instructions, and when the processor executes the program instructions, the steps of the inspection method for in-vivo smear microscopy of the respiratory system provided in the above embodiments of the present application are implemented.

[0186] Among them, the computer-readable storage medium may be an internal storage unit of the control module described in the foregoing embodiments, such as the hard disk or memory of the control module. The computer-readable storage medium may also be an external storage device of the control module, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. equipped on the control module.

[0187] The above is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily think of various equivalent modifications or substitutions, and these modifications or substitutions should all be covered within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A test system for microscopic examination of living smears of the respiratory system, characterized in that, Comprising: A microscopic image acquisition module for detecting a digital microscopic image of a living smear sample; A blood detection module for detecting the eosinophil count corresponding to the living smear sample; A control module, which performs multi-scale feature extraction on the digital microscopic image according to a preset convolutional neural network, and identifies a candidate region corresponding to Charcot-Leyden crystals in the digital microscopic image; The control module obtains the morphological parameters and optical parameters corresponding to the candidate region, and calculates a similarity matrix between the morphological parameters and optical parameters corresponding to the living smear sample and a standard Charcot-Leyden crystal database; the morphological parameters at least include area, perimeter, and roundness, and the optical parameters at least include brightness gradient and polarization characteristics; The control module establishes a Charcot-Leyden crystal density distribution heat map corresponding to the living smear sample according to the similarity matrix, and obtains a crystal quantification count value corresponding to the Charcot-Leyden crystal density distribution heat map; The control module performs multi-dimensional fitting on the crystal quantification count value corresponding to the living smear sample and the eosinophil count to generate joint test information corresponding to the living smear sample.

2. The system according to claim 1, wherein The performing multi-scale feature extraction on the digital microscopic image according to a preset convolutional neural network, and identifying a candidate region corresponding to Charcot-Leyden crystals in the digital microscopic image, includes: Constructing a multi-scale feature extraction layer; Inputting the digital microscopic image into the multi-scale feature extraction layer for feature fusion to generate a feature tensor; the feature tensor has a spatial pyramid structure; Deploying a sliding window mechanism on the feature tensor to generate a candidate region bounding box based on a region proposal network, and identifying the candidate region in the digital microscopic image according to the candidate region bounding box.

3. The system according to claim 2, wherein The first scale layer of the multi-scale feature extraction layer is provided with a downsampling structure with a 3×3 convolution kernel and a stride of 2, and the second scale layer is provided with a parallel structure of 1×1 and 5×5 convolution kernels; The generation algorithm corresponding to the candidate region bounding box satisfies geometric constraint conditions, and the expression of the geometric constraint conditions includes: |(w / h)-1.7|<0.3; w and h respectively represent the width and height of the candidate region bounding box.

4. The system according to claim 1, characterized in that The calculating a similarity matrix between the morphological parameters and optical parameters corresponding to the living smear sample and a standard Charcot-Leyden crystal database, includes: Performing a dynamic time warping algorithm on the morphological parameter sequence to construct a morphological similarity component; Calculating an optical similarity component for the optical parameters according to a spectral angle matching algorithm; Constructing the similarity matrix according to the morphological similarity component and the optical similarity component.

5. The system according to claim 1, characterized in that, The establishing a Charcot-Leyden crystal density distribution heat map corresponding to the living smear sample according to the similarity matrix, includes: Based on a kernel density estimation algorithm, generating a target bandwidth according to the sample position standard deviation and the number of candidate regions corresponding to the candidate region; Calculating a probability density function in a two-dimensional spatial domain according to the target bandwidth; Superimpose the element values corresponding to the similarity matrix as weight coefficients into the probability density function to generate a heat map of Charcot-Leyden crystal density distribution with color gradient mapping.

6. The system according to claim 5, wherein The intensity of the red channel of the heat map of Charcot-Leyden crystal density distribution has an S-shaped function relationship with the probability density function.

7. The system according to claim 1, wherein Performing multi-dimensional fitting on the crystal quantization count value corresponding to the live smear sample and the eosinophil count to generate the joint test information corresponding to the live smear sample, including: Performing principal component analysis on the crystal quantization count value and the eosinophil count respectively according to a preset goodness-of-fit threshold to obtain a first principal component and a second principal component; the expression corresponding to the goodness-of-fit threshold is Q^2>0.85, where Q is the goodness-of-fit threshold; Performing multi-dimensional fitting according to the first principal component and the second principal component to generate the joint test information.

8. The system according to claim 1, wherein After generating the joint test information corresponding to the live smear sample, it further includes: Constructing a clinical decision tree model, setting a first decision node corresponding to the clinical decision tree model, and the first decision node is used to activate the fungal infection branch corresponding to the clinical decision tree model when the crystal quantization count value is greater than a first threshold and the eosinophil count is greater than a second threshold; Associating a preset treatment strategy database at the end node of the clinical decision tree model to generate diagnostic plan information corresponding to the joint test information.

9. A test method for microscopic examination of living body smears of the respiratory system, characterized in that, Applied to the control module of the system according to any one of claims 1-8, the method includes: Obtaining a digital microscopic image of a live smear sample detected by a microscopic image acquisition module; Obtaining the eosinophil count corresponding to the live smear sample detected by a blood detection module; Performing multi-scale feature extraction on the digital microscopic image according to a preset convolutional neural network, and identifying a candidate region corresponding to Charcot-Leyden crystals in the digital microscopic image; Obtaining the morphological parameters and optical parameters corresponding to the candidate region, and calculating a similarity matrix of the morphological parameters and optical parameters corresponding to the live smear sample and a standard Charcot-Leyden crystal database; the morphological parameters at least include area, perimeter and roundness, and the optical parameters at least include brightness gradient and polarization characteristics; Establishing a heat map of Charcot-Leyden crystal density distribution corresponding to the live smear sample according to the similarity matrix, and obtaining the crystal quantization count value corresponding to the heat map of Charcot-Leyden crystal density distribution; Performing multi-dimensional fitting on the crystal quantization count value corresponding to the live smear sample and the eosinophil count to generate the joint test information corresponding to the live smear sample.

10. A test device for microscopic examination of living body smears of the respiratory system, characterized in that, Including: An image acquisition unit for obtaining a digital microscopic image of a live smear sample detected by a microscopic image acquisition module; A first acquisition unit for obtaining the eosinophil count corresponding to the live smear sample detected by a blood detection module; A region recognition unit for performing multi-scale feature extraction on the digital microscopic image according to a preset convolutional neural network, and identifying a candidate region corresponding to Charcot-Leyden crystals in the digital microscopic image; A parameter acquisition unit, configured to acquire the morphological parameters and optical parameters corresponding to the candidate region, and calculate a similarity matrix between the morphological parameters and optical parameters corresponding to the live smear sample and a standard Charcot-Leyden crystal database; the morphological parameters at least include area, perimeter, and roundness, and the optical parameters at least include brightness gradient and polarization characteristics; A second acquisition unit, configured to establish a Charcot-Leyden crystal density distribution heat map corresponding to the live smear sample according to the similarity matrix, and acquire a crystal quantification count value corresponding to the Charcot-Leyden crystal density distribution heat map; An inspection generation unit, configured to perform multi-dimensional fitting on the crystal quantification count value corresponding to the live smear sample and the eosinophil count, and generate joint inspection information corresponding to the live smear sample.

Images