A nasal secretion smear diagnosis method and system based on deep learning

Abstract

The present invention discloses a nasal secretion smear diagnosis method and system based on deep learning, including: reading the nasal secretion smear to be diagnosed in blocks in a sliding window manner; performing a preprocessing operation on the read window image and inputting it into a trained cell detection model, performing a series of feature extraction on the input image, enhancing the pyramid feature map by bottom-up and top-down feature fusion, and finally obtaining the coordinates and score of the rectangular frame of the detected target according to the feature map of each scale, filtering out the rectangular frames below the threshold according to a set score threshold, and obtaining a rectangular frame set R; according to the coordinates of the rectangular frames in R, cutting the corresponding image from the original window image and performing a preprocessing operation, inputting it into a trained cell classification model, outputting the cell category of the corresponding image and performing post-processing, and obtaining the cell category of the original nasal secretion smear to be diagnosed. The present invention can diagnose nasal secretion smears efficiently and accurately.

Description

Technical Field

[0001] The present invention relates to the technical field of nasal secretion smear diagnosis, and in particular to a nasal secretion smear diagnosis method and system based on deep learning. Background Art

[0002] The traditional method of nasal secretion smear diagnosis mainly relies on doctors to examine the types and numbers of various types of cells in nasal secretions under a microscope. Due to the huge number of cells, as many as tens of thousands of cells, doctors are unable to accurately count the number of cells of each type and can only rely on experience and intuition. This has technical problems such as strong subjectivity and reliance on doctor's experience. In addition, since cells are scattered on the smear and the single field of view of the microscope is limited, doctors need to keep moving the slide to view all the cells. This method is also prone to problems such as missing cells or repeated counting, and the diagnostic efficiency is low. Summary of the invention

[0003] The present invention provides a nasal secretion smear diagnosis method and system based on deep learning to solve the problems existing in the above-mentioned prior art. The technical solution is as follows:

[0004] On the one hand, a deep learning-based nasal secretion smear diagnosis method is provided, comprising:

[0005] S1, reading the nasal secretion smear to be diagnosed in blocks in a sliding window manner, and reading the image of the window range each time is called the window image;

[0006] S2, performing preprocessing operation on the read window image;

[0007] S3, input the preprocessed window image into the trained cell detection model, the cell detection model performs a series of feature extraction on the input image, enhances the pyramid feature map by bottom-up and top-down feature fusion, and finally obtains the coordinates and score of the rectangular box of the detected target according to the feature map of each scale, and filters out the rectangular boxes below the threshold according to the set score threshold to obtain the rectangular box set R;

[0008] S4, according to the coordinates of the rectangular box in R, cutting the corresponding image from the original window image and performing a preprocessing operation, inputting the preprocessed corresponding image into the trained cell classification model, and the cell classification model outputs the cell category of the corresponding image;

[0009] S5. Post-process the cells of the corresponding image to obtain the cell categories of the original nasal secretion smear to be diagnosed.

[0010] On the other hand, a deep learning-based nasal secretion smear diagnosis system is provided, the system comprising:

[0011] A reading module is used to read the nasal secretion smear to be diagnosed in blocks in a sliding window manner, and each time the image of the window range is read, it is called a window image;

[0012] A preprocessing module, used for performing preprocessing operations on the read window image;

[0013] The cell detection module is used to input the preprocessed window image into the trained cell detection model. The cell detection model performs a series of feature extraction on the input image, and then enhances the pyramid feature map by bottom-up and top-down feature fusion. Finally, the coordinates and scores of the rectangular boxes of the detected targets are obtained according to the feature maps of each scale. According to the set score threshold, the rectangular boxes below the threshold are filtered out to obtain the rectangular box set R.

[0014] A cell classification module, used for cutting a corresponding image from the original window image and performing a preprocessing operation according to the coordinates of the rectangular frame in R, and inputting the preprocessed corresponding image into a trained cell classification model, and the cell classification model outputs the cell category of the corresponding image;

[0015] The post-processing module is used to perform post-processing on the cells of the corresponding image to obtain the cell categories of the original nasal secretion smear to be diagnosed.

[0016] On the other hand, an electronic device is provided, comprising a processor and a memory, wherein the memory stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the above-mentioned deep learning-based nasal secretion smear diagnosis method.

[0017] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction is stored in the storage medium, and the at least one instruction is loaded and executed by a processor to implement the above-mentioned deep learning-based nasal secretion smear diagnosis method.

[0018] The beneficial effects brought about by the technical solution provided by the present invention include at least:

[0019] The present invention reads the nasal secretion smear to be diagnosed in blocks in a sliding window manner, divides an image into multiple blocks and inputs them into a deep learning model, thereby improving the efficiency of diagnosis. In addition, through the deep learning model of the present invention (in order to overcome the problems of large computational complexity and poor real-time performance of the deep learning model, the present invention chooses to implement an efficient improved RTMDet as the detection network and an improved MobileNetV3 as the classification network, instead of using the same network for detection and classification as in the prior art), not only the doctor's diagnostic efficiency is greatly improved, but the accuracy is also higher than that of traditional methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 This is a flow chart of a nasal secretion smear diagnosis method based on deep learning provided by an embodiment of the present invention;

[0022] Figure 2 is a schematic diagram of an improved RTMDet network structure provided by an embodiment of the present invention;

[0023] Figure 3 is a schematic diagram of the backbone network structure of the improved RTMDet provided in an embodiment of the present invention;

[0024] Figure 4 is a schematic diagram of the neck network structure of the improved RTMDet provided in an embodiment of the present invention;

[0025] Figure 5 It is a schematic diagram of the structure of SPPELAN-E provided in an embodiment of the present invention;

[0026] Figure 6 It is a schematic diagram of the existing SPP structure;

[0027] Figure 7 It is a schematic diagram of the structure of the ECA attention mechanism provided by an embodiment of the present invention;

[0028] Figure 8 Schematic diagram of the improved MobileNetV3 network structure provided by an embodiment of the present invention;

[0029] Fig. 9 It is a schematic diagram of the MobileNetV3 bneck-E structure provided by an embodiment of the present invention;

[0030] Fig.10 This is a schematic diagram of the existing MobileNetV3 bneck structure;

[0031] Fig.11 is a schematic diagram of window cell deduplication provided by an embodiment of the present invention;

[0032] Fig.12 It is a block diagram of a nasal secretion smear diagnosis system based on deep learning provided by an embodiment of the present invention;

[0033] Fig.13 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0034] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0035] The embodiment of the present invention provides a nasal secretion smear diagnosis method based on deep learning, which can be implemented by an electronic device, which can be a terminal or a server. Figure 1 A flowchart of a nasal secretion smear diagnosis method based on deep learning is shown, and the processing flow of the method may include the following steps:

[0036] S1, reading the nasal secretion smear to be diagnosed in blocks in a sliding window manner, and reading the image of the window range each time is called the window image;

[0037] The image of the nasal secretion scan is huge, generally with a size of 200000x200000 pixels (the effective physical area of ​​the slide is about 50mmX24mm, the secretion smear generally occupies half, that is, 24mmX24mm, the mpp value of the shooting light path is about 0.1um / pix=1e-4mm / pix, 24mm / (1e-4mm / pix) = 240000pix), and it cannot be read completely into the memory, so it needs to be read in blocks. The embodiment of the present invention reads data in blocks in a windowing manner, and the window size is 2048 2048, the overlap between windows is 128, and the image in the window range is read each time, which is called the window image.

[0038] S2, performing preprocessing operation on the read window image;

[0039] Optionally, the preprocessing includes:

[0040] Scaling the window image to the size required by the cell detection model for the input image (the size required by the improved RTMDet in the embodiment of the present invention for the input image is 640x640, so the window image is scaled to this size);

[0041] A predefined mean vector is subtracted from each pixel value in the image, and then divided by a predefined standard deviation vector (the embodiment of the present invention uses the mean and standard deviation calculated on the large public dataset imagenet) to make the pixel values ​​in the image evenly distributed, which is beneficial to the subsequent model detection and classification.

[0042] Assume that the window image is a bgr image I of uint8 type with a size of 2048x2048x3. Scale the image I first, and then standardize the scaled image I to obtain image C = (I- (123.675, 116.28, 103.53)) / (58.395, 57.12, 57.375), where (123.675, 116.28, 103.53) is the predefined mean vector, and (58.395, 57.12, 57.375) is the predefined standard deviation vector.

[0043] S3, input the preprocessed window image into the trained cell detection model (the nasal secretion smear diagnosis method based on deep learning in the embodiment of the present invention is a technology that uses a deep learning algorithm to analyze and process the nasal secretion smear scan image. By using a large amount of nasal secretion scan image data, a large amount of nasal secretion cell annotation data, and the powerful pattern recognition ability of the deep learning algorithm, the cell detection model and the cell classification model can be trained to automatically detect and identify neutrophils and eosinophils in the nasal secretions, and assist doctors in disease diagnosis). After the cell detection model performs a series of feature extraction on the input image, the pyramid feature map is enhanced by bottom-up and top-down feature fusion. Finally, the coordinates and scores of the rectangular boxes of the detected targets are obtained according to the feature maps of each scale. According to the set score threshold (such as 0.3), the rectangular boxes below the threshold are filtered out to obtain the rectangular box set R;

[0044] Alternatively, if Figure 2 As shown, the cell detection model is an improved RTMDet network structure, and the improved RTMDet includes: input, backbone network, neck network and head network, loss function and output;

[0045] The input image enters the backbone network, such as Figure 3 As shown in the figure, the image first passes through four Conv Module structures to reduce the size, reducing the dimension while retaining important features, and then passes through three structures composed of CSPLayer and ConvModule to continue to extract important features of the context, and generate two output feature maps of different sizes, recorded as output 2 and output 3, for subsequent feature fusion, and then passes through the improved spatial pyramid pooling structure SPPELAN-E to generate multi-scale features for detecting targets of different sizes, and finally passes through a CSPLayer structure to further extract features, enhance the performance of the model, and generate another output feature map, recorded as output 1;

[0046] The three outputs 1, 2, and 3 enter the neck network as the three inputs 1, 2, and 3, respectively. Figure 4As shown in the figure, first, input 1 passes through a Conv Module structure and an upsampling operation to adjust the size and number of channels of the feature map, and then it is spliced ​​and fused with input 2 to enrich the multi-scale information, and a CSPLayer structure is used to enhance the feature expression ability. Since the nasal secretion target is small, dense and adherent, a lot of noise will be generated during feature fusion, which will affect the quality of the final fused feature map. Therefore, after each splicing and fusion and a CSPLayer structure operation, an ECA attention mechanism structure is added to suppress the fused noise and enhance the attention to nasal secretions. Then, a Conv Module structure and an upsampling operation are performed to adjust the size and number of channels of the feature map, and then it is spliced ​​and fused with input 3 to further enrich the multi-scale information, and a CSPLayer structure is used to enhance the feature expression ability; then, an ECA attention mechanism structure is passed, and two branches are generated at this time, one of which is used as the output 3 of the neck network, and the other branch passes through a Conv After the Module structure performs a downsampling operation to adjust the size, it is further fused with the previously generated feature map to enrich more target information; then it passes through a CSPLayer structure to enhance the feature expression ability, and then passes through the ConvModule structure to perform a downsampling operation to adjust the size, and then it is further fused with the previously generated feature map to enrich more target information. At this time, two branches are also generated, one of which is used as the output 2 of the neck network, and the other branch passes through a CSPLayer structure to enhance the feature expression ability, and then passes through the ECA attention mechanism structure, and finally outputs 1;

[0047] In the head network part, the three outputs 1, 2, and 3 of the neck network are respectively decoded through some convolution modules to finally generate the coordinates and scores of the rectangular box of the target (detected cells).

[0048] Optionally, the loss function includes classification loss and bounding box regression loss;

[0049] The classification loss is formulated as follows:

[0050]

[0051]

[0052] in Represents the predicted probability of the true category of the sample, and is an adjustable weight parameter, Represents the actual label of the sample, the scaling factor of Folcal Loss It can reduce the weight of the easy classes on the loss during training and quickly focus the model on the hard classes. Used to adjust the ratio between positive and negative sample losses;

[0053] In order to solve the inconsistency problem between the training and testing stages, Quality Folcal Loss combines the positioning quality IOU value and the classification score on the basis of FolcalLoss. Since its label is a continuous value between 0 and 1, the two parts of Folcal Loss are improved. The formula is as follows:

[0054]

[0055] in represents the actual label of the sample, Represents the label value obtained after combining the IOU value;

[0056] The bounding box regression loss uses GIoU Loss, which is used to calculate the relationship between the overlapping areas of two boxes. The larger the overlapping area, the smaller the loss, and vice versa. In addition, GIoU Loss is between [0,2], and its value is limited to a smaller range, so the network will not fluctuate violently, and thus has better stability. The formula is as follows:

[0057]

[0058]

[0059]

[0060] Where A and B represent two bounding boxes, C is the smallest bounding box that can enclose them, and IOU is the intersection-union ratio of A and B.

[0061] Optionally, the improved spatial pyramid pooling structure SPPELAN-E is used to enable the network to process input images of different sizes without forcibly cropping or scaling the input images, thereby reducing information loss and improving model performance;

[0062] like Figure 5 As shown, the SPPELAN-E retains one of the four branches of the existing SPP structure (such as Figure 6As shown in the figure, the existing SPP structure performs four different operations on the four input branches, retains one of the branches without any change, and generates feature maps of different sizes through maximum pooling operations on the remaining three branches. Finally, the output results of the four different branches are spliced ​​into the final output), a hollow convolution is used to expand the receptive field, and a residual structure is added to the output so that the input of the remaining branches comes from the output of the previous branch, which not only expands the receptive field but also enhances the feature expression ability of the model. Finally, the output results of the four different branches are spliced ​​into the final output.

[0063] Compared with the existing SPP structure, the SPPELAN-E of the embodiment of the present invention retains more semantic information and has better performance in detecting small targets such as nasal secretions. This is because the SPP structure uses the maximum pooling operation multiple times, and this operation only takes the maximum value of each element and ignores other elements, thus resulting in the loss of useful information in some feature maps. However, the dilated convolution expands the receptive field by introducing a dilation factor, and does not lose useful information. In addition, the continuous residual structure can also retain certain effective information.

[0064] Alternatively, if Figure 7 As shown in the figure (X represents the input feature map, W represents the width of the feature map, H represents the height of the feature map, C represents the number of channels of the feature map, and K represents the size of the convolution kernel used to calculate the attention weight of each channel. Represents the sigmoid activation function), the ECA attention mechanism structure first compresses the spatial dimension of the input feature map to 1 while keeping the number of channel dimensions unchanged;

[0065] Secondly, a one-dimensional convolution of size K is used to capture local cross-channel interaction information. This local cross-channel interaction strategy without dimensionality reduction helps to more effectively interact between channels while maintaining the correlation between channels, thereby improving the network's expressiveness and performance.

[0066] Then the output is passed through the sigmoid activation function to ensure that the output is between 0 and 1, and the channel attention weight is generated;

[0067] Finally, the channel attention weight is multiplied by the input original feature map to obtain the final feature map.

[0068] Since ECA can capture the relationship between different channels, thereby improving the ability of feature representation, by utilizing this mechanism, the embodiment of the present invention can effectively enhance the representation ability of the network without increasing too many parameters and computational costs, thereby increasing the accuracy of nasal secretion diagnosis.

[0069] S4, according to the coordinates of the rectangular box in R, cutting the corresponding image from the original window image and performing a preprocessing operation, inputting the preprocessed corresponding image into the trained cell classification model, and the cell classification model outputs the cell category of the corresponding image;

[0070] The cropped image in the embodiment of the present invention does not have a rectangular frame (that is, the image is cropped from the original image according to the coordinates of the rectangular frame), and one image will only have one cell (the number of detected cells is the number of images, and each image has one cell). Then the cell classification model classifies each cell. Specifically, a rectangular frame r = (x, y, w, h) is taken from the set R in order, and the rectangular frame r is cropped from the original window image I to obtain image A. Image A is scaled to a size of 224x224 (the size required by the improved MobileNetV3 for the input image of the cell classification model) to obtain image B. Image B is standardized to finally obtain image D = (B- (123.675, 116.28, 103.53)) / (58.395, 57.12,57.375). Image D is input into the improved MobileNetV3 of the cell classification model to obtain the scores of the three categories of eosinophils, neutrophils, and other cells. The category corresponding to the highest score is selected from the scores output by the model as the category of the cell.

[0071] Alternatively, if Figure 8 As shown, the cell classification model is an improved MobileNetV3 network structure, including input, backbone network, head network, loss function and output;

[0072] The input image enters the backbone network for feature extraction, such as Fig. 9 As shown, the backbone network of the improved MobileNetV3 includes multiple bneck-Es. For multiple bneck structures of the backbone network of the existing MobileNetV3 (such as Fig.10 As shown in the figure, the existing MobileNetV3 backbone network has multiple bneck structures. When the step size is 1, the feature map first passes through a 1 1 convolution to expand the number of channels, followed by a depthwise separable convolution to reduce model parameters while extracting features, and finally 1 1 convolution restores the original number of channels; when the step size is 2, the feature map will pass through two branches, of which the main branch is similar to the step size of 1. The difference is that an SE attention mechanism is added after the depthwise separable convolution to enhance the expression of important features, and the result is added to another branch to avoid gradient explosion and increase the performance of the model. However, for nasal secretion images, due to the differences in staining and scraping techniques, the existing network cannot extract key features well). When the step size is 1, no modification is made. In the bneck structure with a step size of 2, the SE attention mechanism (SE attention mechanism is similar to the ECA attention mechanism, but a fully connected layer is used to obtain channel information, which increases the computational efficiency) is modified to the ECA attention mechanism to speed up the computational efficiency and reduce the number of parameters. In addition, a branch is added after the depthwise separable convolution to enhance the performance of the network.

[0073] After feature extraction, the head network, including some convolutional layers and fully connected layers, calculates the classification loss value of the extracted feature map, and finally outputs the cell classification results (the output cell categories include eosinophils, neutrophils, and other cells).

[0074] Optionally, the loss function uses cross entropy loss, and the formula is as follows:

[0075]

[0076] in Represents the predicted probability of the sample in this category, Represents the sample label.

[0077] S5. Post-process the cells of the corresponding image to obtain the cell categories of the original nasal secretion smear to be diagnosed.

[0078] The post-processing comprises:

[0079] Restore the coordinates of the cell detected in the window image with the coordinates of the upper left corner of the window as the origin to the original image. Assuming that the coordinates of the cell are r = (x, y, w, h), and the coordinates of the upper left corner of the window are (l, t), then the coordinates of the cell on the original image are (x+l, y+t, w, h);

[0080] The cells in the overlapping area of ​​the window are removed. For each cell, the IOU value between it and all cells detected in the adjacent window is calculated. If the IOU value is greater than 0.5, it is considered to be the same cell.

[0081] Fig.11This is a schematic diagram of window cell deduplication. As shown in the figure, the central area is the cells of the current window, and the triangular area is the overlapping cells with the eight adjacent windows. Therefore, in order to remove duplicate cells, it is necessary to calculate the IOU value with each cell in the surrounding eight windows to achieve the purpose of deduplication.

[0082] like Fig.12 As shown, an embodiment of the present invention further provides a nasal secretion smear diagnosis system based on deep learning, the system comprising:

[0083] The reading module 1210 is used to read the nasal secretion smear to be diagnosed in blocks in a sliding window manner, and the image of the window range is read each time, which is called the window image;

[0084] A preprocessing module 1220 is used to perform a preprocessing operation on the read window image;

[0085] The cell detection module 1230 is used to input the preprocessed window image into the trained cell detection model. After the cell detection model performs a series of feature extraction on the input image, the pyramid feature map is enhanced by bottom-up and top-down feature fusion. Finally, the coordinates and scores of the rectangular boxes of the detected targets are obtained according to the feature maps of each scale. According to the set score threshold, the rectangular boxes below the threshold are filtered out to obtain the rectangular box set R.

[0086] A cell classification module 1240 is used to cut a corresponding image from the original window image and perform a preprocessing operation according to the coordinates of the rectangular box in R, and input the preprocessed corresponding image into a trained cell classification model, and the cell classification model outputs the cell category of the corresponding image;

[0087] The post-processing module 1250 is used to perform post-processing on the cells of the corresponding image to obtain the cell categories of the original nasal secretion smear to be diagnosed.

[0088] An embodiment of the present invention provides a nasal secretion smear diagnosis system based on deep learning, and its functional structure corresponds to a nasal secretion smear diagnosis method based on deep learning provided by an embodiment of the present invention, which will not be repeated here.

[0089] Fig.13It is a structural diagram of an electronic device 1300 provided in an embodiment of the present invention. The electronic device 1300 may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 1301 and one or more memories 1302, wherein the memory 1302 stores at least one instruction, and the at least one instruction is loaded and executed by the processor 1301 to implement the steps of the above-mentioned deep learning-based nasal secretion smear diagnosis method.

[0090] In an exemplary embodiment, a computer-readable storage medium is also provided, such as a memory including instructions, which can be executed by a processor in a terminal to complete the above-mentioned deep learning-based nasal secretion smear diagnosis method. For example, the computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0091] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0092] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A nasal secretion smear diagnosis method based on deep learning, characterized in that: The method comprises: S1, reading the nasal secretion smear to be diagnosed in blocks in a sliding window manner, and reading the image of the window range each time is called the window image; S2, performing preprocessing operation on the read window image; S3, input the preprocessed window image into the trained cell detection model, the cell detection model performs a series of feature extraction on the input image, enhances the pyramid feature map by bottom-up and top-down feature fusion, and finally obtains the coordinates and score of the rectangular box of the detected target according to the feature map of each scale, and filters out the rectangular boxes below the threshold according to the set score threshold to obtain the rectangular box set R; S4, according to the coordinates of the rectangular box in R, cutting the corresponding image from the original window image and performing a preprocessing operation, inputting the preprocessed corresponding image into the trained cell classification model, and the cell classification model outputs the cell category of the corresponding image; S5, post-processing the cells of the corresponding image to obtain the cell types of the original nasal secretion smear to be diagnosed; The cell detection model is an improved RTMDet network structure, and the improved RTMDet includes: input, backbone network, neck network and head network, loss function and output; The input image enters the backbone network, and the image first passes through four Conv Module structures to reduce the size, reducing the dimension while retaining important features, and then passes through three structures composed of CSPLayer and Conv Module to continue to extract important features of the context, and generate two output feature maps of different sizes, recorded as output 2 and output 3, for subsequent feature fusion, and then passes through the improved spatial pyramid pooling structure SPPELAN-E to generate multi-scale features for detecting targets of different sizes, and finally passes through a CSPLayer structure to further extract features, enhance the performance of the model, and generate another output feature map, recorded as output 1; The three outputs 1, 2, and 3 enter the neck network as the three inputs 1, 2, and 3 respectively. First, input 1 passes through a Conv Module structure and an upsampling operation to adjust the size and number of channels of the feature map, and then is spliced ​​and fused with input 2 to enrich multi-scale information, and a CSPLayer structure is used to enhance the feature expression ability. Since the nasal secretion target is small, dense and adherent, a lot of noise will be generated during feature fusion, which will affect the quality of the final fused feature map. Therefore, after each splicing and fusion and a CSPLayer structure operation, an ECA attention mechanism structure is added to suppress the fused noise and enhance the attention to nasal secretions. Then, a Conv Module structure and an upsampling operation are performed to adjust the size and number of channels of the feature map, and then it is spliced ​​and fused with input 3 to further enrich the multi-scale information, and a CSPLayer structure is used to enhance the feature expression ability; then, an ECA attention mechanism structure is passed through, and two branches are generated at this time, one of which is used as the output 3 of the neck network, and the other branch passes through a Conv After the Module structure performs a downsampling operation to adjust the size, it is further fused with the previously generated feature map to enrich more target information; then it passes through a CSPLayer structure to enhance the expressiveness of the features, and then it passes through a Conv Module structure to perform a downsampling operation to adjust the size, and then it is further fused with the previously generated feature map to enrich more target information. At this time, two branches are also generated, one of which is used as the output 2 of the neck network, and the other branch passes through a CSPLayer structure to enhance the expressiveness of the features, and then passes through the ECA attention mechanism structure, and finally outputs 1; In the head network part, the three outputs 1, 2, and 3 of the neck network are respectively decoded through some convolution modules to finally generate the coordinates and score of the target rectangle. The cell classification model is an improved MobileNetV3 network structure, including input, backbone network, head network, loss function and output; The input image enters the backbone network for feature extraction. The backbone network of the improved MobileNetV3 includes multiple bneck-Es. For the multiple bneck structures of the existing MobileNetV3 backbone network, no modification is made when the step size is 1. In the bneck structure with a step size of 2, the SE attention mechanism is modified to the ECA attention mechanism to speed up the calculation efficiency and reduce the number of parameters. In addition, a branch is added after the depthwise separable convolution to enhance the performance of the network. After feature extraction, the head network, including some convolutional layers and fully connected layers, calculates the classification loss value of the extracted feature map, and finally outputs the cell classification result.

2. The method according to claim 1, characterized in that The preprocessing comprises: Scaling the window image to the size required by the cell detection model for the input image; The scaled image is normalized by subtracting a predefined mean vector from each pixel value in the image and dividing it by a predefined standard deviation vector to make the pixel values ​​in the image evenly distributed.

3. The method according to claim 1, characterized in that The loss function of the cell detection model includes classification loss and bounding box regression loss; The classification loss is formulated as follows: loss cls =Quality Focal Loss where p t Represents the predicted probability of the true category of the sample, α t and γ are adjustable weight parameters, y represents the actual label of the sample, and the scaling factor of Folcal Loss (1-p t ) γ During training, it can reduce the weight of the simple class on the loss and quickly focus the model on the difficult class. t Used to adjust the ratio between positive and negative sample losses; Quality Folcal Loss combines the positioning quality IOU value and the classification score on the basis of Folcal Loss. Since its label is a continuous value between 0 and 1, the two parts of Folcal Loss are improved. The formula is as follows: Quality Focal Loss(σ)=-|y-σ| β ((1-y)log(1-σ)+ylog(σ)) Where y represents the actual label of the sample, and σ represents the label value obtained after combining the IOU value; The bounding box regression loss uses GIoU Loss, which is used to calculate the relationship between the overlapping areas of two boxes. The larger the overlapping area, the smaller the loss, and vice versa. In addition, GIoU Loss is between [0,2], and its value is limited to a small range, so the network will not fluctuate violently, and thus has good stability. The formula is as follows: GIOU Loss=1-GIOU Where A and B represent two bounding boxes, C is the smallest bounding box that can enclose them, and IOU is the intersection-union ratio of A and B.

4. The method according to claim 1, characterized in that: The improved spatial pyramid pooling structure SPPELAN-E is used to enable the network to process input images of different sizes without forcibly cropping or scaling the input images, thereby reducing information loss and improving the performance of the model; The SPPELAN-E retains one of the four branches of the existing SPP structure, uses a dilated convolution to expand the receptive field, and adds a residual structure to the output so that the inputs of the remaining branches come from the output of the previous branch. This not only expands the receptive field but also enhances the feature expression ability of the model. Finally, the output results of the four different branches are spliced ​​into the final output.

5. The method according to claim 1, characterized in that: The ECA attention mechanism structure first compresses the spatial dimension of the input feature map to 1 while keeping the number of channel dimensions unchanged; Secondly, a one-dimensional convolution of size K is used to capture local cross-channel interaction information. This local cross-channel interaction strategy without dimensionality reduction helps to more effectively interact between channels while maintaining the correlation between channels, thereby improving the network's expressiveness and performance. Then the output is passed through the sigmoid activation function to ensure that the output is between 0 and 1, and the channel attention weight is generated; Finally, the channel attention weight is multiplied by the input original feature map to obtain the final feature map.

6. The method according to claim 1, characterized in that The loss function of the cell classification model adopts cross entropy loss, and the formula is as follows: Where p represents the predicted probability of the sample in this category, and y represents the sample label.

7. The method according to claim 1, characterized in that The post-processing comprises: Restore the coordinates of the cell detected in the window image with the coordinates of the upper left corner of the window as the origin to the original image. Assuming that the coordinates of the cell are r = (x, y, w, h), and the coordinates of the upper left corner of the window are (l, t), then the coordinates of the cell on the original image are (x+l, y+t, w, h); The cells in the overlapping area of ​​the window are removed. For each cell, the IOU value between it and all cells detected in the adjacent window is calculated. If the IOU value is greater than 0.5, it is considered to be the same cell.

8. A nasal secretion smear diagnosis system based on deep learning, characterized in that: The system comprises: A reading module is used to read the nasal secretion smear to be diagnosed in blocks in a sliding window manner, and each time the image of the window range is read, it is called a window image; A preprocessing module, used for performing preprocessing operations on the read window image; The cell detection module is used to input the preprocessed window image into the trained cell detection model. The cell detection model performs a series of feature extraction on the input image, and then enhances the pyramid feature map by bottom-up and top-down feature fusion. Finally, the coordinates and scores of the rectangular boxes of the detected targets are obtained according to the feature maps of each scale. According to the set score threshold, the rectangular boxes below the threshold are filtered out to obtain the rectangular box set R. A cell classification module, used for cutting a corresponding image from the original window image and performing a preprocessing operation according to the coordinates of the rectangular frame in R, and inputting the preprocessed corresponding image into a trained cell classification model, and the cell classification model outputs the cell category of the corresponding image; A post-processing module, used for post-processing the cells of the corresponding image to obtain the cell categories of the original nasal secretion smear to be diagnosed; The cell detection model is an improved RTMDet network structure, and the improved RTMDet includes: input, backbone network, neck network and head network, loss function and output; The input image enters the backbone network, and the image first passes through four Conv Module structures to reduce the size, reducing the dimension while retaining important features, and then passes through three structures composed of CSPLayer and Conv Module to continue to extract important features of the context, and generate two output feature maps of different sizes, recorded as output 2 and output 3, for subsequent feature fusion, and then passes through the improved spatial pyramid pooling structure SPPELAN-E to generate multi-scale features for detecting targets of different sizes, and finally passes through a CSPLayer structure to further extract features, enhance the performance of the model, and generate another output feature map, recorded as output 1; The three outputs 1, 2, and 3 enter the neck network as the three inputs 1, 2, and 3 respectively. First, input 1 passes through a Conv Module structure and an upsampling operation to adjust the size and number of channels of the feature map, and then is spliced ​​and fused with input 2 to enrich multi-scale information, and a CSPLayer structure is used to enhance the feature expression ability. Since the nasal secretion target is small, dense and adherent, a lot of noise will be generated during feature fusion, which will affect the quality of the final fused feature map. Therefore, after each splicing and fusion and a CSPLayer structure operation, an ECA attention mechanism structure is added to suppress the fused noise and enhance the attention to nasal secretions. Then, a Conv Module structure and an upsampling operation are performed to adjust the size and number of channels of the feature map, and then it is spliced ​​and fused with input 3 to further enrich the multi-scale information, and a CSPLayer structure is used to enhance the feature expression ability; then, an ECA attention mechanism structure is passed through, and two branches are generated at this time, one of which is used as the output 3 of the neck network, and the other branch passes through a Conv After the Module structure performs a downsampling operation to adjust the size, it is further fused with the previously generated feature map to enrich more target information; then it passes through a CSPLayer structure to enhance the expressiveness of the features, and then it passes through a Conv Module structure to perform a downsampling operation to adjust the size, and then it is further fused with the previously generated feature map to enrich more target information. At this time, two branches are also generated, one of which is used as the output 2 of the neck network, and the other branch passes through a CSPLayer structure to enhance the expressiveness of the features, and then passes through the ECA attention mechanism structure, and finally outputs 1; In the head network part, the three outputs 1, 2, and 3 of the neck network are respectively decoded through some convolution modules to finally generate the coordinates and score of the target rectangle. The cell classification model is an improved MobileNetV3 network structure, including input, backbone network, head network, loss function and output; The input image enters the backbone network for feature extraction. The backbone network of the improved MobileNetV3 includes multiple bneck-Es. For the multiple bneck structures of the existing MobileNetV3 backbone network, no modification is made when the step size is 1. In the bneck structure with a step size of 2, the SE attention mechanism is modified to the ECA attention mechanism to speed up the calculation efficiency and reduce the number of parameters. In addition, a branch is added after the depthwise separable convolution to enhance the performance of the network. After feature extraction, the head network, including some convolutional layers and fully connected layers, calculates the classification loss value of the extracted feature map, and finally outputs the cell classification result.

Images