Sputum image evaluation method, system and device and storage medium

Abstract

The invention discloses a sputum image evaluation method, system and device and a storage medium. The method comprises the steps that a sputum image is collected; performing image preprocessing on the sputum image to obtain a preprocessed sputum image; inputting the preprocessed sputum image into a trained image processing model for feature extraction to obtain sputum image features; performing dimension reduction processing and linear transformation processing on the sputum image features to obtain a sputum image classification result; and converting the sputum image classification result into corresponding prediction probability distribution based on a preset algorithm, and determining an evaluation result corresponding to the sputum image based on the prediction probability distribution. The embodiment of the invention can quickly and accurately predict whether the respiratory tract MDRO occurs, and can be widely applied to the technical field of medical image processing.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and particularly to a sputum image evaluation method, system, device and storage medium. Background Art

[0002] Multidrug-resistant organisms (MDROs) in the lower respiratory tract refer to bacteria that are resistant to three or more types of antibacterial drugs used clinically. It does not include natural resistance, but only acquired resistance. MDROs are highly contagious and can be transmitted through contact, droplets and other routes, and are likely to cause large-scale infections in crowded places such as hospitals. Moreover, the treatment of patients infected with MDROs is difficult and the fatality rate is high. Therefore, the infection of MDROs in the lower respiratory tract has posed a serious threat to medical safety and quality and has become a serious problem in the field of clinical medicine.

[0003] However, there is no existing evaluation method that can quickly and accurately predict whether MDRO in the respiratory tract occurs. Summary of the Invention

[0004] In view of this, to solve one of the above problems, an object of the embodiments of the present invention is to provide a sputum image evaluation method, system, device and storage medium, which can quickly and accurately predict whether MDRO in the respiratory tract occurs.

[0005] On the one hand, an embodiment of the present invention provides a sputum image evaluation method, including the following steps:

[0006] Collect a sputum image; perform image preprocessing on the sputum image to obtain a preprocessed sputum image;

[0007] Input the preprocessed sputum image into a trained image processing model for feature extraction to obtain sputum image features;

[0008] Perform dimensionality reduction processing and linear transformation processing on the sputum image features to obtain a sputum image classification result;

[0009] Based on a preset algorithm, convert the sputum image classification result into a corresponding predicted probability distribution, and determine the evaluation result corresponding to the sputum image based on the predicted probability distribution.

[0010] Specifically, the performing image preprocessing on the sputum image includes:

[0011] Annotate the sputum image based on sputum sample information to obtain a sputum annotated image;

[0012] Perform data augmentation on the sputum annotated image to obtain a preprocessed sputum image; the data augmentation includes image flipping, horizontal mirroring and translation transformation.

[0013] Specifically, the training method of the image processing model is implemented by the following method:

[0014] Obtain a sputum image sample set and a feature extraction result sample set; the sputum image sample set includes a training sample set; the extraction result sample set includes a training result sample set;

[0015] Define a gradient calculation method based on a preset loss function and preset parameters of the image processing pre-model;

[0016] Randomly select a training sample and the corresponding training result sample based on the training sample set and the training result sample set, and input the training sample into the image processing pre-model for calculation to obtain a training result;

[0017] Calculate the training sample and the training result according to the gradient calculation method to obtain a gradient value;

[0018] Calculate the training result and the corresponding training result sample according to the preset loss function to obtain a training loss value;

[0019] Perform a first update on the preset parameters of the image processing pre-model based on the gradient value, and determine the updated image processing pre-model based on the first update result; re-obtain the gradient value and the training loss value until the updated image processing pre-model meets the preset conditions to obtain a trained image processing model.

[0020] Furthermore, the sputum image sample set further includes a validation sample set; the extraction result sample set further includes a validation result sample set; the determining the updated image processing pre-model based on the first update result further includes:

[0021] Determine the first updated image processing pre-model according to the update result;

[0022] Randomly select a validation sample and the corresponding validation result sample based on the validation sample set and the validation result sample set;

[0023] Input the validation sample into the first updated image processing pre-model for calculation to obtain a validation result;

[0024] Calculate the validation result and the corresponding validation result sample according to the preset loss function to obtain a validation loss value;

[0025] Perform a second update on the preset hyperparameters of the first updated image processing pre-model based on the validation loss value, and determine the updated image processing pre-model according to the second update result.

[0026] Specifically, the dimensionality reduction processing and linear transformation processing of the sputum image features to obtain a sputum image classification result include:

[0027] Perform dimensionality reduction on the sputum image features multiple times to form one-dimensional sputum image features;

[0028] Classify the element values on the one-dimensional sputum image features to obtain element values of several categories;

[0029] Multiply the element values corresponding to several categories on the one-dimensional sputum image features by weights and add a bias term to obtain logical values corresponding to several categories, and obtain a sputum image classification result according to the logical values corresponding to several categories.

[0030] Further, the conversion of the sputum image classification result into a corresponding predicted probability distribution based on a preset algorithm includes:

[0031] Perform an exponential operation based on the logical values corresponding to several categories to obtain exponential values corresponding to several categories;

[0032] Perform a quotient calculation based on the exponential values corresponding to several categories, and convert the exponential values corresponding to several categories into predicted probability distributions corresponding to several categories according to the quotient calculation result.

[0033] Further, the determination of the evaluation result corresponding to the sputum image based on the predicted probability distribution includes:

[0034] Calculate according to the actual sputum sample data to obtain the true probability distributions corresponding to several categories;

[0035] Calculate according to the predicted probability distributions corresponding to several categories and the true probability distributions corresponding to several categories, and determine the evaluation result corresponding to the sputum image according to the calculation result.

[0036] On the other hand, an embodiment of the present invention also provides a sputum image evaluation system, including:

[0037] The first module is used to collect sputum images; perform image preprocessing on the sputum images to obtain preprocessed sputum images;

[0038] The second module is used to input the preprocessed sputum images into a trained image processing model for feature extraction to obtain sputum image features;

[0039] The third module is used to perform dimensionality reduction and linear transformation processing on the sputum image features to obtain a sputum image classification result;

[0040] The fourth module is used to convert the classification result of the sputum image into a corresponding predicted probability distribution based on a preset algorithm, and determine the evaluation result corresponding to the sputum image based on the predicted probability distribution.

[0041] On the other hand, an embodiment of the present invention further provides a sputum image evaluation device, including:

[0042] At least one processor;

[0043] At least one memory for storing at least one program;

[0044] When the at least one program is executed by the at least one processor, the at least one processor implements the method as described above.

[0045] On the other hand, an embodiment of the present invention further provides a computer-readable storage medium, in which a program executable by a processor is stored, and the program executable by the processor is used to execute the method as described above when executed by the processor.

[0046] In summary, implementing the embodiments of the present invention at least includes the following beneficial effects:

[0047] This embodiment provides a sputum image evaluation method, system, device and storage medium. Through a series of processes such as image preprocessing, feature extraction, dimensionality reduction, and linear transformation on the collected image, the classification result of the sputum image is obtained, and finally the classification result of the sputum image is converted into a corresponding predicted probability distribution for evaluation; by using the trained image processing model, the feature information of the sputum image can be quickly and effectively extracted, improving the efficiency and accuracy of the image processing model. Further, converting the result of the image processing model into a corresponding predicted probability distribution can visually and visually evaluate the sputum image, and then quickly and accurately predict whether respiratory tract MDRO occurs. Description of the Drawings

[0048] Figure 1 It is a schematic flowchart of the steps of a sputum image evaluation method provided by an embodiment of the present invention;

[0049] Figure 2 It is a loss image curve and an accuracy graph curve drawn according to the model training process of an embodiment of the present invention;

[0050] Figure 3 It is an ROC curve drawn according to the model training process of an embodiment of the present invention;

[0051] Figure 4 It is a schematic flowchart of the steps of another sputum image evaluation method provided by an embodiment of the present invention;

[0052] Figure 5It is a structural block diagram of a sputum image evaluation system provided by an embodiment of the present invention;

[0053] Figure 6 It is a structural block diagram of a sputum image evaluation device provided by an embodiment of the present invention. Specific embodiments

[0054] The following further elaborates on the present invention in detail in conjunction with the accompanying drawings and specific embodiments. For the step numbers in the following embodiments, they are only set for the convenience of elaboration and explanation, and no limitation is imposed on the order between the steps. The execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0055] The explanations of several terms involved in this application are as follows:

[0056] Data augmentation: A method of generating new data samples by expanding and transforming existing data, aiming to increase the scale and diversity of the training dataset, thereby improving the generalization ability and performance of machine learning models.

[0057] ResNet (Residual Network): A deep convolutional neural network architecture that, by introducing the concept of residual learning, allows the input signal to be directly passed to subsequent layers through skip connections in residual blocks, making the deep network more efficient and stable during training, solving the problems of gradient disappearance and network degradation in deep network training, and significantly improving the accuracy of image recognition and other tasks.

[0058] 50-layer residual network architecture (ResNet-50): A deep convolutional neural network with 50 layers deep, which solves the degradation problem in deep networks by introducing residual blocks and skip connection mechanisms. This architecture includes an initial convolutional module, four residual learning stages, a global average pooling layer, and a fully connected layer, and can efficiently extract image features for tasks such as image classification.

[0059] Convolution: A mathematical operation, specifically referring to an operation in deep learning where a convolutional kernel (or filter) slides over the input data (such as an image) and performs local weighted summation. This process can extract local features of the input data, such as edges and textures, and is the basis for constructing convolutional neural networks (CNNs), widely used in fields such as image recognition, speech recognition, and natural language processing.

[0060] Residual learning stage: By constructing residual blocks, the learning objective of the network is transformed into learning the residual mapping between the input and the output, that is, the difference between the direct mapping of the input and the expected output. This mechanism helps alleviate the vanishing gradient problem in the training of deep networks by introducing skip connections, allowing the input signal to be directly passed to subsequent layers, making the training of deep networks more efficient and stable.

[0061] Global Average Pooling (GAP): A commonly used pooling operation in deep learning. It obtains a value representing the global information of the feature map by averaging all pixel values of the feature map for each channel. This operation is mainly used to replace the traditional fully connected layer to reduce model parameters and computational complexity, improve the generalization ability of the model, and can retain spatial and semantic information while enhancing the interpretability of the model.

[0062] Flatten layer: A common neural network layer in deep learning. Its main function is to convert (or "flatten") multi-dimensional input data (such as images) into a one-dimensional array so that subsequent layers (such as fully connected layers) can correctly process the data.

[0063] Fully Connected Layer (FC layer): A basic layer in a neural network where each neuron is connected to all neurons in the previous layer. It receives the input from the previous layer (usually the data processed by the Flatten layer), performs a linear transformation through a weight matrix and a bias vector, and then applies an activation function (such as ReLU, Sigmoid, etc.) for non-linear processing, and outputs to the next layer or as the final output.

[0064] Softmax function: A mathematical function commonly used in the output layer of multi-classification problems. It can map a K-dimensional vector containing arbitrary real numbers to a K-dimensional probability distribution with values in the range (0, 1), and the sum of all elements is 1. In a neural network, the Softmax function is often used to probabilize the output of the model, making the output interpretable as the probability of each class, thus facilitating subsequent decision-making or classification operations.

[0065] Cross-entropy: A metric for measuring the difference between two probability distributions, commonly used in machine learning and deep learning. Specifically, it is used to quantify the difference between the probability distribution predicted by the model and the probability distribution of the true labels. The smaller the value of the cross-entropy, the closer the two distributions are, that is, the more accurate the model's prediction.

[0066] Matlab: A commercial mathematical software that integrates many powerful functions such as numerical analysis, matrix calculation, scientific data visualization, and modeling and simulation of nonlinear dynamic systems in an easy-to-use window environment. It provides a comprehensive solution for scientific research, engineering design, and many scientific fields that require effective numerical calculations, and is mainly used for algorithm development, data visualization, data analysis, and numerical calculation.

[0067] Conv2D (two-dimensional convolutional layer): A basic operation in deep learning and computer vision. It performs a convolutional operation on the input data (such as an image) by sliding a two-dimensional convolutional kernel, thereby extracting the feature information in the input data. The Conv2D layer plays a key role in feature extraction in ResNet and can learn local features such as edges and textures in the image, which are crucial for subsequent tasks such as image recognition, classification, and object detection.

[0068] Batch Norm: Batch Normalization, a normalization method used in deep learning to improve the training process of neural networks. It applies normalization processing to the input of each layer (usually after a linear transformation and before an activation function), keeping the data distribution stable, with the mean close to 0 and the variance close to 1. This method can alleviate the problem of internal covariate shift, accelerate the training process of neural networks, improve the stability and performance of the model, and reduce the risk of overfitting.

[0069] ReLu: (Rectified Linear Unit) is an activation function widely used in deep learning. Its definition is very simple. For the input x, the output of the ReLU function is max(0, x), that is, when x is greater than 0, the output is x; when x is less than or equal to 0, the output is 0. The ReLU function has non-linear characteristics, is computationally simple and efficient, and can alleviate the problem of gradient disappearance and accelerate the training process of neural networks. In addition, the ReLU function also has a certain degree of sparsity, that is, when the input is less than 0, the neurons will not be activated, which helps to improve the generalization ability of the model.

[0070] Max pool (max pooling): A commonly used pooling operation in convolutional neural networks. It slides a fixed-size window on the input feature map and selects the maximum value in each window as the output to reduce the spatial dimension of the feature map, retain the main features, reduce the computational complexity, and enhance the translational invariance of the model.

[0071] ID block: (ID block layer), also known as Identity Block, is the basic building block in the ResNet network. Its main function is to alleviate the vanishing gradient problem in the training of deep neural networks through residual connections. In ResNet-50, the ID block allows the network to learn the residual between the input and output by directly adding the input to the output of multiple convolutional layers, which helps to optimize the training process and improve the performance of the model.

[0072] Loss Image: Usually in deep learning or computer vision tasks, it is an image representation used to visually display the changing trend of the loss value calculated from the model prediction results. This kind of image can help researchers and developers quickly identify and understand the performance bottlenecks of the model on a specific dataset, and then guide the adjustment of model optimization and training strategies.

[0073] Accuracy Image: An image used to visually display the prediction accuracy of the model in a visualization presentation. It compares the prediction results of the model and encodes the accuracy of each prediction point with different colors or brightness, thus helping researchers and developers quickly identify the performance distribution and potential error patterns of the model, providing strong support for model tuning and improvement.

[0074] ROC curve: (Receiver Operating Characteristic Curve) is a graphical tool for evaluating the performance of classification models. It shows the performance of the classifier under different threshold settings by plotting the curve of the True Positive Rate (TPR) against the False Positive Rate (FPR). The Area Under the Curve (AUC) of the ROC curve provides a quantitative measure of the overall performance of the model. The larger the AUC value, the better the model performance.

[0075] As Figure 1 shown, the embodiments of the present invention provide a sputum image evaluation method, and the steps included are as follows:

[0076] S100: Collect sputum images; perform image preprocessing on the sputum images to obtain preprocessed sputum images.

[0077] Collect sputum images of patients and perform image preprocessing on them to obtain sputum images that are convenient for the image processing model to calculate.

[0078] Specifically, performing image preprocessing on the sputum images includes:

[0079] S110: Annotate the sputum image based on the sputum sample information to obtain an annotated sputum image.

[0080] The sputum sample information includes detailed clinical information records of each sample by clinicians and laboratory technicians; the clinical information records include the patient's basic information, disease description, pathogen detection results, etc. Annotate the collected sputum image based on the sputum sample information so that the image processing model can obtain features related to multi-drug resistant bacteria from it.

[0081] S120: Perform data augmentation on the annotated sputum image to obtain a preprocessed sputum image; the data augmentation includes image flipping, horizontal mirroring, and translation transformation.

[0082] To avoid overfitting, the present invention uses 30 enhancement methods to perform data augmentation and expansion on the training data; the data augmentation methods include but are not limited to image flipping, horizontal mirroring, and translation transformation.

[0083] S200: Input the preprocessed sputum image into the trained image processing model for feature extraction to obtain sputum image features.

[0084] The present invention extracts features from the preprocessed sputum image through the trained image processing model to obtain multi-dimensional sputum image features for subsequent analysis.

[0085] Optionally, the image processing model in the embodiments of the present invention can select the residual neural network architecture in the deep learning model.

[0086] Specifically, the training method of the image processing model can be implemented by the following method:

[0087] S210: Obtain a sputum image sample set and a feature extraction result sample set; the sputum image sample set includes a training sample set; the extraction result sample set includes a training result sample set.

[0088] Through random sampling, sample the sputum image sample set and the feature result sample set according to a self-set ratio (generally 80%), and use the sampled samples as the training sample set and the training result sample set.

[0089] S220: Define a gradient calculation method based on a preset loss function and preset parameters of the image processing pre-model.

[0090] Optionally, the preset parameters include but are not limited to weights, input layer parameters, hidden layer parameters, output layer parameters, activation function parameters, optimizer parameters, and regularization parameters, etc.

[0091] S230: Randomly select a training sample and its corresponding training result sample based on the training sample set and the training result sample set, and input the training sample into the image processing pre-model for calculation to obtain a training result.

[0092] Specifically, a training sample includes a single sample or a mini-batch sample. Each time the model is trained, only one sample or a mini-batch sample is needed, with fast calculation speed and suitability for large-scale datasets.

[0093] S240: Calculate the training sample and the training result according to the gradient calculation method to obtain a gradient value.

[0094] Specifically, in the embodiments of the present invention, the training of the image processing model adopts the stochastic gradient descent algorithm. By calculating the objective function (such as the loss function) for the randomly selected training sample and the training result, the gradient of the image processing model is obtained, and then the parameters of the model are updated and adjusted using the gradient.

[0095] S250: Calculate the training result and the corresponding training result sample according to the preset loss function to obtain a training loss value.

[0096] Calculate the training result and the corresponding training result sample using the preset loss function to obtain the training loss value as a standard to evaluate the training of the image processing model.

[0097] Optionally, the preset loss function includes but is not limited to the mean squared error loss function, the absolute error loss function, the logarithmic loss function, and the cross-entropy loss function, etc.

[0098] S260: Perform a first update on the preset parameters of the image processing pre-model based on the gradient value, determine the updated image processing pre-model based on the first update result; re-obtain the gradient value and the training loss value until the updated image processing pre-model meets the preset conditions to obtain the trained image processing model.

[0099] After a single training of the image processing model by the stochastic gradient descent algorithm, update the model according to the result of the single training, and determine the image processing model updated by this training according to the update result. Further, training on the adjustment of hyperparameters can also be performed on the image processing model updated by this training, which is specifically implemented as follows:

[0100] S261: Determine the image processing pre-model after the first update according to the update result.

[0101] S262: Randomly select a validation sample and its corresponding validation result sample based on the validation sample set and the validation result sample set.

[0102] The validation sample set and the validation result sample set are independent of the above-mentioned training sample set and training result sample set, and are also extracted from the sputum image sample set and the feature extraction result sample set according to a certain ratio (100% - the ratio of the extracted training samples).

[0103] S263: Input the validation sample into the first updated image processing pre-model for calculation to obtain the validation result.

[0104] S264: Calculate the validation result and the corresponding validation result sample according to the preset loss function to obtain the validation loss value.

[0105] Optionally, the preset loss function includes but is not limited to the mean square error loss function, the absolute error loss function, the logarithmic loss function, the cross-entropy loss function, etc.

[0106] S265: Perform a second update on the preset hyperparameters of the first updated image processing pre-model based on the validation loss value, and determine the updated image processing pre-model according to the second update result.

[0107] In the embodiments of the present invention, the training of the image processing model for hyperparameter adjustment is to calculate according to the training result of the validation sample and the validation result sample to obtain the validation loss value, and fine-tune the hyperparameters of the image processing model after a single training according to the validation loss value to update the image processing model after a single training again.

[0108] Specifically, as Figure 2 shown Figure 2 is the loss image curve and the accuracy graph curve drawn according to the model training process of the embodiments of the present invention. Figure 2 In FIG. a in Figure 2 is the loss image (loss) curve of the training group and the validation group during the model training process. The abscissa represents the training Epoch. In model training, Epoch refers to the process of traversing the entire data set completely once, and the ordinate represents the loss change of training and validation. The smaller the value, the better the model performance.

[0109] Furthermore, as Figure 3 shown Figure 3 is the ROC curve drawn according to the model training process of the embodiments of the present invention. Figure 3It includes the ROC curves of the training group and the validation group during the model training process. The abscissa represents specificity (100% - specificity%), and the ordinate represents sensitivity (specificity%). The larger the area under the ROC curve, the better the model accuracy.

[0110] S300: Perform dimensionality reduction processing and linear transformation processing on the sputum image features to obtain the sputum image classification result.

[0111] By performing dimensionality reduction processing and linear transformation processing on the sputum image features, the sputum image features are converted into a one-dimensional sputum image classification result.

[0112] Specifically, the process of performing dimensionality reduction processing and linear transformation processing on the sputum image features can be achieved through the following method:

[0113] S310: Perform dimensionality reduction on the sputum image features multiple times to form one-dimensional sputum image features.

[0114] Specifically, the extracted sputum image features are integrated and converted into a one-dimensional vector, thereby realizing an end-to-end learning process. This integration helps to reduce the influence of feature positions on the classification result and improve the robustness of the neural network.

[0115] S320: Classify the element values on the one-dimensional sputum image features to obtain element values of several categories.

[0116] S330: Multiply the element values corresponding to several categories on the one-dimensional sputum image features by weights and add a bias term to obtain logical values corresponding to several categories, and obtain the sputum image classification result according to the logical values corresponding to several categories.

[0117] Specifically, by performing a linear transformation on the obtained one-dimensional sputum image features, adjusting the weights and introducing a bias term can extract and learn useful features from the input data and map these features to a new space suitable for specific tasks (such as classification or regression).

[0118] S400: Convert the sputum image classification result into a corresponding predicted probability distribution based on a preset algorithm, and determine the evaluation result corresponding to the sputum image based on the predicted probability distribution.

[0119] In the embodiment of the present invention, by converting the result of the image processing model into a corresponding predicted probability distribution, the sputum image can be evaluated intuitively and visually, and then whether the respiratory tract MDRO occurs can be predicted quickly and accurately.

[0120] Specifically, the process of converting the sputum image classification result into a corresponding predicted probability distribution based on a preset algorithm can be achieved through the following method:

[0121] Perform exponential operations on the logical values corresponding to several categories to obtain the exponential values corresponding to several categories;

[0122] Perform quotient calculations on the exponential values corresponding to several categories, and convert the exponential values corresponding to several categories into the predicted probability distributions corresponding to several categories according to the quotient calculation results.

[0123] Further, the above calculation method for converting the classification results into the corresponding predicted probability distributions can be calculated using the softmax function. Specifically:

[0124]

[0125] where S i is the converted predicted probability distribution, e i is the exponential value corresponding to several categories, ∑ j e j is the sum of the exponential values corresponding to several categories.

[0126] Specifically, by converting the scores into probabilities, the output values in the multi-classification model can be probabilities, which is more conducive to backward derivation and model iteration. The distance between probabilities can be calculated better, while the distance calculated between numerical values is meaningless. Using softmax is to use e as the base for exponential operations to achieve the conversion, making the difference in probabilities between two classifications with close scores larger; the exponential makes the final probability of the classification with a large score larger, and the final probability of the classification with a small score smaller, and the classification with a negative score can be almost ignored.

[0127] Further, the process of determining the evaluation result corresponding to the sputum image based on the predicted probability distribution can be achieved through the following method:

[0128] Calculate according to the actual sputum sample data to obtain the true probability distributions corresponding to several categories;

[0129] Calculate according to the predicted probability distributions corresponding to several categories and the true probability distributions corresponding to several categories, and determine the evaluation result corresponding to the sputum image according to the calculation results.

[0130] Measure the accuracy of the model output by measuring the difference between the predicted distribution and the true distribution, and correct the predicted probability distribution to determine the evaluation result corresponding to the sputum image.

[0131] Specifically, the process of calculating according to the predicted probability distributions corresponding to several categories and the true probability distributions corresponding to several categories can be calculated by the cross-entropy formula:

[0132]

[0133] Where S is the cross-entropy result, N is the number of categories, and y i is the true probability distribution, and is the predicted probability distribution.

[0134] Implementing the embodiments of the present invention includes the following beneficial effects:

[0135] This embodiment provides a sputum image evaluation method, system, device, and storage medium. Through a series of processes such as image preprocessing, feature extraction, dimensionality reduction, and linear transformation on the collected images, the classification result of the sputum image is obtained. Finally, the classification result of the sputum image is converted into the corresponding predicted probability distribution for evaluation. Using the trained image processing model, the feature information of the sputum image can be quickly and effectively extracted, improving the efficiency and accuracy of the image processing model. Further, converting the result of the image processing model into the corresponding predicted probability distribution can visually and visually evaluate the sputum image, and then quickly and accurately predict whether respiratory MDRO occurs.

[0136] As Figure 4 shown, the embodiments of the present invention also provide another sputum image evaluation method. By collecting sputum image data and performing sufficient image preprocessing, a prediction model based on deep learning (ResNet-50, a 50-layer residual network architecture) is constructed. This model uses a deep learning model to extract the features of the sputum image, and performs classification prediction through a global pooling layer AVG pool (i.e., Global Average Pooling, GAP), a flattening layer (flatten layer), and a fully connected layer (FullyConnected Layer, FC layer). Finally, the model is optimized through methods such as cross-validation and parameter tuning, and its performance is evaluated.

[0137] Specifically, Figure 4The design and process of the sputum image evaluation method shown are as follows: (1) First, collect images of fresh sputum specimens, with the time controlled within 10 minutes; (2) Use matlab to label the collected sputum images, and at the same time perform data augmentation and supplementation on the labeled images; (3) Input the preprocessed images into the Resnet50 model for training and fine-tuning. The model architecture ResNet50 consists of five basic modules, including Conv2D + Batch Norm + ReLu + Maxpool, stage 2, stage 3, stage 4, and stage 5 (the functions of stage2 to 5 are similar to those of stage1), which jointly participate in the key process of feature extraction, and use the sigmoid activation function to convert the output into a probability distribution; (4) Use the global pooling layer, flattening layer, and fully connected layer to classify and output "MDRO (YES / NO)"; (5) Verify the evaluation results of the sputum images.

[0138] As Figure 5 shown, an embodiment of the present invention also provides a sputum image evaluation system, including:

[0139] The first module is used to collect sputum images; perform image preprocessing on the sputum images to obtain preprocessed sputum images;

[0140] The second module is used to input the preprocessed sputum images into a trained image processing model for feature extraction to obtain sputum image features;

[0141] The third module is used to perform dimensionality reduction and linear transformation processing on the sputum image features to obtain sputum image classification results;

[0142] The fourth module is used to convert the sputum image classification results into corresponding predicted probability distributions based on a preset algorithm, and determine the evaluation results corresponding to the sputum images based on the predicted probability distributions.

[0143] It can be seen that the content in the above method embodiments is applicable to the system embodiments of the present invention. The functions specifically implemented by the system embodiments of the present invention are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those of the above method embodiments.

[0144] As Figure 6 shown, an embodiment of the present invention also provides a sputum image evaluation device, including:

[0145] At least one processor;

[0146] At least one memory for storing at least one program;

[0147] When the at least one program is executed by the at least one processor, the at least one processor implements the method steps described in the above method embodiments.

[0148] Among them, as a non-transitory computer-readable storage medium, the memory can be used to store non-transitory software programs and non-transitory computer-executable programs. The memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some embodiments, the memory optionally includes a remote memory remotely disposed relative to the processor, and these remote memories can be connected to the processor through a network. Examples of the above networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0149] It can be seen that the content in the above method embodiments is applicable to the device embodiments of the present invention. The functions specifically implemented by the device embodiments of the present invention are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those of the above method embodiments.

[0150] In addition, the embodiments of the present application also disclose a computer program product or a computer program. The computer program product or the computer program is stored in a computer-readable storage medium. The processor of the computer device can read the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device executes the above method.

[0151] The embodiments of the present invention also provide a computer-readable storage medium. The computer-readable storage medium stores a program executable by a processor. The program executable by the processor is used to implement the above method when executed by the processor. Similarly, the content in the above method embodiments is applicable to the storage medium embodiments of the present invention. The functions specifically implemented by the storage medium embodiments of the present invention are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those of the above method embodiments.

[0152] It will be understood that all or some of the steps and systems disclosed in the above methods may be implemented as software, firmware, hardware, and appropriate combinations thereof. Some or all of the physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or transitory medium). As is well known to those of ordinary skill in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those of ordinary skill in the art that communication media typically embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0153] The above is a specific description of the preferred embodiments of the present invention. However, the present invention is not limited to the described embodiments. Those skilled in the art can make various equivalent deformations or substitutions without departing from the spirit of the present invention, and these equivalent deformations or substitutions are all included within the scope defined by the claims of this application.

Claims

1. A sputum image evaluation method, characterized in that: include: Collect sputum images; Performing image preprocessing on the sputum image to obtain a preprocessed sputum image; Inputting the pre-processed sputum image into a trained image processing model for feature extraction to obtain sputum image features; Performing dimension reduction processing and linear transformation processing on the sputum image features to obtain a sputum image classification result; The sputum image classification result is converted into a corresponding predicted probability distribution based on a preset algorithm, and an evaluation result corresponding to the sputum image is determined based on the predicted probability distribution.

2. The method according to claim 1, characterized in that The performing image preprocessing on the sputum image comprises: Annotating the sputum image based on the sputum sample information to obtain a sputum annotated image; Data enhancement is performed on the sputum annotated image to obtain a preprocessed sputum image; the data enhancement includes image flipping, horizontal mirroring and translation transformation.

3. The method according to claim 1, characterized in that The training method of the image processing model is implemented by the following method: Acquire a sputum image sample set and a feature extraction result sample set; the sputum image sample set includes a training sample set; the extraction result sample set includes a training result sample set; Define a gradient calculation method based on a preset loss function and preset parameters of an image processing pre-model; Based on the training sample set and the training result sample set, randomly select a training sample and a corresponding training result sample, and input the training sample into the image processing pre-model for calculation to obtain a training result; Calculate the training sample and the training result according to the gradient calculation method to obtain a gradient value; Calculating the training result and the corresponding training result sample according to the preset loss function to obtain a training loss value; Based on the gradient value, the preset parameters of the image processing pre-model are first updated, and the updated image processing pre-model is determined based on the first update result; the gradient value and the training loss value are reacquired until the updated image processing pre-model meets the preset conditions, thereby obtaining a trained image processing model.

4. The method according to claim 3, characterized in that The sputum image sample set also includes a verification sample set; the extraction result sample set also includes a verification result sample set; and the method of determining the updated image processing pre-model based on the first update result also includes: Determine a first updated image processing pre-model according to the update result; Based on the verification sample set and the verification result sample set, randomly select a verification sample and a corresponding verification result sample; Inputting the verification sample into the first updated image processing pre-model for calculation to obtain a verification result; Calculate the verification result and the corresponding verification result sample according to the preset loss function to obtain a verification loss value; A second update is performed on the preset hyperparameters of the first updated image processing pre-model based on the verification loss value, and an updated image processing pre-model is determined according to the second update result.

5. The method according to claim 1, characterized in that The sputum image features are subjected to dimensionality reduction processing and linear transformation processing to obtain a sputum image classification result, and the sputum image classification result is obtained, including: Performing multiple dimensionality reduction on the sputum image features to form one-dimensional sputum image features; Classifying the element values ​​on the one-dimensional sputum image features to obtain element values ​​of several categories; The element values ​​corresponding to the several categories on the one-dimensional sputum image feature are weighted and multiplied and a bias term is added to obtain the logical values ​​corresponding to the several categories, and the sputum image classification result is obtained according to the logical values ​​corresponding to the several categories.

6. The method according to claim 5, characterized in that The converting of the sputum image classification result into a corresponding predicted probability distribution based on a preset algorithm comprises: Performing exponential operations based on the logical values ​​corresponding to the categories to obtain exponential values ​​corresponding to the categories; A quotient calculation is performed based on the index values ​​corresponding to the several categories, and the index values ​​corresponding to the several categories are converted into predicted probability distributions corresponding to the several categories according to the quotient calculation result.

7. The method according to claim 6, characterized in that The determining, based on the predicted probability distribution, an evaluation result corresponding to the sputum image comprises: Calculate according to the actual sputum sample data to obtain the true probability distribution corresponding to the several categories; Calculation is performed based on the predicted probability distributions corresponding to the several categories and the true probability distributions corresponding to the several categories, and the evaluation result corresponding to the sputum image is determined based on the calculation result.

8. A sputum image evaluation system, characterized in that: include: The first module is used to collect sputum images; Performing image preprocessing on the sputum image to obtain a preprocessed sputum image; The second module is used to input the pre-processed sputum image into a trained image processing model for feature extraction to obtain sputum image features; The third module is used to perform dimension reduction and linear transformation processing on the sputum image features to obtain a sputum image classification result; The fourth module is used to convert the sputum image classification result into a corresponding predicted probability distribution based on a preset algorithm, and determine the evaluation result corresponding to the sputum image based on the predicted probability distribution.

9. A sputum image evaluation device, characterized in that: include: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a program executable by a processor, characterized in that: The processor-executable program is used to perform the method according to any one of claims 1 to 7 when executed by the processor.

Images